Reference · The Applied Higher-Order Functions catalogue

rainbow.algorithms.proofs.ahof.gradient-boosting

The Applied Higher-Order Functions cataloguesrc/rainbow/algorithms/proofs/ahof/gradient_boosting.clj1 definitions, 2 theorems

AHoF Gradient Boosting (GBM / the core of XGBoost, LightGBM): build a strong predictor as an additive ensemble, where each stage fits a weak learner (usually a shallow tree) to the negative gradient of the loss (the residuals) and adds a SHRUNKEN version of it to the running model: boost-update current learning-rate weak = current + learning-rate * weak The learning rate (shrinkage) < 1 regularizes, trading more boosting rounds for better generalization. Proved native and axiom-free: the additive update; a worked boosting step; and that at learning rate 0 the model is unchanged (no learning) -- reducing by reflexivity. The weak-learner fit and the residual / gradient computation are executable; what is proved is the shrunken additive update. Requires nnrat/add, nnrat/multiply (both in the D63 checkpoint). Scope (measured by the kernel): this module has 2 worked instances (statements about specific values). None of its theorems is a guarantee about the algorithm for all inputs; `lein verify rainbow.algorithms.proofs.ahof.gradient-boosting` prints this classification.

gradient-boosting/update :
  (x₁ x₂ x₃ : ℚ≥0)
  → ℚ≥0

gradient-boosting/boost-step-example :
  → gae54/gae-advantage 2 1/2 1 = 5/2

gradient-boosting/zero-rate-unchanged :
  → gae54/gae-advantage 3 0 9 = 3