Reference · The Applied Higher-Order Functions catalogue

rainbow.algorithms.proofs.ahof.gini-impurity

The Applied Higher-Order Functions cataloguesrc/rainbow/algorithms/proofs/ahof/gini_impurity.clj1 definitions, 3 theorems

AHoF Gini Impurity: the node-purity criterion used by CART decision trees (and a cheaper alternative to entropy). For a node with class proportions p_i, the Gini impurity is 1 - sum p_i^2 -- the probability that two randomly drawn labels differ; it is 0 for a pure node (one class) and maximal for a uniform mix. A split is chosen to MINIMIZE the weighted child impurity, so comparing two candidate splits reduces to comparing their impurities: purer-split? impurity-a impurity-b = decide (impurity-a <= impurity-b) (the lower-impurity split is the purer, better one), over exact-rational impurities. Proved native and axiom-free: the purity-comparison test; that the lower-impurity split is preferred; the certificate that a chosen split's impurity is at most the alternative's (native nnrat/at-most) -- greedy purity maximization; and that a higher-impurity split is not preferred. The 1 - sum p_i^2 computation is executable; what is proved is the purer-split comparison. Requires nnrat/at-most, audit/decide-nnrat-at-most, nnrat/decide-at-most-true-sound (all in the D63 checkpoint). Scope (measured by the kernel): this module has 3 worked instances (statements about specific values). None of its theorems is a guarantee about the algorithm for all inputs; `lein verify rainbow.algorithms.proofs.ahof.gini-impurity` prints this classification.

gini-impurity/purer-split? :
  (x₁ x₂ : ℚ≥0)
  → data/Bool

gini-impurity/lower-impurity-preferred :
  → ce-is/elite? 1/4 1/2 = data/true

gini-impurity/chosen-split-purer :
  → 1/4 ≤ 1/2

gini-impurity/higher-impurity-not-preferred :
  → ce-is/elite? 1/2 1/4 = data/false