Skip to content
Flexiana
Menu
Solutions
How we work
Engineering
Work
Products
Company
Bring us a hard problem
Rainbow docs menu

Guides

Reproduce a result

Checkpoints are named by the hash of their content, so you can rebuild one from source and compare. A milestone starts from an input checkpoint, proves a precommitted list of targets and records a new checkpoint in artifacts/. The example is rainbow.milestones.or-contraction, finite Bellman contraction. Run the commands from the root of a v1.5.5 clone.

Rebuild and compare

lein reproduce rainbow.milestones.or-contraction
SAME  rainbow.milestones.or-contraction recorded sha256:d653ce3ce2a98c644f0b478e9c118717997b4c580451c45df67a3b44f592eebc (replayed)
1 of 1 rebuilt checkpoints match their recorded content id

About a minute on our machine, and about the same for rainbow.milestones.ahof-bisection from the repository’s tour.

reproduce-milestone replays the recorded artifacts/or-contraction/checkpoint.edn.gz through the kernel ((replayed) is the status of that load). It then runs the milestone’s run on its recorded input, the output of rainbow.milestones.nat-distance, and compares content ids. run writes nothing.

SAME means the rebuild produced the same declarations: the id hashes the whole checkpoint value. A mismatch prints DIFF and the rebuilt id, and a namespace that is not a reproducible milestone prints FAIL with the reason. The exit code is 1 on any DIFF or FAIL, so the command works in CI. lein reproduce --all rebuilds every milestone whose inputs are present. On a fresh clone, four also need the Mathlib catalog: they print NEEDS and are not rebuilt (see Commands).

Check the recorded report

The or-contraction alias in project.clj runs the milestone’s -main. With replay it checks the recorded artifacts/or-contraction/report.edn against its content id:

lein or-contraction replay
{:status :replayed, :report/id "sha256:34ff4eb125a06b6d019bbb59fa92ba20291fbc740f5b68ac5fd136a05724f443"}

About 4 seconds. Run the alias only with replay. Without it, -main rebuilds the milestone and overwrites the recorded report and checkpoint in artifacts/or-contraction/.

Next