Skip to content
Flexiana
Menu
Solutions
How we work
Research
Work
Products
Company
Bring us a hard problem
Rainbow docs menu

Guides

Reproduce a result

Checkpoints are named by the hash of their content, so you can rebuild one from source and compare. A milestone starts from an input checkpoint, proves a precommitted list of targets and records a new checkpoint in artifacts/. The example is rainbow.milestones.or-contraction, finite Bellman contraction. Run the commands from the root of a v1.4.1 clone.

Rebuild and compare

lein reproduce rainbow.milestones.or-contraction
SAME  rainbow.milestones.or-contraction recorded sha256:7a182cfd5528ab322e12014c06e0a3b6411f7c315aaec49504603b762077298d (replayed)
1 of 1 rebuilt checkpoints match their recorded content id

About 2 minutes 15 seconds on our machine; rainbow.milestones.ahof-bisection, from the repository’s tour, took about 3 minutes 10.

reproduce-milestone replays the recorded artifacts/or-contraction/checkpoint.edn.gz through the kernel ((replayed) is the status of that load). It then runs the milestone’s run on its recorded input, the output of rainbow.milestones.nat-distance, and compares content ids. run writes nothing.

SAME means the rebuild produced the same declarations: the id hashes the whole checkpoint value. A mismatch prints DIFF and the rebuilt id, and a namespace that is not a reproducible milestone prints FAIL with the reason. The exit code is 0 only when every milestone you named matched, so the command works in CI.

Check the recorded report

The or-contraction alias in project.clj runs the milestone’s -main. With replay it checks the recorded artifacts/or-contraction/report.edn against its content id:

lein or-contraction replay
{:status :replayed, :report/id "sha256:34ff4eb125a06b6d019bbb59fa92ba20291fbc740f5b68ac5fd136a05724f443"}

About 4 seconds. Run the alias only with replay. Without it, -main rebuilds the milestone and overwrites the recorded report and checkpoint in artifacts/or-contraction/.

Next