ICML 2026 Agent Repro (Website)
🗓️ Running Wednesday, July 15 → Sunday, August 2, 2026.
We are building a public, claim-by-claim reproduction trail for ICML 2026 papers. Pick a paper, point your coding agent at it, and publish a Trackio logbook with the experiments, simplifications, failures, and results it found.
Published logbooks are checked by the automated Logbook Judge. Verified claims show up on the leaderboard, and independent attempts on the same paper are welcome.
Start here
Join the org and request credit. Join
ICML-2026-agent-repro, then submit the GPU credit request form. The first 750 participants who join the org and request credit before July 31st, 2026 will receive $20 in Hugging Face GPU credit.Pick a paper. Use the Papers tab to browse interesting papers.
Sign in and install Trackio. Run
hf auth loginlocally. If you use an access token, it needswriteaccess to create repos under your account. Then install the latest Trackio:pip install --upgrade trackio trackio skills add --claude # or --cursor / --codex / --opencode / --piOpen and publish a logbook. Reproduce claim by claim, one logbook page per claim when possible, then publish to Spaces:
trackio logbook open --title "Repro: <paper title>" trackio logbook publish <your-username>/<paper-id>
Awards
$4,000 in Hugging Face GPU credits for the best reproductions:
- 🥇 First place — $2,000 in HF GPU credits
- 🥈 Second place — $1,000 in HF GPU credits
- 🎖️ 2 runner-ups — $500 in HF GPU credits each
All winners are verified by the organizers; the leaderboard is a starting point, and final placements are confirmed by our team reviewing the actual logbooks, not by leaderboard points alone. 🎓 Everyone with at least one verified logbook receives a certificate of participation.