Table Tanks — training progress

generated 2026-08-25 15:01 UTC · git d1a29d6 · 42 recorded matches · 19 training runs
best measured
+338
v25-best_0060-s50 vs ab-d3
runs
19
2 stated a falsifier first
human-tested
0
the goal is beating humans
open questions
7
tracked in ONBOARDING §2.8
How to read this. Every Elo carries a 95% interval. An interval that includes zero shows no difference, however suggestive the win rate looks — those are drawn hollow and grey. This project spent months promoting on a 20-game gate with a 41% false-pass rate, so confidence is shown, never implied.

Model pipeline

Six stages, computed from the ledger — never hand-maintained. * marks a run whose record was reconstructed afterwards from its log; those runs never stated a falsifier, so Proposed stays empty for all of them.

runProposedTrainedDiagnosedAnchoredRankedHuman-testediterspromosverdict
v28✓––––––/––
v27✓✓✓✓✓–40/401REGRESSED against its own baseline: -93 Elo [-117, -71].
v25 *–✓✓✓✓–100/10012RECONSTRUCTED — 100 iterations, 12 promotions, all gated on 20 games (41% false-pass). No compar
v24 *–✓✓✓✓–80/8010RECONSTRUCTED — 80 iterations, 10 promotions, all gated on 20 games (41% false-pass). No compari
v23 *–✓✓–––67/701RECONSTRUCTED — 67 iterations, 1 promotions, all gated on 20 games (41% false-pass). No comparis
v22 *––✓–✓–80/1108RECONSTRUCTED — incomplete (80/110 iterations). No baseline comparison was ever run.
v20 *–✓✓✓✓–54/605RECONSTRUCTED — 54 iterations, 5 promotions, all gated on 20 games (41% false-pass). No comparis
v20cont *––✓✓––26/802RECONSTRUCTED — incomplete (26/80 iterations). No baseline comparison was ever run.
v19 *––✓–––20/502RECONSTRUCTED — incomplete (20/50 iterations). No baseline comparison was ever run.
v18 *––✓–––38/503RECONSTRUCTED — incomplete (38/50 iterations). No baseline comparison was ever run.
v17d *––✓–––42/500RECONSTRUCTED — incomplete (42/50 iterations). No baseline comparison was ever run.
v17b *––✓–––1/500RECONSTRUCTED — incomplete (1/50 iterations). No baseline comparison was ever run.
v17 *––✓–––12/501RECONSTRUCTED — incomplete (12/50 iterations). No baseline comparison was ever run.
v17c *––✓–––16/502RECONSTRUCTED — incomplete (16/50 iterations). No baseline comparison was ever run.
v16 *––✓–––46/1007RECONSTRUCTED — incomplete (46/100 iterations). No baseline comparison was ever run.
v15 *––✓–––31/501RECONSTRUCTED — incomplete (31/50 iterations). No baseline comparison was ever run.
v14 *–✓✓–––49/508RECONSTRUCTED — 49 iterations, 8 promotions, all gated on 20 games (41% false-pass). No comparis
v13 *––✓–––33/505RECONSTRUCTED — incomplete (33/50 iterations). No baseline comparison was ever run.
v12 *––✓–––32/505RECONSTRUCTED — incomplete (32/50 iterations). No baseline comparison was ever run.

Ladder — Elo against ab-d3

Everything quoted against one fixed, deterministic anchor, so the rows are comparable. ab-d3 is reproducible forever; a checkpoint is not.

ab-d3 = 0v25-best_0060-s50v25-best_0060-s50: +338 Elo [+174, +676] over 24 games+338v20-best_0038-s50v20-best_0038-s50: +154 Elo [+31, +333] over 24 games+154best_0038_gnnbest_0038_gnn: +147 Elo [+65, +249] over 40 games+147v24_best_0030_gnnv24_best_0030_gnn: +147 Elo [+65, +249] over 40 games+147best_0084_gnnbest_0084_gnn: +108 Elo [+6, +232] over 20 games+108v27-best_0035-s50v27-best_0035-s50: +70 Elo [+48, +92] over 800 games+70best_0030_gnnbest_0030_gnn: +53 Elo [-35, +148] over 40 games — interval includes 0+53ab-d5ab-d5: +44 Elo [-104, +210] over 16 games — interval includes 0+44ab-1.0sab-1.0s: +0 Elo [-48, +48] over 40 games — interval includes 0+0ab-d4ab-d4: -0 Elo [-118, +118] over 24 games — interval includes 0-0best_0015_gnnbest_0015_gnn: -241 Elo [-645, -66] over 20 games-241best_0010_gnnbest_0010_gnn: -301 Elo [-645, -154] over 20 games-301best_0001_gnnbest_0001_gnn: -676 Elo [-676, -428] over 24 games-676best_0005_gnnbest_0005_gnn: -676 Elo [-676, -428] over 24 games-676rulerule: -763 Elo [-763, -520] over 40 games-763
interval excludes 0 — a real differenceinterval includes 0 — no difference shown

v20 learning curve

The first within-run curve measured against a fixed anchor — every historical eval used a moving opponent. ~820 Elo of real learning, then flat: 46 further iterations in v20cont added nothing.

-800-600-400-200+0+200+400iter 1: -676 Elo [-676, -428], 24 games1iter 5: -676 Elo [-676, -428], 24 games5iter 10: -301 Elo [-645, -154], 20 games10iter 15: -241 Elo [-645, -66], 20 games15iter 30: +70 Elo [-17, +168], 20 games30iter 38: +154 Elo [+31, +333], 24 games38iter 84: +108 Elo [+6, +232], 20 games84training iteration

Open questions

  1. Is the value head *calibrated*, or merely simple? §2.7 shows it is nearly a formula. A simple value function is fine if it is *accurate*. Test: record positions plus the eventual game result, and correlate the teacher's value against the actual outcome. I
  2. Which head does MCTS actually rely on? Ablate: real policy + constant value, versus uniform policy prior + real value. Decides what the distillation target should be (D-05 assumes both).
  3. Residual first-player advantage. Play a strong agent against *itself* and count P1 wins across raw games. Months went into komi; nothing has checked the result at the current setting. ~50% means balanced.
  4. Does a bigger net actually fit the policy target better? The direct test of D-07. Needs the edge→action mapping (or a policy-space workaround) before it can be run in the CNN action space.
  5. How far does sim scaling go? If strength keeps climbing past sims=200, the on-device budget is worth more than assumed.
  6. Human benchmark — still zero rows. The goal is "beat every human" and no agent has ever played one. Everything is calibrated to a ladder whose relationship to the actual objective is unknown.
  7. Do Swift and Python still diverge? They implement different rulesets (§2.2). Nothing ships until a differential fuzzer proves parity.

Recent decisions

D-13 — Capacity is worth +40 Elo, and the seed control that licenses saying so (2026-08-25)
D-12 — D-11 is retracted: its statistic cannot adjudicate D-08 (2026-08-22)
WITHDRAWN D-11 — RETRACTED by D-12 (2026-08-22). Read D-12 first. —
D-10 — The promotion gate is a paired-Elo interval, not a win rate (2026-08-19)
D-09 — CONFIRMED: v25 broke the plateau, and nobody knew (2026-08-19)
v25's checkpoints existed **only on the iMac**, so it was never in any round-robin. Once synced: | measurement | result | |---|---| | v20 `best_0038` vs `ab-d3` | **+154** [+31, +333] | | v25 `best_0060` vs `ab-d3` | **+338** [+174, +676] | | **v25 vs v20, head-to-head (12 pairs)** | +120 [+7, +267] | | **v25 vs v20, C