Tonight I retrained my writer model twice on my Mac Studio. The first run was routine, but the second run trained on 36 new examples written against my style gate's measured failures. The model drafting this post is that second model, running on an 18GB M3 Pro MacBook. Its average on my own 12-brief bench went from 88.8 to 97.0 in one night, and all 12 briefs passed at the end.
The gate told me what was wrong
Across the evening's bench runs, 10 of the 16 gate failures tripped one check, rhythm uniformity. When I read the failing texts they were repetition loops, one sentence structure restated with small changes. My training corpus was 65 percent rewrite pairs and held only 30 short-form examples. The failures clustered in short formats, emails and notes and README sections.
So the fix was new training examples that demonstrate varied sentence rhythm in exactly those short formats. I have not verified whether this holds beyond this single night, but the numbers are what I saw.
Two fixes, measured separately
The gate told me what was wrong. I wrote 36 new short-form training examples against those measured failures, each scoring 100 on the deterministic gate before entering the corpus. I excluded bench topics so the eval stayed honest, growing the corpus from 287 to 323 examples.
The second fix was a repetition penalty of 1.05 at serving time, tested as its own A/B on the finished model. The 95.0 average came with the penalty off, leaving 11 of 12 briefs passing. The 97.0 average came with the penalty on, clearing all 12 briefs.
Keeping the two changes separate meant each earned its own number.
The seat rule kept it honest
My rule is that a challenger takes the production seat only by winning both the deterministic style score and a blind pairwise judge, and a challenger that loses gets its weights deleted. Version 2 won both tests against version 1, style 90.7 to 88.8 and judge 4 to 2, and took the seat.
Version 3, trained on the 36 new examples, then won both tests against version 2, style 93.2 to 90.7 and judge 8 to 3 with one tie, and took the seat from it. The judge was a 27B model shown each pair of texts blind, order randomized. Rhythm failures dropped from 4 to 0.
What broke on the way
The gate reported a score of 88.8 before I retrained and 97.0 after, yet two bench runs at temperature 0.7 returned byte-identical outputs, an anomaly since two different models cannot produce identical strings by chance.
The serving library had silently dropped the adapter weights during a recent upgrade, so the fallback writer lane served the bare base model with no error; I fused the adapter into standalone weights to resolve the mismatch.
Separately, the v3 training run died at iteration 600 of 800 while reporting a success exit code. The check that flagged failure belonged to a grep command at the end of a pipeline rather than to the trainer itself. Checkpoint 400 sat on disk and represented the validation minimum for the third run in a row, so I restored from that save point once I noticed the missing checkpoints.
Shipping proceeded with the fused weights and the restored checkpoint, but one live sample still scored 76 on the gate. A 12-brief average is a statistic while one sample is one data point, so the production prose still goes through a gate-and-revise loop.