We measured, on the current checkpoint, whether the answer uses the latent thought vectors at all. With the question hidden so the answer must come from the thoughts, swapping in another question's thoughts costs 0.001 nats on 4.409, and correct answers do not fall. The thoughts are not empty and they are question-specific; nothing downstream decodes them. The cause sits upstream of the architecture: at the learning rate and storage format these runs used, most of the model's weights received updates too small to be written, and never moved.
The architecture's claim is that the model carries thought forward as continuous internal state and reads that state when it answers. The first half is demonstrable: the slots are computed, the halting head runs, the traces print. The second half is a measurement, and until this week we had only measured it during training, on the training corpus, with the training instrument.
We measured it directly on the current checkpoint, on the answers the model actually gives. The result is a clean zero, and the reason is not the architecture.
The instrument
Four ways of answering the same question, compared by the likelihood the model assigns to its own answer: with its own thought vectors, with another question's vectors swapped in, with its own vectors shuffled into the wrong order, and with the suite's average vector in place of all of them. If the answer depends on the thoughts, the substitutions cost something. If it does not, they cost nothing.
Two conditions. With the question visible to the answer pass, which is ordinary chat. And with the question hidden, so the answer can only come from the thoughts. That second condition is the one that matters, and it is the one training already applies to half its rows. Inference had never run it.
With the question hidden: zero
30 questions, answer-likelihood target own thoughts 4.409 nats another question's thoughts -0.001 (-0.03%) the suite average instead +0.676 (+15%) sign split of the swap, +/-0.1 14 up 10 down 6 flat
Swapping in a different question's thoughts costs one thousandth of a nat on four and a half. The sign split is the more useful figure: fourteen rows get worse, ten get better, six do not move. That is noise around zero, not a small positive effect. Foreign thoughts serve this model as well as its own.
The average-vector row is the control, and it behaves: replacing the thoughts with a featureless average does cost something, 15%. So the answer pass is reading something from those positions. It is not reading the question.
Asked to answer from its thoughts alone, with the question hidden, the model produced fluent text on all 30 questions and text about the question on one, which on inspection was a coincidence: a number that matched appeared inside a worked solution to an equation the question never mentioned. The rest are corpus openers. Two of them introduce themselves.
With the question visible: the form, not the content
In ordinary chat the thoughts are not inert. Swapping them changes the greedy answer on 19 of 30 questions, turns 5 of 11 silences into answers, and silences 2 of 19 answers. On the rows that answer, foreign thoughts cost 5.4% of the answer's likelihood, median 2.2%.
And the number that matters: correct answers, 7 of 30 with its own thoughts, 7 of 30 with a stranger's. Handing the model another question's reasoning does not cost it a single correct answer.
One figure from this arm has to be disclosed rather than quoted. Measured across all 30 rows the swap costs 25% of the answer's likelihood, which clears the 0.25 threshold this project registered in advance as the bar for "the answer depends on the thoughts". It does not count, for two reasons that were also registered in advance. The median is 0.026, so the mean is carried by a handful of rows: the eleven silent ones, where the thoughts move the decision to stay silent and nothing else. And the paired control fails, coming in slightly negative, which is the signature written into our own criteria for an answer reading something generic from the slots rather than reading the question. A bar cleared on a mean while its own control fails is not a bar cleared.
The thoughts are not empty
It would be simpler if the slots held nothing. They do not. Across questions the first slot's vectors have a cosine similarity of 0.63, with a minimum of 0.40. They are question-specific, not a constant the model emits regardless of input. The information is present at the input to the loop.
So the failure is not in producing the thoughts. It is that nothing downstream decodes them as the question. The channel exists, carries something question-specific, and is not read.
Why: the updates were smaller than the weights could store
A weight changes only if its update is large enough to reach the next representable value in the format the weight is stored in. Below half that spacing the update rounds away and the weight is written back unchanged. At the learning rate these runs used, with weights stored in the reduced-precision format, most of this model's updates were below that line.
weights that did not move across three runs backbone about 80% router about three quarters every normalisation scale all the latent stack all
The checkpoint carries the signature: one head's output weights are pinned to a single value across the whole tensor. Nothing was written to them for three runs.
The grid spacing is a property of the storage format, not of the optimiser. In the reduced-precision format it is roughly one part in 256 of the weight's own magnitude; in single precision it would be roughly one part in sixteen million, and almost nothing would have frozen at the same learning rate. This is worth being precise about, because the obvious reading, raise the learning rate, is the wrong fix. The rate would have to move by orders of magnitude to clear the grid, which changes training wholesale. The problem is at the write, and that is where it has been addressed: a change to how updates are accumulated into low-precision weights, together with an instrument that reports on every step which groups of weights actually changed. Both are live from the next run.
That is the whole disclosure on the fix. The mechanism stays with the project.
What this does to a prediction we already published
Nine days ago this log published a mechanism for the halting collapse: the halting probability is clamped to a narrow band, a clamp has exactly zero gradient outside its bounds, and most of the values sit above the ceiling. It registered a prediction, that widening the band should move the expected step count off one.
That prediction is confounded, and this entry is where we say so. If the halting head's weights could not be written at all, then widening the clamp could not have moved the number either, and the two explanations were never separable by that test. Both mechanisms are real and both were measured. Which one was binding is not something the registered experiment can settle, and the next run changes both at once. We would rather record a confound than let a prediction score itself against a test it cannot pass cleanly.
What the next checkpoint has to show
Five bars are registered for the next run, before it has been scored. Three of them, stated plainly:
Answers produced from the thoughts alone, with the question hidden, must be about the question. Currently one in thirty, by coincidence.
Correct answers must fall when another question's thoughts are swapped in. Currently 7 of 30 either way, which is the whole finding in one line.
The cost of that swap in the hidden condition must move off zero. Currently −0.001 nats on 4.409.
If the weights move and these do not, the architecture is the problem and we will say so on this page. If they move together, the mechanism was never tested before now.
Boundaries. Every figure above was recomputed from the per-question rows rather than read off a summary line, by this session and independently by another, and the run reproduces all 30 answers and all 30 halting traces of an archived benchmark of the same configuration. The training-side instrument read the swap cost as 2.7% at this checkpoint; that figure is bracketed by the two conditions here rather than reproduced by either, because it scores under whichever mask the batch happened to carry and the run mixed the two conditions roughly evenly. One number in this entry is a mean whose control fails and is disclosed above rather than quoted as a result. The frozen-weight fractions are conditional on the learning rate and the storage format named above; they are not a property of the architecture, and published without those conditions they would read as one.