Monarch Chrysalis 1, a sparse mixture-of-experts (MoE) model with latent-space reasoning. Three training runs are complete and the architecture provably works end to end. Still an early research model. Training is ongoing, and the halting behaviour it was built to produce has not moved across those three runs, see the research log for why.
Architecture
Chrysalis is a sparse mixture of experts with latent-reasoning modules built into it. 64 experts per layer, 8 active per token, about 6.44B of the 6.93B total parameters living in the expert stack. The latent loop runs on top of all of it: instead of decoding a token at every reasoning step, the model can carry its hidden state forward as continuous internal state and only spend words when it has something to say.
Four latent steps are compiled. A learned halting head decides how many to weight. The base is an openly licensed sparse mixture-of-experts model, extended with our own modules rather than trained from scratch, the point is not capacity. Routing is the part of a sparse model we understand least, and a small model is the honest way to study it. The base model is named on release.
It runs
The step-2126 checkpoint was byte-verified at 13,865,515,469 bytes and runs end to end on a desktop: a Ryzen 7 5700X with 31 GiB of DDR4, entirely on CPU, no GPU involved. Cold load is 80 seconds and it decodes at 3.69 tokens per second across the items that generated anything, on 8 physical cores. Averaged over the whole 29-item run including the four items that emitted nothing it reads 3.18, which is throughput rather than decode speed. That is slow, and it is the honest figure: a latent step costs about 93% of what emitting a token costs, since only the vocabulary projection is skipped. Nothing here is faster than writing the reasoning out. After the load it is a terminal you can talk to, with the latent trace printing alongside every answer.
That is what the architecture had to demonstrate, and it does. The routing works. The latent loop executes and its halting behaviour is measurable rather than theoretical. The model produces coherent English, and given an instruction rather than a question it follows the instruction and attempts the task.
Run progress
This section covers the whole training chain, to step 10,877. The detailed figures elsewhere on this page were measured on the step-2126 checkpoint and have not been re-measured. Where the two disagree, this section is the later reading.
Two panels, because two things happened and only one of them is good news. The model got better at predicting text. Its latent thinking did not start carrying the answer. Every point below is the value at the end of a run, read from that run's own instrument; series from different instruments are never merged, and a run that produced no reading gets no point rather than a zero.
The training loss (excluding routing terms) fell from 4.03 nats at the end of the root run to 3.09 nats at the end of run 3, the last run trained on that objective; the later runs changed the objective, so their loss is not comparable and is shown apart. Run 5 is the first run that logs the answer's own cross-entropy, 0.83 nats. The expected number of latent steps the halting head uses started near 1.8 of the four computed, collapsed to 1.02 within that first run, and has stayed there (1.06 at the latest run, where it sits at the floor its halting bound guarantees). On the two runs where the training-time transplant meter produced readings, swapping in another question's thoughts cost under 3% of the answer's cross-entropy. On the offline causal battery the model's own thoughts were indistinguishable from a null vector on every graded checkpoint (run 3, run B, run 5). The training loss fell through run 3; use of the latent thoughts did not improve.
Run 5 changed the training objective on purpose, to push information into the latent slots. It did not widen the channel: the slot-discrimination ratio fell from 0.34 to 0.15, and the causal battery repeated run B. The halting head's expected steps sit on the floor of its bound. A further run is in preparation; nothing from it is a result yet.
Halting behaviour
E[N] is the expected number of latent steps used, under the model's own halting distribution. Four slots are compiled; it weights about one. That figure holds across three environments and two independently written mixture-of-experts forward implementations:
TPU training telemetry Kaggle TPU v5e-8, static path 1.023 - 1.070 Kaggle inference Kaggle CPU, static path 1.068 local inference Ryzen 7 5700X, sparse path 1.04 - 1.18 arithmetic probe Ryzen 7 5700X, 29 items 1.04 - 1.16
All four slots are always computed. E[N] of about 1.09 does not mean three steps were skipped and the work was saved. It means the model computes four and weights its answer on roughly one. That is a finding about how the halting head has learned to behave so far, at step 2126 of a run that is still going.
The fourth row is a different kind of evidence from the first three and is worth separating. It shares hardware, weights and code path with the local inference row, so it is a fourth set of conditions rather than a fourth independent instrument, the cross-machine replication still stands at three. What it adds is that the collapse holds across outcome: the same E[N] on items the model answered correctly, answered wrongly, ran to the token cap on, and said nothing at all to. Prompt phrasing does not move it either. Phrasing changes whether the model talks. It does not change whether it thinks.
Measured behaviour
A 29-item arithmetic probe, with the scoring rules fixed in the source before any output existed, gives the clearest picture of what the checkpoint currently does. The partition matters more than the score:
A boundary added 2026-08-31, after these figures were published. These figures were measured on the step-2126 checkpoint, on the pre-fix recurrence, confirmed from the fetch log, the checkpoint history and a routing-deviation fingerprint in the run log itself, not from memory. Every inference tool in the project. This probe included, carried the prompt's hidden state forward raw between latent steps instead of through the learned update gate the model trains with. The gate's weights were in the checkpoint and loaded by all six tools and called by none of them. The tools have since been corrected and now share a single carry step, so what follows describes the path as it stood when these figures were measured rather than the code as it stands today. The numbers here are real measurements of what that path produced, and they are not measurements of the model as trained. The effect is not small: on a fixed benchmark suite, with the same checkpoint and decode settings and a byte-identical determinism control, correcting the recurrence moved 25 of 30 answers. Everything in this section is being re-measured on the corrected path, and both sets will be published side by side rather than one quietly replacing the other.
attempted 29 emitted nothing at all 4 ran to the token cap, never committed 11 committed an answer 14 -> 3 correct
Of the 11 committed-wrong answers, one emitted a single token with no number in it. Of the remaining 10, six land within 2% of the truth and three of the other four are digit-count failures off by roughly a factor of ten. Carry load splits it cleanly: 3 of 5 right where the carry load is at most two, 0 of 9 where it is four or more. Those are the failure modes of a circuit computing badly, not of a lookup table being read wrongly.
Two behaviours matter more than the accuracy for anyone reading the traces.
The answer channel is faithful to work that is wrong. A prediction registered before the run. That where the model emits a fenced output block, its committed answer will equal that block, held on all 7 rows that could test it. 5 of those 7 blocks were wrong, and the answer transcribed each one exactly. In one case the model wrote Python containing an undefined variable, invented the result of running it, and then cited its own invention as confirmation. A downstream check comparing the answer against the working will find perfect agreement.
Prompt phrasing selects the failure mode. Same operands, one trailing clause between them: a bare question produces silence on 4 of 13 items; adding "verify your answer with Python" produces zero silences and 6 commitments out of 8; adding "state your confidence as a percentage" produces 7 non-terminating runs out of 8. Three distinct pathologies, not a severity gradient. If you are prompting this checkpoint, the phrasing decides which way it fails.
What it cannot do yet
The latent channel is not yet carrying the reasoning. By our own instruments it is not: transplant delta is approximately zero, and the arithmetic that does work arrives in visible English rather than through the latent path. Grounding is instrumented and reads above chance, but a defect found on 2026-08-31 means the figures it has produced so far are not yet a clean measurement, so no number for it is printed here. On the questions examined closely, the model's stated answer does not always follow from its own correct working.
That is the open problem and it is the entire point of the project. It is also now something that can be watched happening rather than inferred from telemetry, which is the difference this checkpoint makes. Training continues. The sharper dataset this page previously promised has since gone in, and it did not move the halting behaviour: E[N] measured on the training telemetry is 1.0152 across two consecutive runs, roughly 6,200 cumulative steps of training between them. That is a null result rather than an absence of one, and it is the honest state of the thing this project exists to test.
Licence and disclaimer
Weights are not released yet. This is an early research checkpoint of a training run still in progress, published because the architecture works end to end and that is worth recording. It is not a product, it is not tuned for use, and it should not be deployed for anything.
Figures on this page come from the training telemetry and from local inference runs. Where a number could not be measured, it is not printed. Anything that changes with the next run will be corrected here.