Monarch Chrysalis v1 — a sparse mixture-of-experts (MoE) model with latent-space reasoning. The model's first training run is finished, and the architecture now provably works end to end. Still an early research model, but training is ongoing and the model is improving.
Architecture
Chrysalis is a sparse mixture of experts with latent-reasoning modules built into it. Sixty-four experts per layer, eight active per token, about 6.44B of the 6.93B total parameters living in the expert stack. The latent loop runs on top of all of it: instead of decoding a token at every reasoning step, the model can carry its hidden state forward as continuous internal state and only spend words when it has something to say.
Four latent steps are compiled. A learned halting head decides how many to weight. The base is an openly licensed sparse mixture-of-experts model, extended with our own modules rather than trained from scratch — the point is not capacity. Routing is the part of a sparse model we understand least, and a small model is the honest way to study it. The base model is named on release.
It runs
The step-2126 checkpoint was byte-verified at 13,865,515,469 bytes and runs end to end on a desktop: a Ryzen 7 5700X with 31 GiB of DDR4, entirely on CPU, no GPU involved. Cold load is eighty seconds. After that it is a terminal you can talk to, with the latent trace printing alongside every answer.
That is what the architecture had to demonstrate, and it does. The routing works. The latent loop executes and its halting behaviour is measurable rather than theoretical. The model produces coherent English, and given an instruction rather than a question it follows the instruction and attempts the task.
Halting behaviour
E[N] is the expected number of latent steps used, under the model's own halting distribution. Four slots are compiled; it weights about one. That figure holds across three environments and two independently written mixture-of-experts forward implementations:
TPU training telemetry Kaggle TPU v5e-8, static path 1.023 - 1.070 Kaggle inference Kaggle CPU, static path 1.068 local inference Ryzen 7 5700X, sparse path 1.04 - 1.18
All four slots are always computed. E[N] of about 1.09 does not mean three steps were skipped and the work was saved — it means the model computes four and weights its answer on roughly one. That is a finding about how the halting head has learned to behave so far, at step 2126 of a run that is still going.
What it cannot do yet
The latent channel is not yet carrying the reasoning. By our own instruments it is not: transplant delta is approximately zero, grounding accuracy is flat between 11% and 21%, and the arithmetic that does work arrives in visible English rather than through the latent path. On the questions examined closely, the model's stated answer does not always follow from its own correct working.
That is the open problem and it is the entire point of the project. It is also now something that can be watched happening rather than inferred from telemetry, which is the difference this checkpoint makes. Training continues, and the next dataset is considerably sharper than the one that produced this checkpoint.
Licence and disclaimer
Weights are not released yet. This is an early research checkpoint of a training run still in progress, published because the architecture works end to end and that is worth recording. It is not a product, it is not tuned for use, and it should not be deployed for anything.
Figures on this page come from the training telemetry and from local inference runs. Where a number could not be measured, it is not printed. Anything that changes with the next run will be corrected here.