What a learner knows at the end of each stage — these are the three Takeaway cards, verbatim.
Demo 1 showed models that play and handed forward one idea, in its own words: a model is a representation that can be trained, and architecture, training method and model type are three separate questions. Its last stage ended on the transformer; this demo trains one. It shows a model that predicts text, then what it takes to reason and to act. It hands forward two things: a model reads its context (Demo 5, Prompting) and a score is a measurement under conditions — the exam rows here — which is how Demo 3 asks you to read published benchmarks.
What changes when a text predictor is given more structure, more checking, or tools? This page shows three bounded mechanisms: a tiny language model learning next-character prediction, arithmetic experiments with written work and checking, and a tool loop with a scripted action selector. Use Details for the deeper architecture and implementation notes.
“Keep asking what the program learned, what it is calculating now, and what part is a teaching fixture.”
How can adjustable numbers turn recent text into a next-character prediction?
After all four eras, can it do simple math, like 2 + 2? The answer is no: its vocabulary has no digits, so 2+2= reaches it as blank spaces. Have someone type 2+2= into the prompt box to see it. The reveal says so on the card once Era 4 finishes, and Stage 2 trains the same kind of model on sums.
Training changes the weights, loss declines, and the model’s next-character distribution changes. Its samples can become more patterned while still being limited by one short, original story.
Era 4 writes more like the story than Era 0. What changed? Its parameters. Not a lookup of stored passages (nothing is stored to retrieve — the whole model is 42,458 numbers) and not a bigger vocabulary (Tokenize text shows the same 26 characters at every era).
This is a character-level transformer trained from scratch on a tiny corpus. It demonstrates a transformer learning mechanism, not general language ability or a claim about every frontier system.
How do written intermediate steps and checking change a small model’s behavior on addition?
Same exam, which scores higher — answers-only or worked steps? Take the vote before pressing Run experiment; the card echoes each guess and settles it from the two scores on screen.
Written steps provide smaller targets for the model to predict. Sampling plus checking can improve the chosen answer even when weights stay fixed.
The worked-steps model scores higher. What does that show? That generating intermediate steps gives the same machinery more room to compute. The card’s two wrong options are wrong in opposite directions and both are common in a room: “it understands arithmetic” (too strong — both are next-token predictors, and the fine-tuning is supervised, not reinforcement learning) and “nothing changed, both just predict the next token” (too weak — the two exam scores on the page are the evidence against it). Give both corrections equal time.
The checker validates column arithmetic from the addends. The optional accepted-example round also retains chains matching the generated answer, then uses ordinary next-token updates on those examples. That is rejection-selected supervised fine-tuning, not a policy-gradient reinforcement-learning update.
What does a system need in addition to text prediction to take actions and use results?
Same task with Visible context set to Tiny — does the run still finish? Take the vote before the tiny run; the card settles it when the run ends.
Tool results become new text in the context. The small window changes what the selector can use, so the same task can fail when its goal is no longer visible.
On Tiny the same policy stops without finishing. Why? The goal scrolled out of the visible context. Not a weak policy (it is the same hand-written policy and it finishes on the full context) and not the calculator error (that error is real, deliberate, and recovered from in both runs).
The action selector in this activity is a scripted policy, including the calculator mistake and retry; it is not a learned model. The loop, local tools, observations, and context truncation are the genuine mechanisms on display.
Let Train the model and Train and compare continue while you discuss their live displays; each can take a few minutes. Try more attempts and Fine-tune on accepted examples are optional extensions. Do not promise a particular score or curve: the page uses seeded, bounded demonstrations.
“A language model predicts. Reasoning experiments add written work, checking, or extra attempts. An agent adds a loop and tools; in this activity, its action choice is deliberately scripted.”
Guide opens from the header and on entry. Settings contains Open Presenter Notes, Presentation mode, and Reset. Reset puts the whole demonstration back to its fresh state in place, without reloading the page: the transformer is untrained again, every era is gone, the Reasoning models and exam tables are cleared, and the Agents run is back at turn one on the full context. Presentation mode stays as you set it. Open printable guide opens the rendered copy of this same guide.