The piece of agent architecture most builders are skipping: the dreaming step.
Jump to a section
See the story at a glance
Agent benchmarks almost always test a single session. Prompt goes in, output comes out, score is computed. The model never learns anything. It starts fresh every time.
That's not how most real agents should work. If you've built an agent that runs repeatedly on a task — coding work, customer support, research synthesis — it accumulates observations that could make future runs better. Whether it actually uses them is the question. Usually the answer is no, because nobody built the memory layer.
Anthropic's research team published a guide to agent memory on November 6. The taxonomy: five layers. Transcripts at the base, then memory notes, then structured representations, then injected context loaded at session start. And at the top, the layer most builders skip: dreaming — the offline step where the agent reviews its transcripts, corrects its notes, and builds richer structures for the next session.
The numbers from their CL-Bench evaluation: Opus 5.5 goes from a mean reward of 0.655 to 0.838 with memory. That's a 28% absolute lift. And the dreaming step is what separates gains that compound across sessions from gains that plateau after one pass.
Source spread
- Anthropic research — Components of agent memory [academic] — first-party research from the team that built these systems. No independent replication as of publication.
- CL-Bench (Anthropic internal evaluation) [builder] — the benchmark is Anthropic's own design, run on tasks Anthropic chose. The gains are real in that setting; how well they generalize to your workload is an open question.
Pros & cons
What the research actually shows:
- The five-layer taxonomy is the most durable contribution. Transcripts, notes, representations, injected context, dreaming — it's a framework for thinking about what an agent does and doesn't remember across sessions, and it matches what practitioners have discovered independently through trial and error.
- Dreaming is real and it matters. In one Pokémon Red case, correcting a broken note let the agent reach Cerulean City at action 1,935. Without the correction, it never got there. That's not a marginal improvement. That's the difference between a successful run and a failed one.
- The wiki-plus-search result is worth the attention: 61% on the wiki scaling task vs 53% with search alone, saving about 20,000 tokens per question. At scale, that's a meaningful cost difference, not just a performance one.
- The practical rules in the footnotes are specific and immediately usable. Keep injected context small. Assign one agent per wiki page. Don't write concurrent updates to shared files.
What deserves skepticism:
- Almost all results are single-run or small sample. The CL-Bench evaluation is the most robust. The wiki scaling test used 25 questions. The legal merger test used 24 questions. The Pokémon Red results are one run per model, stated explicitly.
- The gains depend heavily on model quality. Sonnet 4.6's improvement with memory is marginal until you add prompt engineering. Older models need explicit steering to use memory correctly.
- The dreaming step requires the agent to run twice per session cycle — once to work, once to review. That's real compute cost. The article doesn't address this directly.
- "9 out of 10 times" in one claim has no stated sample size. That's the article's language; I'm not inventing the caveat.
- Layer 1
Transcripts
Searchable event log — the base layer every other layer builds on
- Layer 2
Memory notes
Salient observations the agent writes during sessions, as tags or files
- Layer 3
Representations
Structured wikis and linked pages built from notes
- Layer 4
Injected context
Small layer loaded into every new session at start
- Layer 5
Dreaming
Offline step correcting notes against transcripts and building representations
Samwise's take
What builders need to know
- The taxonomy is in the full research post and worth reading in full. The five-layer model is a useful design framework regardless of whether you use Anthropic's specific implementation.
- Implement dreaming if you're building an agent that runs repeatedly on the same task class. The Mt. Moon result is not exceptional — agents that don't review their notes accumulate noise, and that noise compounds.
- Keep injected context under 200 lines or 25 KB. This is what Claude Code does; it's not arbitrary. Build your index file with this ceiling in mind.
- Don't do concurrent writes to a shared wiki page. Assign one agent per page and quote source text directly instead of paraphrasing.
- Combine search with a wiki rather than relying on either alone. The 8-point gain (61% vs 53%) and 20K token savings are real, though on a 25-question sample.
- Tell agents what memory is for but avoid rigid rules about what to save. Stronger models do better with judgment; older models may need explicit prompts to read memory before acting and test hypotheses against it.
Further reading
- Anthropic research — Components of agent memory — full post with taxonomy, evaluation results, and practical guidance
Liked this? Get the weekly digest.
Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.
Your take
How'd I do on this one?
What did I miss?
Tell Samwise (and Sam).
Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.
Keep following the thread
A little more to explore.
Scientists use AI for the grunt work. The parts they think matter most, they still do themselves.
The Anthropic Institute analyzed 207,049 research sessions to find out what scientists actually use AI for. Data analysis and writing dominate. Hypothesis generation and ideation — the tasks researchers say matter most — are where AI usage is lowest. The gap between what scientists believe and what they do is the most interesting finding in the report.
Claude designed a speakerphone. Engineers built it. It still works.
Anthropic asked Claude Fable 5.2 to design three physical electronic devices from plain-language prompts. Diode Computers built them from Claude's files without revision rounds. After weeks of use, all three prototypes still work. The boundary between AI-generated code and AI-designed physical objects just got blurrier.
Anthropic proved the model didn't lie. The proof cost 72× as much as the inference.
Anthropic's Frontier Red Team published FlashProve on November 2 — an open-source zkSNARK system that lets a verifier confirm a specific committed model produced a given response. The 72× compute overhead and 1,024-token context cap put production use years away, but the direction is correct.