Why Context Erodes

The degradation is not random. It follows from how large language models process context.

Every model has a finite context window: a hard limit on how many tokens it can attend to at once. Current models offer windows ranging from hundreds of thousands to over a million tokens. These numbers sound generous, but a productive development session generates context quickly: code snippets, design discussions, decision rationale, file contents. The window fills faster than most developers expect.

Research confirms what practitioners experience intuitively. A 2023 study from Stanford and Berkeley (“Lost in the Middle” by Liu et al.) demonstrated that language models perform significantly worse on information placed in the middle of long contexts compared to the beginning or end. The effect is substantial: recall accuracy drops measurably for content that is neither recent nor at the very start of the conversation. This is not a quirk of a particular model; it is a property of the attention mechanism itself. Recent tokens and system-level instructions receive disproportionate weight. Everything in between competes for a shrinking share of the model's focus.

The study establishes that things fade by position. What it does not address — and what I have observed repeatedly in practice — is what fades first. In my experience, the reasoning behind decisions degrades faster than the decisions themselves. The AI might remember “we are using PostgreSQL” but forget why PostgreSQL was chosen over MongoDB: the need for JSONB support, the team's operational expertise, the multi-tenancy requirements that ruled out document stores. This is a subtle but expensive failure mode: the AI continues to follow the stated decision while making suggestions that violate its intent. It proposes a schema structure that would work well in a document store but fights against PostgreSQL's relational strengths. Technically compliant with the stated choice, but architecturally misguided.

The solution is the same one developers apply instinctively to their own cognition: externalize what matters. Persist it outside the medium that forgets.

Some tools attempt to manage this problem automatically, compacting or summarizing earlier conversation history as the context window fills. But this introduces a different concern: the compaction is a black box. The developer has no visibility into what was preserved verbatim, what was summarized, and what was silently dropped. The algorithm optimizes for general coherence, not for the specific nuances that matter to a particular design decision. And the reasoning behind decisions, being verbose, explanatory, and contextual, is precisely the kind of content most vulnerable to automated compression. The what survives; the why does not. Trusting an opaque process to preserve what matters is not a strategy; it is a hope.

This is the missing piece in the alignment techniques I have described elsewhere — sharing curated project context with AI (what I call Knowledge Priming) and structuring design conversations in sequential levels (Design-First collaboration) both build a shared mental model between human and AI. But that alignment is, by default, as transient as the conversation that created it. The shared mental model we invest in building erodes as the session lengthens — and vanishes entirely when the session ends.

Context anchoring is the practice of making that alignment durable.

External Memory

The solution is to treat decision context as external state: a living document that exists outside the conversation, captures decisions as they happen, and serves as the authoritative reference for both human and AI across sessions.

This is not the same as the priming document from that earlier work. The distinction matters:

A priming document captures project-level context: the tech stack, architecture patterns, naming conventions, code examples. It is relatively stable, updated quarterly, or when significant architectural changes occur. It is shared across all features and all sessions. It tells the AI “here is how this project works.”

A feature document captures feature-level context: the specific decisions made during development, the constraints that shaped them, what was considered and rejected, what remains open, and the current state of progress. It evolves rapidly, potentially every session. It tells the AI “here is where we are on this specific piece of work, and how we got here.”

Together, they form two layers of the same context strategy. When starting a new session, both are loaded: the project context as the stable foundation, the feature context as the record of where things stand. The priming document provides the vocabulary. The feature document provides the history.

A natural objection is that modern AI tools (Cursor with its file references, Copilot with workspace indexing) can already read codebases directly. If the AI can see the code, why maintain a separate document?

Because code captures outcomes, not reasoning. A codebase that uses BullMQ directly for retry handling tells the reader nothing about whether a RetryQueue abstraction was proposed, debated, and deliberately rejected — or whether the direct approach was simply the first thing generated and never questioned. The rejected alternative is invisible in the code. The constraint that drove the decision is invisible. The open question that remains is invisible.

There is a practical byproduct worth noting. A feature document of fifty lines carries the same decision context that hundreds or thousands of lines of implementation code cannot express at all, and it does so at a fraction of the token cost. Less context in the window means the model's attention holds up better; the degradation that long contexts produce simply has less to degrade. Token efficiency is not the reason to maintain a feature document (reasoning preservation is) but it is a compounding benefit whose cost implications at scale deserve separate examination.

This is exactly the gap that Michael Nygard identified when he proposed Architecture Decision Records (ADRs) in 2011. Code shows what was built. It does not show what was rejected, what constraints shaped the choice, what trade-offs were accepted, or what remains unresolved. ADRs exist because experienced engineers recognized that the reasoning behind code is at least as valuable as the code itself — and far more fragile.

The feature document fills this same gap for AI collaboration. It is, in essence, a living ADR, one that evolves in real-time as decisions are made, rather than being written after the fact.

For teams already using ADRs, the feature document is an ADR in progress. When the feature ships, significant decisions graduate to formal ADRs. For teams not yet using ADRs, this is a natural entry point: lighter-weight, more iterative, and immediately practical.

There is one more dimension that purely individual tools miss: coordination across the team. When multiple developers work on the same feature (each with their own AI sessions) the feature document becomes the shared record. Developer A's design decisions, made with AI in one session, are available to Developer B's AI session started independently. Without the document, Developer B's AI might re-propose the very abstractions Developer A already rejected. The shared mental model is not just shared between one human and one AI; it is shared across the team, across sessions, across time.

The feature doc survives what the context window cannot.

What This Looks Like in Practice

The notification service from that earlier Design-First work provides a useful illustration.

After the design conversation (capabilities confirmed, components debated, contracts agreed) I had a set of decisions worth preserving. BullMQ used directly for retries, no wrapper abstraction. Functional services, no classes. Email-only for v1. SendGrid for delivery. These decisions, and crucially the reasoning behind each, went into a feature document alongside the current constraints, open questions, and the state of implementation.

The document was brief, under fifty lines. Not a formal template, but a working record: decisions with their reasoning, current constraints the AI must respect, open questions that remain unresolved, and a simple checklist of what was done versus what remained. Enough to capture the essential state without becoming documentation for its own sake.

# Feature: Notification Service v1

## Decisions
| Decision                    | Reason                                  | Rejected Alternative                                |
|-----------------------------+-----------------------------------------+-----------------------------------------------------|
| BullMQ directly, no wrapper | Native retry with backoff is sufficient | RetryQueue abstraction (unnecessary indirection)    |
| Functional services         | Match codebase convention               | Class-based (rejected: convention)                  |
| SendGrid for delivery       | Deliverability + team experience        | SES (cheaper, less reliable), Mailgun (no team exp) |

## Constraints
- Email-only for v1 (no SMS/push)
- All queries include tenantId (multi-tenant)
- Must use existing auth middleware

## Open Questions
- [ ] Rate limiting strategy (awaiting product input)

## State
- [x] Design approved (all 5 levels)
- [x] NotificationHandler + TemplateRenderer implemented
- [ ] DeliveryTracker (next session)
  

The value became clear at the start of the third session. Rather than reconstructing forty-five minutes of prior conversation (re-explaining the tech stack, re-establishing the design decisions, re-stating the constraints) I shared the feature document. The AI had full alignment in thirty seconds. Not because it remembered the previous sessions, but because the decisions had been externalized into a form it could read fresh. Every new session became a warm start rather than a cold one. The shared mental model did not need to be rebuilt; it was loaded.

In practice, the updates happened at natural pause points: at the end of a design level, when a significant decision was made, or when an open question was resolved. Sometimes I wrote the update myself. Sometimes I asked the AI to summarize the decision and its reasoning, then edited that summary into the document. The effort was minimal: a few lines after each significant moment, not a documentation exercise.

There was a secondary benefit I had not anticipated. The discipline of updating the document streamlined my own thinking. Writing down why we chose direct BullMQ integration over a wrapper forced me to articulate the reasoning clearly — and occasionally revealed that my reasoning was weaker than I thought. The document was not just external memory for the AI. It was a forcing function for clarity in my own decision-making.

Over three sessions, the document evolved: new decisions accumulated, open questions were resolved, the implementation state progressed. A colleague joining the feature — or a new AI session — could read this document and have the full context of days of work in minutes. No repetition. No re-explanation. The document carried the shared understanding forward.

Calibration

Context anchoring is not needed everywhere. It is specifically valuable when a feature spans multiple sessions — when the risk of losing context is real and the cost of re-establishing it is high.

For a quick debugging question or a one-off utility, the overhead of maintaining a document is not justified. For a feature that takes an afternoon, a lightweight capture may be worthwhile if there is any chance of revisiting. For work that stretches across days, full context anchoring pays for itself many times over. This is where the vicious cycle of clinging to long conversations is most likely to emerge, and where externalizing decisions breaks the cycle most effectively.

The litmus test returns here: if I can close my chat session and start a new one without anxiety, without feeling I have lost something that cannot be recovered — my context is properly anchored. If I feel the need to keep a session alive, that discomfort is the signal. It means decisions exist only in the conversation, and the conversation is the wrong place for them to live permanently.

Conclusion

This is, at its core, a shift from chat-driven development to document-driven development. The conversation remains the medium for making decisions, but the document becomes the record. Conversations are disposable by design — they are where thinking happens, not where conclusions are stored. The document persists.

The shared mental model between human and AI does not have to be transient. It can be documented, durable, and shareable. Together with the techniques that precede it — sharing curated project context before a session begins, structuring design conversations in sequential levels before any code is written — context anchoring completes a progression: static context, dynamic alignment, persistent decisions. Each layer builds on the last.

And the simplest test of whether it is working is also the most practical: close the session. Start fresh. If that feels effortless — if starting over costs thirty seconds of document-sharing rather than thirty minutes of re-explanation — the context is where it belongs. Outside the conversation, in a form that both human and AI can read, anytime.