Your AI coding assistant isn't getting dumber. It's just architecturally amnesiac. Every session, it starts fresh. Every session, you lose ground. And if you've been blaming yourself for not prompting well enough — stop. What you're hitting is a fundamental constraint baked into how large language models are built.

This is a technical explainer for solo founders who want to actually understand AI session context decay — not just feel its pain, but know what's happening under the hood so you can work around it intelligently.

The Token Window: What It Actually Is

A context window is the total information an LLM processes per inference — including instructions, conversation history, retrieved documents, and output. Everything. Your system prompt, all the code you've pasted, every message back and forth — it all counts against one shared budget.

That budget is measured in tokens, roughly 0.75 words each. So a 200,000-token context window sounds enormous — until you realize a moderately sized codebase, a few rounds of conversation, and some file pastes can burn through it fast.

Here's the part that surprises most people: most models market a context window range, but effective context often falls far below the advertised maximum. The headline number is a ceiling. Your real working memory is smaller.

The Attention Problem: Why Bigger Isn't Just Better

Under the hood, a context window is tied to the model's attention mechanism. Transformer-based LLMs use self-attention to let every token "attend to" — consider — every other token in the sequence. This is what makes the model feel coherent. It's also what makes scaling it expensive.

The computational complexity of standard attention is O(n²), meaning that doubling the context length quadruples the computational cost. So when a model advertises a million-token window, it's doing heroic engineering work just to make that possible at all.

But raw capacity isn't the real issue. Attention dilution is. Transformers enable every token to attend to every other token, creating n² pairwise relationships. As context length increases, the model's ability to capture those relationships stretches thin. A critical architectural decision you mentioned 40 messages ago starts to fade — not because it was deleted, but because it's statistically drowning in everything else.

Context Rot: The Invisible Degradation

There's a failure mode that's worse than hitting the hard token limit, and most developers don't know it exists. It's called context rot.

Hitting the hard token limit isn't the only potential failure mode. Before your context window fills up, you may run into context rot, where model performance degrades as input length increases, even when there's technically room left. LLMs don't process all tokens equally — attention concentrates on the beginning and end of the input, so information in middle positions gets less reliable processing. The result: hallucinations, ignored instructions, and contradictory answers, well before you hit any token limit.

This is AI session context decay in its subtlest form — and it's the one that bites you mid-session, not just between sessions. You're still in the same chat. Nothing got truncated. But the AI confidently suggests an approach you already ruled out three exchanges ago, because that context is now sitting in the graveyard of the middle window.

As traces grow, the ratio of useful signal to accumulated noise falls, and model performance degrades even when the window limit has not been reached. Every failed attempt, every correction, every "no wait, do it this other way" — it all accumulates as low-quality tokens that drag on the model's reasoning.

Sliding Windows and Compaction: What Gets Thrown Overboard

So what happens when you do hit the limit? The model has two options: crash, or throw something away.

To mitigate context limits, various methodologies have been developed to divide the input into segments and apply a sliding window approach to manage the context. In practice with coding tools, this means older parts of your conversation scroll out of the window as new tokens come in. If you exceed the limit, older parts of the conversation are typically truncated or removed.

Some tools use compaction instead — summarizing prior context to make room. When agents summarize prior context to make room for new work — a technique called compaction — they introduce a new failure mode. Summarization is lossy by design. The summary of "we decided to use Postgres with a specific schema and here's why" is always worse than the actual decision thread. You lose nuance, specifics, and the reasoning behind choices.

Some models use a sliding window attention pattern where each token only attends to the N most recent tokens rather than the entire context. That's efficient for computation, but it means the model literally cannot see earlier parts of your session even if they haven't been deleted — they're just outside the sliding frame.

What gets lost in all of this? The stuff that took the longest to explain. Your database schema. The unconventional architectural decision you made in week one. The three approaches you already tried that didn't work. That's exactly the context that's hardest to re-create and most expensive to lose.

Between Sessions: The Hard Reset

Everything above happens within a session. Between sessions is even more brutal.

Most AI code assistants treat each session as a clean slate. Close your IDE or start a new chat, and all the context you've built up is gone. This session-based design means you're always starting from scratch.

This is not a bug in any specific tool. It is a fundamental limitation of how large language models work today. Modern AI coding assistants — Claude Code, Cursor, GitHub Copilot, Windsurf, Codex — all share the same architectural constraint. There is no persistent memory layer connecting your sessions. The model weights don't update. Nothing gets saved. The conversation you just had might as well have never happened.

There is no persistent context layer between AI sessions. Your project's architecture, decisions, and constraints exist in two places — the code itself (which the AI can read, but doesn't understand without guidance) and your head (which the AI can't access at all). The gap between those two things is where context loss lives.

What You Can Actually Do About It

Understanding the architecture points you directly at the fix. There are three layers of the problem, and each has a different answer.

For within-session context rot: Keep your sessions shorter and more focused. Don't use one chat to redesign your schema, debug an API issue, and write a new feature. Each new concern you introduce dilutes the signal of everything that came before. One session, one clear objective.

For sliding window / compaction loss: Front-load the most important context. Attention concentrates on the beginning and end of the input — so put your critical architectural constraints, your stack, your non-negotiables at the very top of every session. Don't bury them in the middle where they'll be ignored or compacted away.

For cross-session decay: This is the big one. You need an external, persistent context layer — something that lives outside the model's context window entirely. A structured markdown file is the lowest-friction version of this. Something like a STACK.md that captures your tech stack, current state, key decisions, and what you're building next. Read more about the STACK.md method here — it's the simplest version of this that actually works.

The session handoff habit ties it together: before you close a chat, ask the AI to update that file with everything decided in the current session. This is what AI session memory management looks like in practice. It adds 60 seconds to every session and saves you 20 minutes on the next one.

Bigger context windows don't replace deliberate context management. Even as model providers ship million-token windows, the attention dilution problem doesn't go away — it just moves the goalposts. The founders who ship consistently are the ones who treat context as an asset to be managed, not something the AI handles automatically.

The AI isn't forgetting because it's bad at its job. It's forgetting because that's what the architecture does. Once you understand that, you can stop fighting it and start building systems that work with it.