Ordering, Recency and Attention
Position Is Not Neutral
Models attend unevenly across a long context, with material at the beginning and the end reliably better used than material in the middle. This is a well-replicated effect across model families rather than a quirk of any one of them, and it has a direct engineering consequence: where you put something changes whether it is used. The constraint that governs the whole run belongs in the stable opening region. The material that governs the next action belongs near the end, adjacent to where generation begins. The bulk that is merely available — background documents, older history — belongs in the middle, where degradation is tolerable because you are not depending on it. Any layout that puts the critical constraint in the middle of a long context is one you should expect to fail intermittently, which is the worst way to fail.
- Beginning and end are used reliably; the middle is where material goes to be ignored
- Run-wide constraints at the top; next-action material at the bottom
- Background bulk belongs in the middle precisely because you are not relying on it
- A critical constraint buried mid-context fails intermittently, which is the hardest failure to diagnose
Restate What Must Not Be Forgotten
For constraints that must hold across a long run — the account you are permitted to touch, the format the output must take, the thing you must never do — a single statement at the top of a growing context is not enough. Restate them near the end of the assembled context at each step, ideally rendered from structured run state rather than copied prose, so they are cheap, consistent and impossible to lose to compaction. This looks redundant and is not: the constraint has to survive a context that may be an order of magnitude longer than when it was first stated. Keep the restatement short and specific. A long restated block competes with the observations it sits next to, and the value comes from proximity to the decision point rather than from volume.
- Restate hard constraints near the end of context on every step
- Render from run state, not by copying prose, so they cannot drift or be compacted away
- Keep it short — a long restatement competes with the observation it sits beside
- Redundancy is the point: the constraint must survive a much longer context than it was written into
Mark the Boundary Between Instruction and Data
Everything the agent reads arrives in the same channel as everything it was told. Retrieved documents, tool results, file contents and page text are data, but they are rendered as text in a context whose other text is instructions, and models do not have a reliable structural way to tell the difference. Mark it explicitly: wrap external content in clearly delimited sections, label the source, and state in the system instructions that content inside those sections is material to reason about rather than direction to follow. Be honest about what this achieves. It measurably helps and it is not a boundary — a determined instruction embedded in retrieved content can still be followed, which is why the security treatment of this problem lives in structural controls rather than in framing. Do it anyway, because it is nearly free and it reduces accidental instruction-following, which is more common than the adversarial case.
- External content and instructions share one undifferentiated channel
- Delimit and label external material, and say what its status is
- This reduces accidental instruction-following; it is not a security boundary
- The adversarial case needs structural controls — this is hygiene, not defence
Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.