Where to Put the Important Stuff: Ordering Context in a Prompt

// In plain English

When you write a prompt for a large language model, you naturally think about what to include: the task, the rules, a few documents, maybe some examples. But there's a second decision hiding inside every prompt — where each piece goes. The model does not weigh all parts of your prompt equally. The same facts placed at the top, the bottom, or buried in the middle can produce noticeably different answers.

Think of someone skimming a long memo in a hurry. They read the opening carefully, glance at the closing line, and let their eyes glaze over the dense paragraphs in between. If the one number that matters is stuck in the middle of page 4, there's a good chance they miss it. Models behave in a surprisingly similar way: they pay the most attention to the start and the end of a prompt, and the least to the middle. Researchers named this the lost-in-the-middle effect.

So ordering context is the simple, free skill of arranging the pieces of your prompt — instructions, key facts, retrieved documents, examples — so the parts that matter most land where the model is actually paying attention. No fine-tuning, no extra tokens, no new tools. Just moving text around. It is one of the highest-leverage habits in prompt engineering, and most people never think about it.

// Why it matters

For a one-line question, ordering barely matters — there is no middle to get lost in. It starts to bite the moment your prompt grows: a long system prompt, several retrieved documents, a transcript, and then the actual task. That is exactly the shape of most real applications — chatbots, RAG systems, agents — so the effect shows up in production far more than in a quick demo.

• Wrong answers from correct context. The right fact is in the prompt, but it sat in the dead zone, so the model answered as if it never saw it. Frustrating, because the data was right there — you just placed it badly.
• Ignored instructions. You wrote a clear rule near the top, then pasted a 5,000-word document after it. By the time the model reaches the end, the early instruction has faded and the answer drifts off-task.
• Wasted retrieval. Your retriever found the perfect passage, but it landed as document number seven of ten, smack in the middle. Good retrieval, poor placement, mediocre answer.
• Hard-to-debug quality dips. Because the content is all present, the bug is invisible in a diff. You stare at a prompt that looks complete and wonder why the model keeps missing things.

The payoff is that fixing order is nearly free. You are not adding tokens or buying a bigger model — you are rearranging text you already have. A few placement habits often recover accuracy that people otherwise try to buy with expensive context windows or extra retrieval. This is the practical heart of context engineering: not just what goes in the window, but in what order.

// How it works

To order context well, it helps to picture the prompt as a vertical stack. A typical large prompt has the same recurring layers, and each one has a natural home based on whether the model needs it early, late, or can tolerate it in the middle.

Why the middle fades: attention is not uniform

Under the hood, a model reads your whole prompt through an attention mechanism: every token can, in principle, look at every other token. But the strength of that looking is learned, not flat. In practice models attend more strongly to tokens near the beginning (which set the frame for everything after) and near the end (which are freshest when the model starts writing). Tokens stranded in a long middle get comparatively weak attention, so facts there are recalled less reliably. The longer the prompt, the deeper and more dangerous that middle becomes.

The core moves

Everything below follows from one idea — put what matters where attention is strong — applied to each layer of the stack.

• Lead with instructions, close with the task. Put the role and hard rules at the very top (primacy), and put the actual question or command at the very bottom (recency). After a long paste, the model should read your ask last.
• Bury bulk in the middle, but order it well. Long documents and transcripts have to go somewhere; the middle is the right place for sheer volume. Inside that block, put the most relevant material at the start and end of the block, not buried in its own middle.
• Fence each block clearly. Wrap documents and data in obvious delimiters so the model can tell instructions from content. This pairs naturally with structuring prompts using XML or markdown.
• Restate the task after a big paste. If a huge document sits between your instruction and the model's answer, repeat the instruction at the bottom. The cost is a few tokens; the gain is the model re-anchoring on the task right before it writes.

// A worked example

Here is the same task built two ways. Both contain identical information; only the order differs. Imagine answering a support question from a long policy document.

The fragile version

Answer the customer using only the policy below. Be concise.

--- POLICY (4,000 words) ---
...long sections on shipping, accounts, privacy...
...the refund window is mentioned once, around the middle...
...more sections on warranties, gift cards, returns process...
--- END POLICY ---

Customer: How long do I have to return a physical item?

The single relevant sentence sits in the middle of a long block, and the original instruction ("be concise, use only the policy") is now thousands of tokens behind the model. Both the key fact and the rule are in a weak position.

The robust version

You are a support assistant. Rules:
- Answer ONLY from the policy text provided.
- If the answer isn't in the policy, say you don't know.
- Keep it to one or two sentences.

<policy>
...long sections on shipping, accounts, privacy...
...the refund window is mentioned once...
...more sections on warranties, gift cards, returns process...
</policy>

Reminder: answer ONLY from the <policy> above, in one or two sentences.

Customer question: How long do I have to return a physical item?

Nothing was added except a closing restatement and clear fences. The rules sit in the high-attention opening, the bulky policy is fenced as a clearly-marked middle block, and the task plus a reminder land last — exactly where the model looks before it answers. Same tokens, better placement, more reliable result.

// Where to put each piece

A quick reference for the common ingredients of a large prompt and their best position, with the reason.

Ordering retrieved chunks in RAG

If you do retrieval, you usually get back a ranked list of chunks. The naive move is to paste them in rank order, 1 to 10 — which puts your second-best through ninth-best chunks right in the lost-in-the-middle zone. A known mitigation is to reorder so the strongest chunks sit at the start and the end of the block, and the weaker ones fill the middle. Some teams simply place the single top chunk last, closest to the question. The deeper fix is to retrieve fewer, better chunks so there is less middle to begin with — see putting documents in prompts.

// Common pitfalls

• Instruction-then-wall-of-text. The single most common mistake: one good rule at the top, a giant paste after it, and nothing at the end. Always restate the ask after a long block.
• Pasting retrieved chunks in raw rank order. Rank 1 at the top is fine, but ranks 2–9 then occupy the exact dead zone. Reorder so your best material hugs the edges.
• Burying the actual question. If the user's real question is somewhere in the middle of a long conversation history, the model may answer an earlier turn. Surface the current task at the bottom.
• Over-padding to feel safe. "More context can't hurt" is wrong — more context deepens the middle. Every marginal document you add pushes good material further from the edges. Less, well-ordered context often beats more, dumped in.
• Assuming a bigger context window fixes it. A larger window lets you paste more, which makes the lost-in-the-middle problem worse, not better. Capacity and attention are different things — see context windows.

// Going deeper

Once the basic habits are second nature, a few nuances separate good ordering from careful ordering.

Measure it, don't assume it. The clean way to know whether order is hurting you is a needle-in-a-haystack test: hide one specific fact (the needle) at different depths inside a long filler context (the haystack), then ask the model to retrieve it and chart accuracy by position. A U-shaped curve — strong at the edges, weak in the middle — is the lost-in-the-middle signature for your specific model and prompt length. Run it before you trust any placement rule.

Order interacts with structure. Placement and formatting reinforce each other. Clear delimiters (XML tags, markdown headers) help the model keep separate blocks separate, which makes a restated instruction at the bottom unambiguous rather than blending into the document above it. Good structure makes good ordering legible.

Prompt caching changes the calculus. Many platforms let you cache a stable prefix of a prompt so repeated calls are cheaper and faster. Caching rewards putting the unchanging parts first (system prompt, fixed instructions, static reference docs) and the variable parts last (the user's question). Happily, that is the same ordering attention prefers — stable context up top, fresh task at the bottom — so optimizing for cost and optimizing for attention usually point the same way.

It compounds with other techniques. Ordering is a foundation, not a replacement. It pairs with choosing the right number of few-shot examples, with chain-of-thought reasoning that you place after the context, and with retrieval quality upstream. The honest summary: position is one of the cheapest accuracy levers you have, it is invisible in a code review, and the only way to trust it for your model is to test it. Reorder first, then reach for bigger or more expensive solutions.