Repeat Yourself: How Prompt Repetition Quietly Boosts LLM Accuracy for Free

By Ankit Gubrani on

There is a class of AI optimizations that sounds almost too simple to be real. This is one of them.

A team at Google Research published a paper titled "Prompt Repetition Improves Non-Reasoning LLMs" with a finding that is hard to believe until you see the data: if you repeat your input prompt twice, back to back, LLM accuracy goes up. Often significantly. And it does not increase the number of tokens generated, does not change the output format, and does not add measurable latency.

That last part is worth sitting with for a moment. A free accuracy boost with no latency penalty. Let us understand why this works, what the experiments actually showed, and what it means for how you build AI-powered systems.

The Root Problem: Causal Attention Has a Blind Spot

To understand why prompt repetition helps, you need to understand a fundamental constraint in how most modern LLMs process your input.

LLMs like GPT, Gemini, Claude, and Deepseek are built on a causal language model architecture. In a causal model, tokens can only attend to tokens that came before them. Token number 50 can look back at tokens 1 through 49, but it cannot look forward at tokens 51 onward. This is by design. It is what allows the model to generate text one token at a time, left to right, without cheating by looking at the answer.

This works well for generation. But it creates a subtle problem during the input processing phase, which is called the prefill stage.

Think about a typical prompt structure: <CONTEXT> <QUESTION>. The context comes first, then the question at the end. By the time the model is processing the question tokens, it has already processed all the context tokens. Those context tokens had no way of knowing what the question was going to be. They were processed in isolation, without the question to guide which parts of the context mattered.

Flip it around: <QUESTION> <CONTEXT>. Now the context tokens can attend to the question tokens that preceded them. But then the question tokens themselves had no context when they were first processed. Either way, you get an asymmetry. The order of your prompt changes which tokens can see which, and that changes what the model extracts.

Why Token Order Creates Attention Asymmetry Scenario A: <CONTEXT> <QUESTION> CONTEXT tokens processed first, no question yet QUESTION can see context can attend context blind to question Scenario B: <QUESTION> <CONTEXT> QUESTION no context when processed CONTEXT tokens can see question can attend question blind to context The Fix: <QUERY><QUERY> (Prompt Repetition) First copy: context tokens processed normally Second copy: every token can now attend to every other token from the first pass Full bidirectional attention achieved in the second repetition Causal masking means tokens only attend backward. Repetition breaks the asymmetry.

This asymmetry is not a bug. It is a consequence of the causal architecture. But it does mean the model is sometimes working with incomplete information when processing any given token.

What Prompt Repetition Actually Does

The fix the Google Research team proposes is almost embarrassingly simple. Instead of sending <QUERY>, you send <QUERY><QUERY>. The full prompt, repeated twice, back to back.

Here is why this helps. During the first pass through the prompt, everything behaves as normal. Context tokens process without knowing the question. Question tokens process with context before them but not after.

But in the second repetition, every token in your prompt now has a full copy of the entire prompt sitting in the positions before it. The second occurrence of the question can attend to the first occurrence of the context. The second occurrence of the context can attend to the first occurrence of the question. Every token now has access to the full prompt when computing its representation.

Think of it like reading a complex technical document. On the first read, certain passages are confusing because you have not gotten to the section that explains them yet. On the second read, everything makes more sense because you already know where it is all going. Prompt repetition gives the model that same second-read advantage, but inside the prefill stage rather than through extra generated tokens.

And this is the key engineering insight: the prefill stage is fully parallelizable. Unlike token generation, which is sequential (each token depends on the previous one), the prefill stage processes all input tokens in parallel across your GPU. So doubling the input length during prefill adds some compute, but it does not add to the sequential decode phase. Output latency stays the same.

What the Experiments Actually Showed

The paper tested prompt repetition across seven popular models: Gemini 2.0 Flash, Gemini 2.0 Flash Lite, GPT-4o-mini, GPT-4o, Claude 3 Haiku, Claude 3.7 Sonnet, and Deepseek V3. Seven benchmarks were used: ARC, OpenBookQA, GSM8K, MMLU-Pro, MATH, and two custom tasks called NameIndex and MiddleMatch.

The headline number: prompt repetition won 47 out of 70 benchmark-model combinations, with zero losses. That is not a small sample. That is consistent improvement across the board.

Some results were modest. For multiple-choice benchmarks where the question comes first (so the context already has the question to attend to), the gains were smaller. That makes intuitive sense.

But on the custom tasks, the results were dramatic. On the NameIndex benchmark, Gemini 2.0 Flash-Lite improved from 21% accuracy to 97% accuracy with prompt repetition. That is not a marginal improvement. That is a task the model was essentially failing at that it suddenly does almost perfectly.

Latency stayed flat. Output length stayed flat. Output format stayed unchanged. The paper explicitly verified that padding the prompt with periods to match the same length did nothing, confirming the gains come from the semantic repetition of the prompt itself, not just from making the input longer.

Prompt Repetition: Accuracy Gains (Illustrative from Paper) Accuracy (%) 100 75 50 25 0 NameIndex (Flash-Lite) 21% 97% ARC (GPT-4o) 65% 72% MMLU-Pro (Claude Sonnet) 60% 68% MiddleMatch (Gemini Flash) 20% 67% GSM8K (Deepseek V3) 70% 76% Baseline (no repetition) With Prompt Repetition Values are illustrative representations based on paper findings. Actual results vary by model and benchmark.

The One Rule: Disable Reasoning Mode

There is an important caveat that the paper is explicit about. Prompt repetition is specifically for non-reasoning mode.

When you enable chain-of-thought reasoning (the "think step by step" instruction) or use a model's built-in reasoning mode, the model already compensates for attention asymmetry by thinking through the problem iteratively. The generated reasoning tokens give the model a chance to revisit context and question together, naturally resolving the blind spot that prompt repetition addresses.

The paper found that prompt repetition on top of reasoning was neutral to slightly positive (5 wins, 1 loss, 22 neutral outcomes). Not harmful, but not particularly useful either. The model has already solved the problem through its reasoning process.

The implication for engineers is clear: use prompt repetition when you are in direct-answer mode, typically for classification, data extraction, structured output generation, or any task where you want a fast, direct response without the overhead of chain-of-thought reasoning.

Where This Matters Most in Real Systems

If you are building AI features on top of LLMs, the tasks where prompt repetition will help most are the ones where order sensitivity is highest. These tend to be tasks where the model needs to relate a question or instruction to a large body of context.

Document-grounded question answering is a clear case. You send a long document followed by a question. With prompt repetition, the document tokens in the second pass can attend to the question in the first pass, and the question tokens can more fully integrate the document. Classification over long text works similarly. Extraction tasks where you ask the model to pull specific fields from a lengthy input benefit from the same mechanism.

The paper also notes that repeating three times (QUERY x3) sometimes substantially outperforms repeating twice, particularly on the harder retrieval tasks. It is worth experimenting with repeat count for your specific use cases.

Prompt Repetition: When to Use It Use Prompt Repetition Reasoning mode OFF Classification tasks Data / field extraction QA over long documents Structured output generation Latency-sensitive pipelines Skip It Reasoning mode ON (neutral effect) Very long prompts at scale Context window near limit Creative generation tasks Conversational / multi-turn chats Token budget is extremely tight

The Connection to How Reasoning Models Actually Work

The paper makes an interesting observation worth calling out. Reasoning models trained with reinforcement learning, like o1 or Gemini's thinking variants, often learn to spontaneously repeat or paraphrase the user's query as part of their internal reasoning trace. The model effectively rediscovers prompt repetition on its own as a useful behavior.

Prompt repetition in the prefill stage is essentially doing the same thing, but more efficiently. Instead of using generated tokens to re-examine the prompt (which adds to sequential decode time), you frontload the repetition into the parallelizable prefill phase. You get the accuracy benefit of re-reading without the latency penalty of reasoning tokens.

This is a clean engineering insight: the model needs to see the full prompt from multiple "perspectives." Reasoning achieves this through generated thinking tokens. Prompt repetition achieves it through prefill. Same outcome, different mechanism, drastically different cost profile.

What Comes Next: Attention Residuals

Prompt repetition is a clever workaround for an architectural limitation. But researchers are also working on addressing that limitation more directly at the architecture level.

The Kimi team recently published a paper on Attention Residuals (AttnRes), which takes aim at a related problem in transformer architecture. Standard residual connections in modern LLMs accumulate all layer outputs with fixed unit weights, which the paper argues causes uncontrolled hidden-state growth as depth increases and progressively dilutes each layer's contribution.

AttnRes replaces this fixed accumulation with softmax attention over preceding layer outputs, letting each layer selectively aggregate earlier representations with learned, input-dependent weights. In practical terms: each layer learns to decide which earlier layers are most relevant for the current token, rather than naively summing all previous layer outputs equally.

For large-scale deployment, they also introduce Block AttnRes, which partitions layers into blocks and attends over block-level representations to reduce memory overhead while preserving most of the gains. The paper reports that scaling law experiments confirm the improvement holds consistently across model sizes.

These two papers together tell an interesting story. Prompt repetition is a zero-code inference-time fix you can use today. Attention Residuals is an architectural direction that may influence how next-generation models are trained. Both are responses to the same underlying reality: causal attention has structural limitations, and researchers are actively working both around and through them.

If you want a deeper dive into how Block AttnRes actually works under the hood, I covered it in full detail in The Same Trick That Made Transformers Great Just Made Them Better.

How to Actually Use This

The implementation is as simple as it sounds. Wherever you are constructing your prompt string before sending it to the API, repeat it.

If your current prompt is: "You are a data extraction assistant. Here is the document: [DOCUMENT]. Extract the following fields: [FIELDS]."

Your repeated prompt becomes: "You are a data extraction assistant. Here is the document: [DOCUMENT]. Extract the following fields: [FIELDS]. You are a data extraction assistant. Here is the document: [DOCUMENT]. Extract the following fields: [FIELDS]."

No special formatting. No separators. No instructions telling the model it is repeated. Just the prompt, twice. The paper found that adding "verbose" separators or instructions about the repetition performed similarly to plain repetition, so you do not need the extra complexity.

A few practical notes from the paper worth flagging:

  • Claude (Haiku and Sonnet) showed latency increases for very long prompts when using repetition, likely due to the prefill stage taking longer at scale. For short to medium prompts this is not a concern, but benchmark your specific payload sizes.
  • The improvement is larger when the prompt structure puts options or context before the question. If your prompts are already question-first, you will see smaller but still positive gains.
  • Repeating three times is worth testing on extraction-heavy or retrieval-heavy tasks.
  • This technique works across all major providers through their public APIs, no special configuration needed.

The Bigger Lesson

What makes this paper interesting beyond the practical trick is what it reveals about how LLMs process information. These models are not reading your prompt the way you read it. They are processing tokens causally, left to right, and the order in which information appears fundamentally shapes what each token can attend to.

Engineers who understand this can make better architectural decisions. When you structure a prompt, you are not just writing instructions. You are shaping the attention graph that determines what context each token has available. That is worth thinking about every time you design a system prompt or a retrieval-augmented generation pipeline.

Prompt repetition is the simplest possible version of exploiting this: give every token a second pass with full context. But the same intuition applies to thinking carefully about prompt ordering, separating different information types, and structuring retrieval results to put the most relevant context adjacent to your question.

The model is only as good as the attention it can form. Anything you do at prompt design time that improves the quality of that attention graph is a free accuracy improvement waiting to be claimed.

You can read the full Google Research paper here: Prompt Repetition Improves Non-Reasoning LLMs.