GPT-5.6 ARC-AGI-3: Two Settings That Tripled Scores

OpenAI shipped GPT-5.6 this week, and the story that matters isn’t the model card — it’s the configuration. Independent testing on ARC-AGI-3, the new interactive reasoning benchmark, found that two API knobs account for roughly a 3x swing in agentic scores on the same model: pushing reasoning effort to its top tier, and turning on the persistence path that carries thinking state across tool calls. Leave the defaults alone and GPT-5.6 looks like an incremental bump over GPT-5.1. Flip both and it reads frontier-class. If you are wiring GPT-5.6 into an agent loop today, the GPT-5.6 reasoning effort settings you choose are not a tuning detail — they decide whether you ship a mediocre agent or a competitive one.

What’s new with GPT-5.6 reasoning effort settings

Two things landed at once, which is why this is confusing. The first is GPT-5.6 itself: a reasoning-model refresh in the GPT-5.x line, positioned as better at long-horizon tool use and multi-step planning than GPT-5.1. The second is ARC-AGI-3, the interactive version of the ARC benchmark family. ARC-AGI-1 and -2 showed a static puzzle and asked for an answer grid. ARC-AGI-3 drops the model into small interactive environments where it must take actions, observe what changed, form a hypothesis about the rules, and act again. It is an agentic benchmark, not a pattern-matching one — which is exactly why configuration dominates.

GPT-5.6’s default API configuration underuses the model. Reasoning models in this family expose a reasoning effort parameter, and the default sits mid-range to keep latency and token cost reasonable for chat-shaped traffic. On an interactive eval, that default caps how much the model can deliberate per move, and in ARC-AGI-3 one bad early move can poison an entire episode. Raising effort to the top tier produced a large single-factor gain.

The second factor is reasoning persistence across tool calls: passing the model’s prior reasoning state back in on the next turn instead of discarding it. Without it, every tool result restarts the model’s chain of thought about the environment. With it, the model accumulates a working theory of the game across dozens of actions. Stack both and the reported agentic score roughly tripled.

The nuance matters: this is not a jailbreak or a prompt trick, and nobody is claiming GPT-5.6 is secretly three times smarter. The claim is about the gap between benchmark-default and benchmark-optimal configuration on agentic evals — and about how much of “model quality” in 2026 is really harness quality. The same pattern keeps showing up across agentic benchmark scores 2026: the published number tells you as much about the scaffold as the weights.

Why it matters

• Your default GPT-5.6 agent is probably leaving most of its capability on the table. If you migrated from GPT-5.1 by swapping the model string and nothing else, you inherited a mid-tier effort default tuned for chat latency, not for multi-turn tool loops.
• Benchmark comparisons between models are now nearly meaningless without config disclosure. A GPT-5.6 vs GPT-5.1 reasoning comparison run at default effort measures something different than one run at high effort. Any leaderboard that omits effort tier and persistence settings compares harnesses, not models.
• Cost modeling has to be redone. Top-tier reasoning effort spends far more reasoning tokens per call, often multiples. A 3x score gain that costs 5x tokens is a real trade, and its value depends entirely on how expensive a failed episode is for you.
• Reasoning persistence changes your context architecture. If prior reasoning must survive tool calls, your agent framework has to round-trip opaque reasoning items faithfully. Frameworks that rebuild message arrays from scratch each turn, or strip unrecognized fields, silently destroy the gain.
• Latency budgets break. Interactive products built on a two-second response assumption cannot simply flip to top-tier effort. This pushes teams toward tiered routing: cheap effort for the easy 80%, high effort for the branch that actually needs planning.
• Evaluation discipline is now a competitive moat. Teams that sweep GPT-5.6 API parameters systematically find these gains within a day. Teams that don’t conclude the model is a disappointment and are wrong.

How to use it today: configuring GPT-5.6 API parameters

1. Confirm the model and available effort tiers. Effort tier names have shifted across the GPT-5.x line, so read them off the API rather than assuming.

   curl https://api.openai.com/v1/models/gpt-5.6 \
     -H "Authorization: Bearer $OPENAI_API_KEY"

2. Move to the Responses API if you’re still on Chat Completions. Reasoning persistence across tool calls depends on passing reasoning items back, which is a Responses API concept. This is the single highest-leverage migration step.

   from openai import OpenAI
   client = OpenAI()
   
   resp = client.responses.create(
       model="gpt-5.6",
       reasoning={"effort": "high"},
       input=[{"role": "user", "content": "Play the level. Explain your rule hypothesis before acting."}],
       tools=TOOLS,
   )

3. Set reasoning effort to the top tier for the agent loop specifically. Don’t set it globally. Set it on the code path where the model plans and calls tools, and leave chat-shaped calls at a lower tier.

   EFFORT_BY_PATH = {
       "chat":        "low",
       "summarize":   "low",
       "agent_loop":  "high",   # top tier — the 3x lives here
       "eval_harness":"high",
   }
   
   def effort_for(path: str) -> str:
       return EFFORT_BY_PATH.get(path, "medium")

4. Enable reasoning persistence by round-tripping the response items. Take everything the model returned, append your tool outputs, and send the whole list back. Do not reconstruct a clean message array — that is what throws the reasoning away.

   history = [{"role": "user", "content": task}]
   
   for turn in range(MAX_TURNS):
       resp = client.responses.create(
           model="gpt-5.6",
           reasoning={"effort": "high"},
           tools=TOOLS,
           include=["reasoning.encrypted_content"],  # keeps state if you're stateless
           store=False,
           input=history,
       )
   
       # Critical: carry ALL output items forward, reasoning items included.
       history += resp.output
   
       calls = [i for i in resp.output if i.type == "function_call"]
       if not calls:
           break
   
       for call in calls:
           history.append({
               "type": "function_call_output",
               "call_id": call.call_id,
               "output": run_tool(call.name, call.arguments),
           })

5. Verify persistence is actually working. The failure mode is silent. Assert that reasoning items survive each turn — if the count stays flat, your framework is dropping them.

   reasoning_items = [i for i in history if getattr(i, "type", "") == "reasoning"]
   print(f"turn={turn} reasoning_items={len(reasoning_items)} "
         f"reasoning_tokens={resp.usage.output_tokens_details.reasoning_tokens}")
   assert len(reasoning_items) >= turn, "reasoning state is being discarded"

6. Run the 2×2 sweep on your own eval before you commit. The ARC-AGI-3 result is a signal, not a guarantee for your workload. Four configurations, your task set, real numbers.

   for effort in low high; do
     for persist in off on; do
       py eval.py --model gpt-5.6 \
                  --effort "$effort" \
                  --reasoning-persistence "$persist" \
                  --episodes 50 \
                  --out "results/gpt56_${effort}_${persist}.json"
     done
   done
   
   py summarize.py results/*.json --metric solve_rate --metric cost_per_episode

7. Try ARC-AGI-3 directly if your product is agentic. The environments are small, adversarial to memorization, and brutally honest about whether your harness carries state. It is the cheapest proxy for “does my agent loop actually accumulate understanding.”

8. Add a per-episode reasoning-token cap. Top-tier effort plus a long tool loop is how you get a surprise invoice. Cap it, log it, alert on it.

   MAX_REASONING_TOKENS_PER_EPISODE = 250_000
   
   budget = MAX_REASONING_TOKENS_PER_EPISODE
   budget -= resp.usage.output_tokens_details.reasoning_tokens
   if budget <= 0:
       downgrade_to("medium")  # or abort the episode

How it compares

Configuration	Reasoning effort	Persistence across tool calls	ARC-AGI-3 relative score	Relative token cost	Best for
GPT-5.6, out of the box	Default (mid)	Off	1.0x (baseline)	1.0x	Chat, single-turn tasks
GPT-5.6, effort only	High / top tier	Off	Large single-factor gain	Several x	Hard one-shot reasoning
GPT-5.6, persistence only	Default (mid)	On	Moderate gain	~1.2x	Long tool loops on a budget
GPT-5.6, both enabled	High / top tier	On	~3x baseline	Highest	Agentic products, evals
GPT-5.1, tuned	High	On	Below tuned 5.6	Lower per call	Cost-sensitive migrations

Read the table as a shape, not as gospel decimals — the exact multiplier depends on the harness and the episode set. Both knobs matter, they compound, and the combined configuration is where the headline number comes from. Note the third row. If you cannot afford top-tier effort, reasoning persistence tool calls is the cheaper half of the gain and costs almost nothing in tokens. It should be on by default in every agent loop regardless of effort tier.

Worth flagging for anyone benchmarking against Claude or Gemini reasoning models: those families expose analogous controls — extended thinking budgets, thinking-block preservation across turns — and the same asymmetry applies. A cross-vendor comparison where one side is tuned and the other runs on defaults is not a comparison. It is a configuration audit with a model name attached.

What’s next

Expect the defaults to move. When a mid-tier default demonstrably hides a 3x agentic gain, that is a product bug as much as a parameter choice, and the obvious fix is either raising the default or making effort adaptive — letting the model spend more when the task looks hard and less when it doesn’t. OpenAI has already drifted that direction across the GPT-5.x line. If adaptive effort lands, the manual sweep above becomes a transitional practice, and the interesting question shifts to whether the router’s judgment beats your own.

Expect leaderboard norms to tighten too. ARC-AGI-3 arrived because static benchmarks stopped discriminating, and its first major result is a config story — which makes mandatory configuration disclosure hard to argue against. Watch for ARC Prize and the major eval harnesses to start publishing effort tier, persistence, turn limits, and token spend alongside every score. When they do, a chunk of the 2026 model-ranking discourse will need revisiting, because a meaningful share of claimed model-to-model gaps will turn out to be harness gaps.

On your own side, watch the cost curves. Top-tier reasoning effort in a long tool loop is the most expensive way to call an LLM that currently exists. The teams who win on agentic products over the next few quarters won’t be the ones who flip every knob to maximum — they’ll be the ones who figure out which 5% of decisions in an episode deserve top-tier deliberation, and route accordingly. That routing layer is where the engineering is now. The model is increasingly the easy part.

Frequently Asked Questions

Does raising reasoning effort really triple performance, or is that benchmark-specific?

Benchmark-specific in magnitude, general in direction. ARC-AGI-3 is unusually sensitive to deliberation and state because a single bad early action can lose an entire episode, so it amplifies the effect. On simpler tasks — classification, extraction, summarization — top-tier effort often buys nothing and costs a lot. Sweep on your own workload before assuming the multiplier transfers.

What exactly is reasoning persistence across tool calls?

Passing the model’s own prior reasoning output back into the next request instead of discarding it. In the Responses API this happens naturally if you append every returned output item — reasoning items included — to your input list for the following turn. Without it, the model re-derives its understanding of the environment from scratch after every tool result, which is both slower and worse.

Will this blow up my API bill?

It can. Top-tier effort generates substantially more reasoning tokens per call, and an agent loop makes many calls. The mitigations that work: apply high effort only on the planning path, cap reasoning tokens per episode, and log reasoning-token usage as a first-class metric rather than noticing it on the invoice. The cheap half of the gain — persistence — costs almost nothing, so start there.

Is GPT-5.6 actually better than GPT-5.1, or is this all configuration?

Both. Tuned GPT-5.6 outperforms tuned GPT-5.1 on agentic evals, so there is real model improvement. But the headline gap that got attention came from comparing configurations, not weights. If you are evaluating a migration, run both models at matched effort and persistence settings — otherwise you are measuring your harness.

Do I have to migrate off Chat Completions?

To get the persistence half of the gain, effectively yes. Chat Completions has no clean way to carry reasoning state across turns; the Responses API is built around it. You can raise reasoning effort without migrating and still capture the larger single factor, but you will leave the cheap half on the table.

How do I know persistence is working rather than silently failing?

Instrument it. Count reasoning items in your input list each turn and log reasoning tokens per call. If the item count stays flat while turns increase, something in your framework is stripping them. This is the most common way teams “enable” persistence, see no improvement, and wrongly conclude the finding doesn’t replicate.