Context Language Models, Dissected: Letting the Model Edit Its Own Context, the Math Behind It, and Where I Disagree

Paper: Context Language Models · Shao, Shen, Yin, et al. (UW, Meta Superintelligence Labs, MIT, Trillium Labs), September 2026 · Code

#TL;DR

Context Language Models let an agent edit its own context file instead of only appending to it and waiting for a harness to compact it. With no training, this beats harness-scheduled and tool-based context management on long-horizon coding, research, and optimization tasks, often at lower compute. Context policies can be steered with one sentence, evolved as a skill document, or trained with RL.

The idea costs something real: editing breaks the prefix property that both serving caches and RL training silently rely on. The paper’s most interesting technical work repairs that loss: a cache-aware cost metric, Suffix Cache Reuse for serving, and stepwise GRPO with a success-gated efficiency advantage for training.

My verdict: a strong, honestly measured paper whose Bitter Lesson framing is mostly earned. Read its “up to” numbers as best cases, and treat stale cache states and self-edited context as security questions before production use.

#How to read this post

This is a paper dissection, not a summary. I go through the paper section by section, work through the math, and say where I agree and where I don’t. Whenever a section leans on a concept you may want to brush up on, there’s a 🔁 Refresher section just before it. Skip those freely if you already know the topic; each one links back to the main track.

#Where this connects to my earlier posts

This paper sits at the intersection of several things I’ve written about here. In Virtual DA, a self-harness loop evolves agent skills from failure traces; the paper’s skill-evolution experiments optimize a context-management skill document the same way. My context-pruning work for stablecoin regulation trained an external pruner; CLMs move that job inside the model. My SGLang deep dive walked through the radix-tree prefix matching that the paper’s Suffix Cache Reuse extends. And my TorchRL post framed the LLM as an RL agent; stepwise GRPO is what training that agent looks like when it edits its own context.

The most interesting connection is a disagreement. In The Leash and the Wolf, I argued that agents need deterministic external orchestration. This paper makes the opposite, Bitter Lesson bet. Part of this review is working out how much of my leash survives; I give my answer in Move 2.

I also put the paper’s learning claim to a test on hardware I control. In Tiny CLM, I trained a 1.7B model with stepwise GRPO for $4.65 of compute. It learned to manage its own memory, and to edit at the cache-friendly tail, but it got there without the efficiency reward. Why that happened says something useful about when the paper’s Eq. 6 can matter.

#Table of contents

#Setting the stage: the paper in three moves

Nathan Lambert, one of the paper’s coauthors, summarized the work in a short post. His framing is a good on-ramp, so I use it as the spine of this opening before the section-by-section dissection. He makes three moves. First, today’s agent harnesses grow a transcript and then compact it. Second, CLMs instead let the model edit its own context while it works. Third, that change turns context management into something reinforcement learning can train. I take them in that order, because each one depends on the one before it.

[
  { "role": "system",    "content": "You are a research agent. Tools: search, read_doc." },
  { "role": "user",      "content": "Who founded the award launched in 2011 ...?" },
  { "role": "assistant", "content": "", "tool_calls": [{ "name": "search", "args": { "q": "award launched 2011" } }] },
  { "role": "tool",      "content": "1. ... 2. ... (10 snippets)" },
  { "role": "assistant", "content": "", "tool_calls": [{ "name": "read_doc", "args": { "id": 58939 } }] },
  { "role": "tool",      "content": "(up to 8K tokens of document text)" }
]

messages = [system_prompt, task]
for turn in range(MAX_TURNS):
    reply = llm(messages)            # the whole list, every time
    messages.append(reply)
    if not reply.tool_calls:
        break                        # final answer
    for call in reply.tool_calls:
        messages.append(tool_result(run(call)))

#Move 1: How agents manage context today (append, then compact)

An agent’s context is a list of messages. At every turn, the harness appends three kinds of things: the model’s reply, the tool calls it made, and the results of those calls. Nothing is ever removed on its own. Stripped to its core, the loop in most harnesses looks like this:

messages = [system_prompt, task]
while not done:
    reply = llm(messages)
    messages.append(reply)
    for call in reply.tool_calls:
        messages.append(tool_result(run(call)))

    # the only context-management step: a fixed rule owned by the harness
    if count_tokens(messages) > 0.75 * BUDGET:
        summary = llm(messages + [SUMMARIZE_PROMPT])
        messages = [system_prompt, task, summary]

Context grows fast. In the paper’s deep-research setup, one search call returns ten 512-token snippets, so a handful of searches fills a 32K window. When the transcript crosses a threshold, the harness asks the model to summarize everything and replaces the history with that summary. Cursor, Codex, and Terminus 2 all work roughly this way, and the paper’s “Summary” baseline uses Codex’s summarization prompts and compacts at 75% of the budget. Plotted over a long run, context size makes a sawtooth:

tokens
32K |      /|       /|       /|
    |     / |      / |      / |
    |    /  |     /  |     /  |
    |   /   |    /   |    /   |
    |  /    |   /    |   /    |
    | /     |__/     |__/     |__
    +--------------------------------> turns
         ^ compact    ^ compact

Research has given the model progressively more say in this process. MEM1 rewrites the context every turn. Self-Compact and AutoCompact let the model decide when to compact. ACM adds offloading and retrieval, and Context Folding lets the model branch into a sub-task and fold it back into a summary. As the paper puts it, the model gains autonomy at each step, but it only ever chooses from a menu of operations that humans defined. (The refresher on compaction, offloading, and retrieval unpacks each of these moves and how it fails.)

Why models “drop things” when they work too long. I see three structural reasons:

1. Compaction is one global, lossy rewrite made under pressure. At the moment the threshold hits, the model has to guess what the rest of the task will need. Anything that doesn’t make it into the summary is gone for good.
2. Summaries can be wrong, not just short. On the paper’s diagnostic benchmark (ContextBench, §3), summary-based compaction loses or hallucinates information that the task explicitly asked it to keep.
3. Between compactions, nothing is cleaned up or updated in place. Stale tool output sits in the context until the next threshold. To change one cell of a board it’s tracking, a harness without in-place editing has to regenerate the entire board.

In all of these approaches, the schedule and the procedure belong to the harness. The model only fills in the blanks.

#Move 2: What CLMs do instead (edit the context while working)

Nathan’s one-line version is that CLMs “edit the context directly while they’re working.” Formally, a standard LM can only append, $c_{t+1} = c_t \oplus f_\theta(c_t)$, while a CLM produces the next context itself, $c_{t+1} = f_\theta^{\text{CLM}}(c_t)$. (I dissect this formalism in §4.1.)

The implementation is refreshingly plain. The harness mirrors the live context into a file (in the paper’s prompts, /tmp/.live_ctx/LIVE_CTX_MAIN.txt) and puts the path in the system prompt. The model edits that file with ordinary Bash or Python, the same way it edits any other file. After each turn, the harness syncs the file back as the model’s next context. If the model doesn’t touch the file, the new tokens are appended as usual, so a standard LM is simply a CLM that never edits. A simplified sketch (see the paper’s repo for the exact semantics):

CTX = "/tmp/.live_ctx/LIVE_CTX_MAIN.txt"
context = [system_prompt_with_path(CTX), task]
write_file(CTX, render(context))

while not done:
    reply = llm(context)
    results = [run(call) for call in reply.tool_calls]  # bash here may rewrite CTX

    edited = read_file(CTX)
    if edited != render(context):
        context = parse(edited)                  # the model's rewrite IS the new context
    else:
        context = context + [reply] + results    # default: plain append
    write_file(CTX, render(context))

The thresholded compaction rule is gone from the loop, replaced by a file the model can read and write.

What this looks like in practice. Take the Sudoku task from ContextBench, where the agent must maintain a 16×16 board as moves stream in. A summary harness has to re-emit the whole board to record a single move. A CLM makes a surgical edit (an illustrative example, not a paper trace):

python3 - <<'EOF'
import pathlib, re
p = pathlib.Path("/tmp/.live_ctx/LIVE_CTX_MAIN.txt")
s = p.read_text()
s = s.replace("(1,1,.)", "(1,1,9)", 1)          # place 9 in r1c1
s = re.sub(r"VERSION: 0\b", "VERSION: 1", s, count=1)
p.write_text(s)
EOF

The behaviors the paper observed in real runs (its Figure 3) go further than anything the harness designers specified:

• Pruning with loops. On BrowseComp-Plus, the model writes a for loop over past turns that replaces irrelevant search results with a one-line stub: Searched: X. No relevant results.
• Defining its own tools. It defines a compact_turns helper that collapses old search results into a pointer to a progress note it maintains, then calls that function 37 times over the run.
• Inventing structure. It creates a new notes role alongside the chat template’s built-in roles, to hold its own working notes.
• Orchestrating subagents. While coordinating subagents on the Erdős minimum-overlap problem, it keeps a live scoreboard in its context and updates it through 163 in-place edits. Its context stays at 6–8K tokens throughout, so instead of a sawtooth you get a flat band.

“A more flexible space of harnesses,” and the Bitter Lesson. Every strategy from Move 1 can be written as a program over the context file. Summarization is rewriting the whole file. Offloading is moving a block to disk and leaving a pointer. MEM1 is rewriting the file every turn. So a CLM contains all of those harnesses as special cases, and it’s also free to invent strategies nobody wrote down. That’s the Bitter Lesson argument (Sutton, 2019): methods that lean on search and learning eventually beat hand-built human knowledge. Here that means we stop hand-designing context policy, give the model a general-purpose tool, and let it find the policy. The policy is also steerable in plain English. When told “compact to 4K once you reach Y tokens,” the model’s first compaction lands at 16.0K, 23.6K, and 30.9K for Y = 16K, 24K, and 32K.

My caveat. “No harness” really means “a minimal harness.” The CLM setup still reports the context size in every tool result, sends a reminder 2,048 tokens before the budget runs out, and rolls back and retries when a request would overflow. The approach also depends on model capability. A 9B model edits its context only 1.4 times per task on TerminalBench, and in the RL experiment’s evaluation it starts six points behind the summary harness (though the main zero-shot evaluation has it slightly ahead; §4.1 looks at that discrepancy). The Bitter Lesson applies here only when the model is strong enough to search the bigger space it has been given.

Where that leaves my leash. In The Leash and the Wolf, I argued that every agent needs a deterministic orchestrator. CLMs don’t refute that argument; they move the leash. The orchestrator no longer decides what the agent keeps; the model does. What stays outside the model is exactly the list in my caveat above: the budget readout in every tool result, the reminder before the limit, and the rollback when a request would overflow. That is the new leash. It’s shorter and lighter, but it hasn’t gone away.

#Move 3: Why this matters for reinforcement learning

With harness-scheduled compaction, RL can only teach the model to write better summaries on a schedule it doesn’t control. That’s what Cursor’s Composer training, CompactionRL, and MEM1 do. With a CLM, when to edit, what to keep, and how to restructure are all just tokens the model generates. That means an outcome reward can train them like any other action.

There is one technical wrinkle. Edits break the append-only transcript: once the model rewrites its context, the final transcript no longer contains the inputs its earlier actions actually saw. The fix is to store every model call as its own segment, keeping the exact input it saw and the output it produced. Each segment is then trained under its own input, and every segment gets the whole trajectory’s GRPO advantage. The paper calls this stepwise GRPO.

Outcome reward alone is weak supervision for editing, and rewarding edits directly invites reward hacking, where the model deletes things to look efficient. So the paper adds a success-gated efficiency advantage. Within each group of rollouts, only successful trajectories are compared on inference cost, and the cheaper ones get a bonus. Failed trajectories get no efficiency credit, so the model can’t win by giving up early. In training, this bonus is weighted at 0.25 and applied to the context-management tokens. I work through the math in §4.2.

The results. The authors trained Qwen3.5-9B for 70 steps on 3,040 OpenResearcher deep-research prompts and evaluated on held-out BrowseComp-Plus:

This table is what grounds Nathan’s “inference efficiency” claim. CLM starts six points behind, because the small model is weak at managing context zero-shot. RL closes that gap entirely, and CLM ends at the same accuracy as the RL-trained summary harness while using about 39% fewer FLOPs per question.

My calibration of “massively”: the robust result is accuracy parity at roughly 39% lower inference cost. The 0.4-point accuracy lead is well within noise; the standard error is about 1.7 points on 830 questions. Measured against its own starting point, CLM’s cost fell 12%. Cheaper trajectories should also make the RL rollouts themselves cheaper, but the paper doesn’t measure training-time compute, so that’s a plausible implication rather than a reported result.

def offload(block_id, text):
    path = f"/tmp/store/{block_id}.txt"
    write_file(path, text)
    return f"[{block_id}: {count_lines(text)} lines saved to {path}; grep to recall]"

#§1 Introduction: the claim and the “Bitter Lesson” framing

The introduction makes its case in two moves. First, it describes the state of the art: context management either lives in the harness, as a fixed schedule and procedure, or it’s handed to the model as a small, fixed set of tools for compaction, offloading, and retrieval (see the refresher above). Second, it proposes removing the restriction entirely: give the model unrestricted access to its own context, and let better strategies emerge.

The authors frame this as an instance of the Bitter Lesson. Here’s how the claim holds up against the four-question checklist from the refresher:

1. Does it improve with scale? There’s partial evidence. Gains grow with model capability: Qwen3.6-27B edits its context 2.6 times per task on TerminalBench against 1.4 for Qwen3.5-9B, and it keeps a much leaner peak context (17.6K vs 30.2K tokens). But the evidence spans only a few models and a single RL run, which is a narrow slice of “scale.”
2. Which human priors were removed, and which remain? The compaction schedule and the fixed tool menu are gone. What remains: a skill document describing the mechanism, context-size readouts in every tool result, a budget reminder, rollback-and-retry, and an exemption from the turn cap for edits. Appendix G motivates the readouts honestly: current models can’t reliably estimate their own context length. The human design moved; it didn’t vanish.
3. Is the compute comparison fair? Largely yes, and this is the paper’s strength. The cost metric charges for the re-prefilling that edits cause (see the next section), and the Software World comparison is at equal spend.
4. Search, learning, or both? Both. Zero-shot use and skill evolution are search, in context and over text; the RL experiment is learning.

My read: this paper earns its Bitter Lesson framing more than most papers that cite Sutton, because it measures cost honestly and shows both search and learning at work. The scale evidence is still thin.

previous turn : [ A ][ B ][ C ]
append        : [ A ][ B ][ C ][ D ]    reuse A, B, C  ->  prefill only D
edit middle   : [ A ][ B'][ C ]         reuse A        ->  re-prefill B' AND C
edit start    : [ A'][ B ][ C ]         reuse nothing  ->  re-prefill everything

#§1 (cont.) The serving problem, sentence by sentence

Most coverage of this paper focuses on the model: let it edit its own context. But tucked into the introduction are three sentences that carry the systems half of the argument, and they’re dense enough to deserve one at a time. Each gets the basics, a worked example, and a program you can run. Each program has View and Download links just above it, and the full, tested version of the cost model lives in my Tiny CLM repo.

#Sentence 1: edits break the cache

Serving engines such as SGLang and vLLM are built around one assumption: conversations grow by appending. An agent resends its whole history every turn, so the engine keeps the keys and values (the KV cache) it computed last time and reuses them for any prompt that starts with the same tokens. (If KV caching is new to you, the refresher just above walks through it.)

The reuse stops at the first token that differs, and that rule is forced by the math, not a lazy implementation choice. A token’s keys and values at every layer above the first depend on every token before it, so once position $m$ changes, every cached entry after $m$ describes a context that no longer exists.

“Arbitrary” is the operative word in the paper’s sentence. A CLM can edit anywhere, in several places, at any turn. That’s exactly the pattern a cache organized around shared prefixes handles worst. Here’s a toy version of what the server sees:

# Toy prefix cache: how many prompt tokens can the server reuse from the previous turn?
# Segments stand in for messages; each has an id and a token length.

def reused_prefix_tokens(prev, curr):
    """Standard prefix caching: reuse stops at the first segment that differs."""
    reused = 0
    for (pid, plen), (cid, clen) in zip(prev, curr):
        if pid != cid:
            break
        reused += clen
    return reused

prev = [("SYS", 1000), ("TASK", 300), ("OUT1", 400), ("OBS1", 5120),
        ("OUT2", 400), ("OBS2", 5120), ("OUT3", 400), ("OBS3", 5120)]

cases = {
    # the next turn just adds the model's reply and a new tool result
    "append":      prev + [("OUT4", 400), ("OBS4", 5120)],
    # the model replaced OBS1 (an old search result) with a 150-token note
    "edit middle": prev[:3] + [("NOTE1", 150)] + prev[4:] + [("OUT4", 400), ("OBS4", 5120)],
    # the model rewrote a status block right after the system prompt
    "edit start":  prev[:1] + [("STATUS", 300)] + prev[1:] + [("OUT4", 400), ("OBS4", 5120)],
}

for name, curr in cases.items():
    total = sum(n for _, n in curr)
    reused = reused_prefix_tokens(prev, curr)
    print(f"{name:12s} prompt={total:6,d}  reused={reused:6,d}  must prefill={total - reused:6,d}")

append       prompt=23,380  reused=17,860  must prefill= 5,520
edit middle  prompt=18,410  reused= 1,700  must prefill=16,710
edit start   prompt=23,680  reused= 1,000  must prefill=22,680

Look at the middle row. The edit made the prompt about 5,000 tokens shorter, yet the server must prefill three times as many tokens as for a plain append, because everything after the edit point has to be recomputed even though it didn’t change. Edit near the top, and you pay for nearly the whole prompt. Hybrid models make this slightly worse: their linear-attention layers keep a recurrent state that SGLang saves only at request boundaries, so an edit inside a cached span can push the reusable prefix back even further (Appendix B).

#Sentence 2a: a cost metric that sees re-prefilling

If edits have a hidden cost, the obvious metrics hide it too. Tokens in context ignores the re-prefill bill. Tokens generated ignores prefill entirely. Wall-clock time depends on hardware and batching. So the paper defines prefix-reuse FLOPs: the theoretical compute a server with standard prefix caching would spend, summed over the whole trajectory, every turn of an agent run. In words (Eq. 3):

To make that computable, Appendix C splits a model’s work into two kinds. Linear work (the MLPs and all the projections) costs the same for every token processed, whatever the context length; call it $C_{\text{token}}$ per token (Eq. 7). Attention work grows with context: each query–key pair costs $C_{\text{attn}}$ (Eq. 8).

Now take one turn. The prompt has $P_t$ tokens, of which $R_t$ match a cached prefix, so the server prefills $U_t = P_t - R_t$ tokens. The model then generates $G_t$ tokens. The turn costs:

Each bracket term is a sum you can check by hand. Prefilled token $i$ attends to about $i$ keys, and $\sum_{i=R_t+1}^{P_t} i \approx \tfrac12(P_t^2 - R_t^2)$. Generated token $j$ attends to the prompt plus the $j$ tokens before it, and $\sum_{j=1}^{G_t} (P_t + j) \approx G_t P_t + \tfrac12 G_t^2$. The trajectory’s cost is $\sum_t F_t$. One subtlety: a response is decoded when it’s generated and prefilled again when it first appears in the next prompt, because servers cache prompts, not outputs.

Here’s Appendix C as code, with Qwen3.6-27B’s constants. It reproduces the paper’s Figure 14 exactly:

# Prefix-reuse FLOPs (Appendix C of the CLM paper), with Qwen3.6-27B constants.

L, d, d_ff = 64, 5120, 17_408
L_attn, h_q, h_kv, d_h = 16, 24, 4, 256          # full-attention layers
L_lin, h_k, h_v, d_k, d_v = 48, 16, 48, 128, 128  # Gated DeltaNet layers

# Eq. 7: cost per processed token, independent of context length (2 FLOPs per multiply-add)
mlp       = 6 * L * d * d_ff
attn_proj = L_attn * (2 * d * (2 * h_q * d_h + 2 * h_kv * d_h) + 2 * h_q * d_h * d)
lin_proj  = L_lin * (2 * d * (2 * h_k * d_k + 2 * h_v * d_v + 2 * h_v) + 2 * h_v * d_v * d)
C_token = mlp + attn_proj + lin_proj

# Eq. 8: cost per query-key pair (score + weighted value, per head dimension)
C_attn = 4 * L_attn * h_q * d_h

def turn_flops(P, R, G):
    """Eq. 9: prompt of P tokens, R of them reused from cache, G generated tokens."""
    U = P - R
    prefill_attn = 0.5 * (P**2 - R**2)       # each prefilled token i attends to ~i keys
    decode_attn  = G * P + 0.5 * G**2        # each generated token attends to the prompt + earlier outputs
    return C_token * (U + G) + C_attn * (prefill_attn + decode_attn)

if __name__ == "__main__":
    print(f"C_token = {C_token/1e9:.2f} GFLOPs/token  (MLP {mlp/1e9:.2f}, attn proj {attn_proj/1e9:.2f}, DeltaNet proj {lin_proj/1e9:.2f})")
    print(f"C_attn  = {C_attn:.3g} FLOPs per query-key pair")
    print(f"C_token / C_attn = {C_token / C_attn:,.0f}\n")

    # Figure 14: P = 20,000 prompt tokens, G = 500 generated, three reusable prefixes
    no_cache = turn_flops(20_000, 0, 500)
    for label, R in [("append only", 18_000), ("edit mid-context", 10_000), ("edit at start", 0)]:
        F = turn_flops(20_000, R, 500)
        print(f"{label:17s} R={R:6,d}  F={F/1e14:5.2f}e14 FLOPs  "
              f"({F / turn_flops(20_000, 18_000, 500):.1f}x append, {1 - F/no_cache:.0%} avoided)")

C_token = 48.70 GFLOPs/token  (MLP 34.23, attn proj 3.36, DeltaNet proj 11.12)
C_attn  = 3.93e+05 FLOPs per query-key pair
C_token / C_attn = 123,853

append only       R=18,000  F= 1.41e14 FLOPs  (1.0x append, 87% avoided)
edit mid-context  R=10,000  F= 5.74e14 FLOPs  (4.1x append, 47% avoided)
edit at start     R=     0  F=10.81e14 FLOPs  (7.7x append, 0% avoided)

Two things are worth noticing. First, the cost ladder: a mid-context edit makes a turn 4.1× as expensive as an append, and an edit at the start 7.7×. Second, the ratio $C_{\text{token}} / C_{\text{attn}} \approx 124{,}000$. Re-prefilling one token costs as much as about 124,000 query–key pairs. At a 20K-token context, each new token’s attention costs about $3.9\times10^5 \times 20{,}000 \approx 7.9$ GFLOPs, roughly a sixth of its 48.7 GFLOPs of linear work. On this hybrid model (48 of 64 layers use linear attention), prefix-reuse FLOPs is mostly a count of tokens processed: decoded, prefilled, and re-prefilled. Keep that in mind whenever a FLOPs number appears in this paper.

The metric is honest about what it is: theoretical compute. It leaves out memory bandwidth (decode is memory-bound, so wall-clock time behaves differently), KV-cache memory, and batching effects.

#Sentence 2b: CLMs stay cheaper, even with the penalty

The claim is that even when you pay the full re-prefill penalty on ordinary serving, CLMs come out cheaper overall. The paper’s numbers back it: on BrowseComp-Plus, CLM uses 21.5% fewer prefix-reuse FLOPs than Codex-style summarization and 28.9% fewer than MEM1. On TerminalBench 2.1 it uses 70% of the summary harness’s FLOPs, and on TBLite 91%.

The gap is widest on long runs: on 12-hour EdgeBench, CLM used 179 PFLOPs per trial against summarization’s 437.

How can a method built on expensive edits be cheaper? It helps to ask when an edit pays for itself. Suppose an edit removes $\Delta$ tokens and leaves $S$ unchanged tokens after the edit point. The edit costs about $C_{\text{token}}\,S$ in re-prefill. Each future token processed then attends to $\Delta$ fewer keys, saving $C_{\text{attn}}\,\Delta$. The break-even point comes after

Deleting a 5,120-token search result that has 10,000 tokens after it needs about 250,000 future tokens to break even, roughly 45 turns of a typical research agent. Deleting the result you just read, right at the tail ($S \approx 0$), pays off immediately. So in FLOPs terms, where you edit matters more than how much you delete.

To see this play out over a whole run, here’s a toy simulator. Each turn the agent emits 400 tokens and receives a 5,120-token search result, for 30 turns, under a 32K budget. It compares four policies:

• append-only: never manages context.
• summary: compacts to a 1,500-token summary at 75% of the budget, like the paper’s Codex-style baseline.
• tail-prune: a CLM-like policy that replaces each search result with a 150-token note right after reading it.
• head-status: the same, but it also rewrites a status block near the top of the context every turn.

The simulator also computes each policy’s cost under Suffix Cache Reuse, which comes up in Sentence 3. It models cost only, not accuracy:

# Toy trajectory simulator: total prefix-reuse FLOPs for three context policies,
# under standard prefix caching and under Suffix Cache Reuse (SCR).
# Illustrative only: it models cost, not accuracy, and the lengths are made up but realistic.
from prefix_reuse_flops import C_token, C_attn

BUDGET, TURNS = 32_768, 30
SYS, TASK = ("SYS", 1000), ("TASK", 300)
OUT, OBS, NOTE, STATUS, SUMMARY = 400, 5120, 150, 300, 1500

def reuse_plan(prev, curr, scr=False, K=6):
    """Return the set of positions (segment indices in curr) whose cache can be reused."""
    reused, i = set(), 0
    while i < min(len(prev), len(curr)) and prev[i] == curr[i]:
        reused.add(i); i += 1                     # standard: the matching prefix
    if scr:                                       # SCR: also relocate surviving spans after the edit
        survivors = set(prev[i:])
        spans, run = [], []
        for j in range(i, len(curr)):
            if curr[j] in survivors:
                run.append(j)
            elif run:
                spans.append(run); run = []
        if run: spans.append(run)
        spans.sort(key=lambda s: -sum(curr[j][1] for j in s))
        for s in spans[:K]:                       # cap at K relocated spans, largest first
            reused.update(s)
    return reused

def call_flops(prev, curr, G, scr):
    reused = reuse_plan(prev, curr, scr)
    flops, pos = 0.0, 0
    for j, (_, n) in enumerate(curr):
        if j not in reused:                       # prefill this segment: linear cost + attention to all earlier tokens
            flops += C_token * n + C_attn * (n * pos + 0.5 * n * n)
        pos += n
    P = pos
    flops += C_token * G + C_attn * (G * P + 0.5 * G * G)   # decode
    return flops

def run(policy, scr=False):
    ctx, prev, total, peak, calls = [SYS, TASK], [], 0.0, 0, 0
    for t in range(1, TURNS + 1):
        if policy == "head-status":               # rewrites a status block near the top every turn
            ctx = [ctx[0], (f"STATUS{t}", STATUS)] + [s for s in ctx[1:] if not s[0].startswith("STATUS")]
        if policy == "summary" and sum(n for _, n in ctx) + OUT > 0.75 * BUDGET:
            total += call_flops(prev, ctx, SUMMARY, scr); calls += 1
            prev, ctx = ctx, [SYS, TASK, (f"SUM{t}", SUMMARY)]
        if sum(n for _, n in ctx) + OUT > BUDGET:
            return total, peak, calls, f"overflow at turn {t}"
        total += call_flops(prev, ctx, OUT, scr); calls += 1
        peak = max(peak, sum(n for _, n in ctx))
        prev = ctx
        ctx = ctx + [(f"OUT{t}", OUT), (f"OBS{t}", OBS)]
        if policy in ("tail-prune", "head-status") and t > 1:
            # after reading OBS(t-1) last turn, the model replaced it with a short note
            ctx = [(f"NOTE{t-1}", NOTE) if s[0] == f"OBS{t-1}" else s for s in ctx]
    return total, peak, calls, "finished"

print(f"{'policy':12s} {'standard':>11s} {'with SCR':>11s} {'peak ctx':>9s} {'calls':>6s}  outcome")
for policy in ["append-only", "summary", "tail-prune", "head-status"]:
    std, peak, calls, outcome = run(policy)
    scr, *_ = run(policy, scr=True)
    print(f"{policy:12s} {std/1e15:8.1f} PF {scr/1e15:8.1f} PF {peak:9,d} {calls:6d}  {outcome}")

policy          standard    with SCR  peak ctx  calls  outcome
append-only       1.7 PF      1.7 PF    28,900      6  overflow at turn 7
summary          10.5 PF     10.5 PF    23,380     37  finished
tail-prune        9.5 PF      9.5 PF    22,220     30  finished
head-status      21.6 PF      9.9 PF    22,520     30  finished

Read the rows top to bottom:

• append-only looks cheap only because it dies at turn 7. That’s why context management exists at all.
• summary is surprisingly cheap in FLOPs. Each compaction rewrites almost everything, but leaves very little behind to re-prefill. Its real cost is the one this toy can’t see: lost details and the rework they cause. EdgeBench’s 437 vs 179 PFLOPs hints at exactly that.
• tail-prune is the cheapest, because every edit lands where nothing after it needed caching anyway.
• head-status does almost the same work as tail-prune, but because its edits sit near the top it costs more than double.

So “better context management” in this sentence has a specific meaning: edits placed where the cache can survive them, plus shorter trajectories. The property isn’t automatic. A CLM that edits carelessly near the top would lose this argument, which is why the paper’s RL reward (§4.2) explicitly rewards cheaper trajectories.

Aside: on a hosted API. Provider prompt caching is also prefix-based, so the same geometry applies, but the economics change: you pay for every input token on every call, cached or not; caching only discounts it. Shrinking the context therefore pays back much faster in dollars than in FLOPs. (Check your provider’s current cache-read and cache-write pricing to compute your own break-even.)

#Sentence 3: Suffix Cache Reuse

If edits are expensive because the cache stops at the first mismatch, the natural fix is to reuse the cache beyond the mismatch. Take a context $[A\,B\,C]$ where an edit replaces $B$ with $B'$. Standard serving reuses $A$ and re-prefills $B'$ and $C$. Suffix Cache Reuse (SCR) reuses $A$, prefills only $B'$, and moves $C$’s cached entries to their new positions:

standard serving  : [ A: reuse ][ B': prefill ][ C: re-prefill        ]
suffix cache reuse: [ A: reuse ][ B': prefill ][ C: reuse, relocated  ]

Mechanically, SCR diffs the new prompt against the session’s previous one, finds the spans that survived the edit, and relocates up to $K = 6$ of them, largest first. For each span it re-rotates the cached keys’ positional encodings to their new positions and splices them in after $B'$. Relocated entries live in session-private cache slots, and the server falls back to standard re-prefilling if those can’t be allocated.

Why moving a cached key is legitimate. With rotary position embeddings (RoPE), a key at position $p$ is the raw key rotated by an angle proportional to $p$: $k_p = R(p)\,k$. Rotations compose, $R(p + \delta) = R(\delta)\,R(p)$, so moving a key by $\delta$ positions is one extra rotation of the cached value, and attention scores depend only on relative position. A quick check:

# Why Suffix Cache Reuse can move cached keys: RoPE rotations compose exactly.
import numpy as np

def rope(x, pos, base=10_000.0):
    """Rotate consecutive pairs of x by angles pos * theta_i (standard RoPE)."""
    d = x.shape[-1]
    theta = base ** (-np.arange(0, d, 2) / d)
    ang = pos * theta
    cos, sin = np.cos(ang), np.sin(ang)
    x1, x2 = x[0::2], x[1::2]
    out = np.empty_like(x)
    out[0::2] = x1 * cos - x2 * sin
    out[1::2] = x1 * sin + x2 * cos
    return out

rng = np.random.default_rng(0)
k_raw = rng.standard_normal(256)          # a key before positional rotation (head dim 256)

old_pos, new_pos = 9_120, 4_150           # an edit earlier in the context removed 4,970 tokens
cached   = rope(k_raw, old_pos)           # what the KV cache holds for this token
moved    = rope(cached, new_pos - old_pos)   # SCR: re-rotate by the shift only
fresh    = rope(k_raw, new_pos)           # what a full re-prefill would compute (same content)

print("max |moved - fresh| =", np.abs(moved - fresh).max())   # ~1e-12: position is fixed exactly

# Attention scores depend only on relative position, so shifting both query and key changes nothing:
q_raw = rng.standard_normal(256)
s1 = rope(q_raw, 10_000) @ rope(k_raw, 9_120)
s2 = rope(q_raw, 10_000 - 4_970) @ rope(k_raw, 9_120 - 4_970)
print("score before shift:", round(s1, 6), " after shift:", round(s2, 6))

max |moved - fresh| = 6.460942891806098e-13
score before shift: 5.934255  after shift: 5.934255

Why it’s still an approximation. The rotation fixes the position exactly, but not the content. $C$’s cached states above the first layer were computed while $B$, not $B'$, sat before them. In the model’s linear-attention layers it goes further: SCR resumes from the recurrent state saved before the edit, so in 48 of Qwen3.6-27B’s 64 layers $B'$ is never processed at all.

The paper presents this staleness as possibly helpful, since it keeps richer information from the past. I’d frame it as a property to understand before deploying. If an agent deletes sensitive or regulated data, or a prompt-injected instruction, from its context file, traces of it can persist in the reused serving state for the rest of the session. The paper’s own discussion section flags editable context as a new safety surface; SCR adds a second, quieter one. Note also that SCR is a serving technique: it helps only if you run your own inference stack, not when you call a hosted API.

“Empirically preserving task performance.” To limit the compounding of approximations, SCR relocates at most $K = 6$ spans per edit and re-prefills the rest. A sensitivity study over $K \in \{1, 2, 3, 6, 12, 64\}$ found no accuracy degradation, but it used only 64 questions, with standard errors around five points. That’s thin evidence on its own.

The main result is stronger: across all 830 BrowseComp-Plus questions, accuracy is 60.2% with or without SCR, while compute drops from 10.98 to 7.14 PFLOPs per question. SCR does the same job at 65% of the cost, which is the “35% less server compute” in the abstract.

Now look back at the simulator’s SCR column. It rescues the careless head-status policy (21.6 down to 9.9 PFLOPs) but does nothing for summary or tail-prune, whose edits never strand a reusable suffix. The paper finds the same pattern from a different angle. Of the 7.8% of prompt tokens SCR reuses beyond prefix hits on BrowseComp-Plus, 5.3 points come from reasoning-token stripping, where chat templates delete earlier reasoning blocks in ordinary chat serving, and only 2.5 from the model’s own edits. SCR is a good serving idea in general, and its benefit to CLMs specifically is the smaller part of its story.

#What these three sentences add up to

1. Edits have a real, position-dependent cost under today’s serving, and a fair evaluation has to charge it. The paper does, which is to its credit.
2. Under that honest metric, CLMs still win on cost, mostly by keeping trajectories short and editing where the cache can absorb it, not by shrinking attention.
3. SCR lowers the penalty further, at the price of an approximation, stale states, that’s empirically harmless on accuracy here but deserves a security review before it touches sensitive data.

#§1 (cont.) What CLMs do, how they were tested, and how they learn

#Sentence 4: emergent behaviors

“Emergent” here has a precise meaning: nobody told the model to do these things, and no tool offers them. The CLM setup gives the model only one fact about its context, which the paper’s skill puts in a single line: the context is mirrored to a file, and changing the file changes what the model holds. Everything else is the model’s own idea.

How the authors found them. These are qualitative observations, collected from the zero-shot evaluation runs in §5.1 and shown in Figure 3 as code excerpts labeled with the task and step number. The paper reports no frequencies: we don’t learn how often each behavior occurs, or whether runs that use it score better. Read them as demonstrations of what the action space makes possible, not as measured effects.

1. Trackers for multi-agent orchestration. While coordinating subagents, the model keeps a status block in its context (“STATE”, or “ORCHESTRATOR STATE”) and rewrites it in place as workers report back: best score so far, how many agents are running, key files, dead ends, what to try next. In one run it made 163 such edits while its context stayed between 6K and 8K tokens.

The difference from an append-only agent is the difference between a whiteboard and a notebook. A notebook can only add pages, so the agent’s picture of the world is scattered across dozens of reports. A whiteboard is erased and rewritten, so there’s always one current picture. Note the “dead ends” line in the circle-packing example: recording what didn’t work is how an agent avoids repeating itself after older details are gone.

2. A new role for internal notes. A chat template defines a fixed set of roles: system, user, assistant, tool. When the model rewrites part of its context file, it labels the replacement role=notes, a role the template never defined, and uses it to hold its working status (verified facts, the current candidate, the next step). This separates durable state from the transient transcript around it. One question I’d put to the authors or the code: how does the harness render an unknown role when the file is converted back into model input? It presumably passes through as plain text, but the paper doesn’t say.

3. Reusable context-management functions. The model defines a helper, compact_turns, that collapses old search results into a one-line pointer to its progress note, then calls it 37 times over the run. This is the paper’s “meta-capability” point from §4.1: instead of choosing from operations the harness predefined, the model defines its own operations. For models trained heavily on code, writing a function the second time you need something is the natural move.

The model also rediscovers the classic strategies. It writes answer-focused summaries, compressing 21K tokens into a few lines, and it keeps a ledger of untried ideas. The baselines’ behaviors are still available to it; they’re just no longer mandatory.

Here are all three behaviors reconstructed on a toy context file, using the paper’s turn-marker format. The text the “model” writes is mine; the mechanics, plain Python editing a file, are the point:

# Reconstructing the three emergent behaviors from Figure 3 on a toy context file.
# The [[CTX_TURN n role=...]] markers follow the format visible in the paper's examples.
# What the model writes is illustrative; the mechanics (plain Python editing a file) are the point.
import re

def tokens(s):                      # rough token estimate: about 4 characters per token
    return len(s) // 4

def turn(n, role, body):
    return f"[[CTX_TURN {n} role={role}]]\n{body}\n"

search_dump = "\n".join(f"{i}. snippet about unrelated award #{i} " + "lorem ipsum " * 40 for i in range(10))
ctx = turn(0, "system", "You are a research agent. Your live context is the file LIVE_CTX.")
ctx += turn(1, "user", "Which award, launched in 2011, matches all the clues in the question?")
for n in range(2, 8):
    ctx += turn(n, "assistant", f"search(query='award 2011 attempt {n}')")
    ctx += turn(n, "tool", search_dump)
print(f"before: {tokens(ctx):6,d} tokens")

# Behavior 1: define a reusable context-management function, then call it repeatedly.
def compact_turns(text, keep_after):
    """Collapse old tool results to a one-line pointer to the progress note."""
    def stub(m):
        n = int(m.group(1))
        if n >= keep_after:
            return m.group(0)
        return f"[[CTX_TURN {n} role=tool]]\n[search results - see progress note]\n"
    return re.sub(r"\[\[CTX_TURN (\d+) role=tool\]\]\n.*?\n(?=\[\[CTX_TURN|\Z)", stub, text, flags=re.S)

ctx = compact_turns(ctx, keep_after=7)          # the paper's example calls its version 37 times
print(f"after compact_turns: {tokens(ctx):6,d} tokens")

# Behavior 2: invent a new role for internal notes, which the chat template never defined.
notes = turn(1, "notes", "STATUS: candidate = award X (launched 2011). VERIFIED: launch year. NEXT: read doc 58939 to check the remaining clues.")
ctx = ctx.replace("[[CTX_TURN 2 role=assistant]]", notes + "[[CTX_TURN 2 role=assistant]]", 1)

# Behavior 3: keep a tracker for subagents and update it in place instead of appending reports.
tracker = "## STATE\nBEST SO FAR: 0.9931\nAGENTS: 5 launched, 5 running\n"
ctx = ctx.replace("[[CTX_TURN 2 role=assistant]]", turn(1, "notes", tracker) + "[[CTX_TURN 2 role=assistant]]", 1)
for best, running in [(0.9952, 4), (0.9977, 3), (0.9992, 1)]:     # each worker report is an in-place edit
    ctx = re.sub(r"BEST SO FAR: [\d.]+", f"BEST SO FAR: {best}", ctx)
    ctx = re.sub(r"\d+ running", f"{running} running", ctx)

print(f"final: {tokens(ctx):6,d} tokens\n")
print(ctx[:900])

before:  7,940 tokens
after compact_turns:  1,523 tokens
final:  1,581 tokens

[[CTX_TURN 0 role=system]]
You are a research agent. Your live context is the file LIVE_CTX.
[[CTX_TURN 1 role=user]]
Which award, launched in 2011, matches all the clues in the question?
[[CTX_TURN 1 role=notes]]
STATUS: candidate = award X (launched 2011). VERIFIED: launch year. NEXT: read doc 58939 to check the remaining clues.
[[CTX_TURN 1 role=notes]]
## STATE
BEST SO FAR: 0.9992
AGENTS: 5 launched, 1 running

[[CTX_TURN 2 role=assistant]]
search(query='award 2011 attempt 2')
[[CTX_TURN 2 role=tool]]
[search results - see progress note]
[[CTX_TURN 3 role=assistant]]
search(query='award 2011 attempt 3')
[[CTX_TURN 3 role=tool]]
[search results - see progress note]
[[CTX_TURN 4 role=assistant]]
search(query='award 2011 attempt 4')
[[CTX_TURN 4 role=tool]]
[search results - see progress note]
[[CTX_TURN 5 role=assistant]]
search(query='award 2011 attempt 5')
[[CTX_TURN 5 role=tool]]
[s

One function call took the context from about 7,900 tokens to about 1,500, and every later tracker update cost a few characters instead of a new report.

A connection to cost. In this toy, as in the paper’s notes example (which rewrites turns 4 through 16), the notes and the tracker live near the top of the context. Sentence 2b showed that frequent edits near the top are the expensive kind under prefix caching. A cache-aware agent would keep a frequently updated tracker at the tail instead. Whether zero-shot CLMs learn this placement is an open question, and it’s exactly the kind of thing the efficiency reward in Sentence 7 could teach.

#Sentence 5: zero-shot across long horizons, and how the authors set it up

What “zero-shot” means here. No model was trained for context management. Each model is used as-is, wrapped in the CLM harness and given a skill document explaining the context file. Even the search tools for deep research are exposed as in-context skills rather than a fixed tool interface, in keeping with what the authors call a “less is more” principle. So every zero-shot result measures what existing models do when handed the file, nothing more.

The controlled part. All methods share the same agent backbone (Mini-SWE-Agent), the same base model, and the same token budget. Baselines run their original implementations where available:

• Summary uses Codex’s summarization prompts and compacts at 75% of the budget.
• Self-Compact asks the model every two turns, once the context passes 37% of the budget, whether to compress.
• RLM and ACM run their released code.
• MEM1 is the authors’ re-implementation.
• Base never manages context.

CLM gets an editing reminder 2,048 tokens before the budget runs out.

The suite. The tasks were chosen to stretch from minutes to a full day:

“Hundreds to thousands of turns.” The long end is real. EdgeBench runs last 12 hours with no turn cap, and Software World runs for over a day. One detail puzzled me, though: Figure 3 shows a BrowseComp-Plus excerpt at step 1509, on a benchmark with a 100-turn cap. The likely explanation is that CLM’s editing turns don’t count toward the cap, which would mean that run spent well over a thousand steps editing. That’s worth confirming, because it changes how “100 turns” compares across methods.

What isn’t controlled, or doesn’t line up.

• Harness features only CLM gets. The turn-cap exemption for edits, rollback-and-retry when a request would overflow, and the budget reminder. The Base harness has no trigger at all.
• Thin statistics in places. The math results are one run per method, and Software World doesn’t state its run count. Treat those as demonstrations. BrowseComp-Plus (830 questions) and EdgeBench (three seeds per task) carry the real statistical weight.
• Two inconsistencies. §5.1.2 says CLM ran with “up to five” concurrent subagents on EdgeBench; Appendix E says up to six. And BrowseComp-Plus is reported “with a 32K context budget” in the main text, but Appendix E specifies a 23,560-token budget. That’s most likely the 32K limit minus reserves for generation and tool output, but the paper doesn’t say.

None of these overturn the results, but a careful reader should know them.

#Sentence 6: steering and evolving context management

Why moving control into the model enables learning at all. If the context policy lives in harness code, changing it is an engineering task: someone edits the compaction rule and redeploys. If it lives in the model’s behavior, it becomes a function of two things the model already learns from: its context (so text can change it, which is in-context learning) and its weights (so training can change it, which is parametric learning). Formally, an instruction or skill document $s$ simply becomes part of the input (Eq. 4):

Steering: one sentence changes the policy. The authors appended a single sentence to the task prompt and compared against a no-instruction control on the same BrowseComp-Plus questions, using Claude 4.6 Sonnet with paired bootstrap confidence intervals:

• Compaction threshold. “Compact to about 4,000 tokens whenever the context exceeds Y tokens.” Measured on 30 long questions, the median context size at the first compaction was 16.0K, 23.6K, and 30.9K for Y = 16K, 24K, and 32K.
• Semantic boundaries. “Only compress at the boundaries of subtasks.” Over 189 sessions of four chained questions, compaction within two turns of a question boundary rose from 0.40 to 0.88.
• Backups. The instruction was to back up the full context file into a dedicated folder before every edit, and never overwrite or delete earlier backups. The fraction of edits preceded by a full backup rose from 0.00 to 0.68, plus 0.09 partial.

This is evidence of controllability: the policy follows instructions without touching the harness or the weights. It isn’t evidence that the steered policies score better. That wasn’t the question being asked here. The backup result is the one I’d take straight to production, since an auditable copy before every self-edit is cheap insurance.

Evolving a skill document. The second half of the sentence treats the context-management skill $s$ as something to optimize (Eq. 5):

The weights stay frozen; only the text changes. The loop follows GEPA-style prompt evolution:

1. The agent runs the current skill on training instances.
2. Four proposer models in parallel each read at least ten rollouts, including successful and failed runs on the same instance, and write at least four full rewrites of the skill, each with a predicted effect on accuracy and cost.
3. A candidate runs on 6 training instances. If it’s at least as accurate as the current frontier, it runs on 12 more, then on a 102-instance development split, the only split used for selection.
4. A lineage stops after five proposals in a row without improvement. The final skill is evaluated once on a held-out test split (102 instances, three seeds).

The selection rules are worth seeing as code. They’re simple, and they shape what “best skill” means:

# The selection rules of the paper's skill-evolution loop (Appendix E), on made-up candidates.
#   Frontier: a skill joins if no EARLIER skill (the starting point included) is
#             at least as accurate AND at least as cheap.
#   Selection: the most accurate skill on the dev split; ties within one standard
#              error are broken by cost.
from math import sqrt

N_DEV = 102  # dev instances per task, one seed

# (name, dev accuracy, cost in PFLOPs per task), in the order the proposers produced them
candidates = [
    ("start (no instruction)", 0.223, 4.10),
    ("r1: offload each batch to disk",  0.480, 3.90),
    ("r1: summarize every 3 ops",       0.300, 3.70),
    ("r2: offload + one-line pointer",  0.700, 3.80),
    ("r2: offload + grep recall",       0.810, 3.95),
    ("r3: offload, verify count, evict", 0.838, 3.72),
    ("r3: same, plus verbose checks",    0.845, 4.30),
]

def se(acc, n=N_DEV):
    return sqrt(acc * (1 - acc) / n)

frontier = []
for i, (name, acc, cost) in enumerate(candidates):
    dominated = any(a >= acc and c <= cost for _, a, c in candidates[:i])
    if not dominated:
        frontier.append((name, acc, cost))

best_acc = max(a for _, a, _ in frontier)
best = min((s for s in frontier if s[1] >= best_acc - se(best_acc)), key=lambda s: s[2])

print("frontier (in proposal order):")
for name, acc, cost in frontier:
    mark = "  <- selected" if (name, acc, cost) == best else ""
    print(f"  {name:34s} acc={acc:.3f} (+/-{se(acc):.3f})  cost={cost:.2f}{mark}")

frontier (in proposal order):
  start (no instruction)             acc=0.223 (+/-0.041)  cost=4.10
  r1: offload each batch to disk     acc=0.480 (+/-0.049)  cost=3.90
  r1: summarize every 3 ops          acc=0.300 (+/-0.045)  cost=3.70
  r2: offload + one-line pointer     acc=0.700 (+/-0.045)  cost=3.80
  r2: offload + grep recall          acc=0.810 (+/-0.039)  cost=3.95
  r3: offload, verify count, evict   acc=0.838 (+/-0.036)  cost=3.72  <- selected
  r3: same, plus verbose checks      acc=0.845 (+/-0.036)  cost=4.30

The starting and selected accuracies (22.3% and 83.8%) are the paper’s KV Store development numbers; the intermediate candidates and all costs are invented for illustration. Notice that the selected skill isn’t the most accurate one: a skill within one standard error of the best wins if it’s cheaper.

Results. In assisted evolution, Qwen3.6-27B is the agent and Claude Fable 5.1 proposes the skills. Starting from no instruction, development accuracy rose from 97.6% to 100% on Needle Retention, 45.3% to 65.8% on Sudoku Sketchpad, 22.3% to 83.8% on KV Store, and 0% to 100% on Log Triage. On the held-out test split, the KV Store skill went from 38.3% to 74.2%. That 35.9-point gain is the headline “up to 35.9 points.” In self-evolution, Opus 5 is both agent and proposer, starts at 94–100% accuracy, and the evolved skills mostly cut cost.

My take.

• The headline is the best case. The only held-out test number reported in the text is KV Store’s. The other three tasks are reported on the development split, which was also used for selection, so they’re optimistic. Notice too that KV Store’s starting accuracy is 22.3% on dev but 38.3% on test, a reminder of how much instance and seed variance there is.
• Assisted evolution is partly distillation. A stronger model writes the strategy that a weaker model then follows. That’s still practically valuable, but it isn’t the smaller model improving itself.
• ContextBench is synthetic, and built around editing. It shows the mechanism works, not how much it helps on natural tasks.
• The practical upshot. A context-management skill is a cheap, versionable text artifact, and you can evolve it on your own failure traces with exactly this loop. I use the same idea in Virtual DA’s self-harness loop.

#Sentence 7: a success-gated efficiency advantage in stepwise GRPO

This sentence packs three ideas: GRPO itself, the “stepwise” variant for agents that edit their own context, and the new efficiency advantage. If GRPO is new to you, start with the policy-gradients-to-GRPO refresher (in “Setting the stage”) and the DeepSeekMath capsule just above. Here’s the short version, then the new math.

GRPO in four lines. For each prompt, sample a group of $G$ complete attempts and score each one, $r_i \in \{0, 1\}$. Normalize within the group: $A_i^{\text{out}} = (r_i - \bar r)/\sigma_r$. Push up the log-probability of every token in attempt $i$ in proportion to $A_i^{\text{out}}$, with PPO-style clipping and a KL leash to a reference model. With binary rewards and a group success rate $\hat p$, this works out to

“Stepwise.” An agent attempt isn’t one response; it’s many model calls. Call $j$ sees context $c_j$ and produces $y_j$, and because the environment doesn’t depend on $\theta$, the policy gradient still decomposes per call:

In an append-only agent, every $c_j$ is a prefix of the final transcript, so you can train on that transcript in one pass. A CLM rewrites its context, so the final transcript no longer contains what earlier calls saw. Stepwise GRPO therefore stores each $(c_j, y_j)$ pair as its own segment, trains each segment under its original input, and gives every segment the attempt’s advantage. ReSum and Sculptor established the same principle for summarization-based agents. Note that nothing is differentiated through the edits: the edit is just part of what the model generated.

One open detail: GRPO’s original objective averages each output’s loss over its own length ($1/|o_i|$). With segments, does that average run per segment or per trajectory? The answer decides whether long, many-call trajectories get more gradient weight, and the paper doesn’t say.

Why a new advantage. Outcome reward alone is weak supervision for editing: successful attempts can contain wasteful edits, and failed attempts can contain good ones. Rewarding edits directly backfires, because the model learns to make pointless edits or delete things it needed. The paper’s fix rewards cost, but only among successes. Let $c_i$ be attempt $i$’s prefix-reuse FLOPs, $\mathcal{G}_g^+$ the successful attempts in group $g$, and $\bar c_g$ their mean cost. Then (Eq. 6):

with $A_i^{\text{eff}} = 0$ for everyone if fewer than two attempts succeed. In training, $w_{\text{eff}} = 0.25$, applied to context-management tokens, alongside penalties for failed tool calls and malformed outputs.

Reading Eq. 6 clause by clause. Five properties follow from the definition, and each is worth deriving:

1. Only successes are ranked by cost. A failure gets no efficiency credit, so the model can’t win by giving up early. That’s the point of the gate.
2. It’s relative and bounded. $(\bar c_g - c_i)/\bar c_g$ is a fraction of the group’s typical cost, so it doesn’t care whether a task costs 1 PFLOP or 100. Because $c_i > 0$, the value is always below 1, so the upper clip never binds. The lower clip binds only for attempts costing more than twice the average.
3. It re-ranks without shifting mass. Before clipping, $\sum_{i \in \mathcal{G}^+} (\bar c_g - c_i) = 0$, so the bonus averages to zero across successes. It reorders successes by cost without changing, on average, how much successes as a group are favored over failures.
4. A success always beats a failure. The weakest success minus the strongest failure is at least $\sqrt{\tfrac{1-\hat p}{\hat p}} + \sqrt{\tfrac{\hat p}{1-\hat p}} - w_{\text{eff}} = \tfrac{1}{\sqrt{\hat p(1-\hat p)}} - w_{\text{eff}} \ge 2 - 0.25 = 1.75$.
5. Only on easy prompts can a success go negative. That requires $\sqrt{(1-\hat p)/\hat p} < w_{\text{eff}} = 0.25$, which means $\hat p > 16/17$. With 32 attempts per group, that’s 31 out of 32. There, a bloated success gets nudged down, which is sensible on a prompt the model has already mastered.

The program below computes the advantages for one group and then checks properties 2, 4, and 5 over 20,000 random groups:

# Stepwise GRPO advantages with the CLM paper's success-gated efficiency advantage (Eq. 6).
import numpy as np

def outcome_advantage(r, eps=1e-8):
    """Standard GRPO: z-score the rewards within the group."""
    r = np.asarray(r, float)
    return (r - r.mean()) / (r.std() + eps)

def efficiency_advantage(r, cost):
    """Eq. 6: among successful rollouts only, reward lower prefix-reuse FLOPs."""
    r, cost = np.asarray(r, bool), np.asarray(cost, float)
    A = np.zeros(len(r))
    if r.sum() < 2:                       # fewer than two successes: no efficiency signal
        return A
    c_bar = cost[r].mean()
    A[r] = np.clip((c_bar - cost[r]) / c_bar, -1, 1)
    return A

W_EFF = 0.25   # the paper's weight, applied only to context-management tokens

# One group of 8 rollouts for a single prompt: did it succeed, and what did it cost (PFLOPs)?
r    = [1,   1,   1,   1,   0,   0,   0,   0  ]
cost = [0.9, 1.2, 1.5, 3.4, 0.6, 1.1, 2.0, 2.5]

A_out, A_eff = outcome_advantage(r), efficiency_advantage(r, cost)
print(" i  success  PFLOPs   A_out   A_eff   A_out + 0.25*A_eff (context-management tokens)")
for i in range(len(r)):
    print(f"{i:2d}  {r[i]:7d}  {cost[i]:6.1f}  {A_out[i]:6.2f}  {A_eff[i]:6.2f}  {A_out[i] + W_EFF * A_eff[i]:8.2f}")

# Property checks over many random groups of 32 (the paper's group size)
rng = np.random.default_rng(0)
worst_gap, flips, max_eff = np.inf, 0, -np.inf
for _ in range(20_000):
    r = rng.random(32) < rng.random()
    if r.all() or not r.any():
        continue                          # dynamic sampling drops groups with no reward variation
    cost = rng.lognormal(0, 0.6, 32)
    A = outcome_advantage(r) + W_EFF * efficiency_advantage(r, cost)
    worst_gap = min(worst_gap, A[r].min() - A[~r].max())   # weakest success vs strongest failure
    flips += (A[r] < 0).any() and r.mean() <= 16 / 17
    max_eff = max(max_eff, efficiency_advantage(r, cost).max())

print(f"\nsmallest gap between any success and any failure: {worst_gap:.2f} (always > 0)")
print(f"groups where a success went negative with success rate <= 16/17: {flips}")
print(f"largest efficiency advantage seen: {max_eff:.2f} (the +1 clip never binds)")

# The one regime where a success can end up with a negative advantage: very easy prompts.
r = [1] * 31 + [0]                    # 31 of 32 succeed
cost = [1.0] * 30 + [2.5, 1.0]        # one success is 2.5x the typical cost
A = outcome_advantage(r) + W_EFF * efficiency_advantage(r, cost)
print(f"31/32 succeed: A_out(success) = {outcome_advantage(r)[0]:.2f}, "
      f"expensive success total = {A[30]:.2f}  (pushed down, still above the failure at {A[31]:.2f})")

 i  success  PFLOPs   A_out   A_eff   A_out + 0.25*A_eff (context-management tokens)
 0        1     0.9    1.00    0.49      1.12
 1        1     1.2    1.00    0.31      1.08
 2        1     1.5    1.00    0.14      1.04
 3        1     3.4    1.00   -0.94      0.76
 4        0     0.6   -1.00    0.00     -1.00
 5        0     1.1   -1.00    0.00     -1.00
 6        0     2.0   -1.00    0.00     -1.00
 7        0     2.5   -1.00    0.00     -1.00

smallest gap between any success and any failure: 1.75 (always > 0)
groups where a success went negative with success rate <= 16/17: 0
largest efficiency advantage seen: 0.95 (the +1 clip never binds)
31/32 succeed: A_out(success) = 0.18, expensive success total = -0.07  (pushed down, still above the failure at -5.57)

The smallest gap observed, 1.75, lands exactly on the bound from property 4, and the 31-out-of-32 case reproduces property 5: the expensive success drops to −0.07, still far above the failure at −5.57.

What’s new compared with earlier work. Conditioning an efficiency reward on success isn’t new by itself. Arora and Zanette penalize length only on correct answers, and DDCA computes a separate length advantage within the correct subset. The difference here is the cost measure: prefix-reuse FLOPs, not length. That charges for re-prefilling, so the reward can teach cache-aware editing, keeping edits near the tail, which a length penalty can’t see (recall the head-status policy from Sentence 2b).

What it bought. Trained on deep-research prompts and evaluated on BrowseComp-Plus, Qwen3.5-9B with CLM went from 28.8% to 42.5% (the “+47.6%” in the abstract is this relative gain) while its cost fell from 1.52 to 1.34 PFLOPs per question. It matched an RL-trained summary harness at 42.1%, using about 39% fewer FLOPs. As discussed in Move 3, the 0.4-point lead is within noise. And Figure 21’s curve for the summary harness with the FLOPs reward deserves a close look against the text’s “no clear loss in accuracy” claim before anyone cites it.

#§4.1 The formalism and context as a file

#Two transition rules

A standard language model, run as an agent, has exactly one way to change its context: add to the end. The paper writes this as Eq. 1,

where $\oplus$ is concatenation and $f_\theta^{\text{LM}}(c_t)$ is what gets appended this turn.

A CLM instead produces the next context directly (Eq. 2):

The paper’s notation is compact, so it’s worth spelling out what it hides. The model is stochastic, and part of the transition is an environment, not the model. A more explicit version: the model samples an output $y_t \sim \pi_\theta(\cdot \mid c_t)$, then an edit operator $\mathcal{E}$, the harness running whatever file edits $y_t$ contains, produces the next context, and the environment appends an observation $o_t$:

That last clause is the paper’s “default append” rule, and it makes the standard LM the special case of a CLM that never edits.

#Two properties append-only gives you for free

Writing it this way exposes two properties of Eq. 1 that Eq. 2 gives up. Between them, they explain most of the paper’s structure.

Property 1: the context only grows. Under Eq. 1, $|c_{t+1}| \ge |c_t|$. If every turn adds at least $\delta$ tokens, a budget $B$ runs out after at most

Plug in the toy simulator’s numbers from earlier: $B = 32{,}768$, $|c_0| = 1{,}300$, $\delta = 5{,}520$ gives $5.7$, so the run fails on the next call, turn 7, exactly what the simulator printed. Every context-management method exists to break this bound.

Property 2: every past context is a prefix of every later one. Under Eq. 1, $c_s$ is a prefix of $c_t$ whenever $s \le t$. Two things depend on that silently:

• Serving. Prefix caching is exact: everything the server computed for $c_s$ is still valid inside $c_t$.
• Training. The final transcript contains every input the model ever saw, so you can train on it in one pass.

A CLM breaks Property 2 by design, and the paper’s two technical contributions are repairs for exactly that loss. Prefix-reuse FLOPs and Suffix Cache Reuse repair the serving side. Stepwise GRPO repairs the training side (§4.2). I find this the clearest way to read the paper: one property lost, two consequences, two fixes.

Here’s a minimal harness that makes both rules concrete. A scripted “model” appends search results or edits its context file with Python, and the harness checks Property 2 at every call:

# A minimal CLM harness (Eq. 2) with a scripted "model", checking the prefix property each turn.
# Simplified from the paper's description; see the official repo for the exact semantics.
import re

def tokens(s):
    return len(s) // 4                            # rough: about 4 characters per token

def render(turns):
    return "".join(f"[[CTX_TURN {i} role={r}]]\n{x}\n" for i, (r, x) in enumerate(turns))

def parse(text):
    parts = re.split(r"\[\[CTX_TURN \d+ role=(\w+)\]\]\n", text)[1:]
    return [(parts[k], parts[k + 1].rstrip("\n")) for k in range(0, len(parts), 2)]

def results(n):
    return "\n".join(f"result {n}.{i}: " + "irrelevant text " * 30 for i in range(5))

# Each step: what the model writes, an optional edit to the context file, and a tool observation.
SCRIPT = [
    ("search('topic A')",                 None, results(1)),
    ("search('topic B')",                 None, results(2)),
    ("python: replace result 1 with a note",            # edit in the middle
     lambda f: re.sub(r"result 1\.0.*?(?=\[\[CTX)", "NOTE: topic A is a dead end.\n", f, flags=re.S), None),
    ("search('topic C')",                 None, results(3)),
    ("python: rewrite the status block near the top",   # edit near the top
     lambda f: f.replace("STATUS: starting", "STATUS: B looks promising; C pending"), None),
    ("python: compact everything before the last search",  # summary-like edit
     lambda f: re.sub(r"(role=user\]\]\n.*?\n).*(?=\[\[CTX_TURN \d+ role=assistant\]\]\nsearch\('topic C'\))",
                      r"\1[[CTX_TURN 99 role=notes]]\nSUMMARY: A dead end; B promising.\n", f, count=1, flags=re.S), None),
    ("answer: B",                         None, None),
]

def run(verbose=True):
    turns = [("system", "Your live context is the file LIVE_CTX."), ("user", "Find the right topic."),
             ("notes", "STATUS: starting")]
    prev_prompt, segments = None, []
    for t, (out, edit, obs) in enumerate(SCRIPT, 1):
        prompt = render(turns)                                  # what the model sees at this call
        is_prefix = prev_prompt is not None and prompt.startswith(prev_prompt)
        common = 0
        if prev_prompt is not None:
            while common < min(len(prev_prompt), len(prompt)) and prev_prompt[common] == prompt[common]:
                common += 1
        segments.append({"input": prompt, "output": out, "edit": edit is not None})
        if verbose:
            reuse = f"{common / len(prompt):4.0%}" if prev_prompt else "   -"
            print(f"turn {t}: prompt {tokens(prompt):5,d} tok | prev is prefix: {str(is_prefix):5s} "
                  f"| cache reuse {reuse} | {out}")
        prev_prompt = prompt
        if edit:                                                # Eq. 2: the model rewrites the file
            turns = parse(edit(render(turns)))
            turns.append(("tool", "context file updated"))
        else:                                                   # Eq. 1: default append
            turns.append(("assistant", out))
            if obs:
                turns.append(("tool", obs))
    return segments, render(turns)

if __name__ == "__main__":
    segments, final = run()
    print(f"\nfinal context: {tokens(final):,d} tokens")

turn 1: prompt    39 tok | prev is prefix: False | cache reuse    - | search('topic A')
turn 2: prompt   673 tok | prev is prefix: True  | cache reuse   6% | search('topic B')
turn 3: prompt 1,308 tok | prev is prefix: True  | cache reuse  52% | python: replace result 1 with a note
turn 4: prompt   710 tok | prev is prefix: False | cache reuse   8% | search('topic C')
turn 5: prompt 1,345 tok | prev is prefix: True  | cache reuse  53% | python: rewrite the status block near the top
turn 6: prompt 1,362 tok | prev is prefix: False | cache reuse   3% | python: compact everything before the last search
turn 7: prompt   701 tok | prev is prefix: False | cache reuse   5% | answer: B

final context: 711 tokens

The “prev is prefix” column is Property 2 in action: every call right after an edit (turns 4, 6 and 7) breaks it, and cache reuse collapses to single digits. Turn 6 follows an edit near the top of the context, and it reuses only 3%.

#What “subsumes” buys, and what it doesn’t

The paper notes that Eq. 2 subsumes earlier approaches. That’s easy to verify by writing each one as a particular edit operator:

So “subsumes” is a statement about expressiveness: the CLM’s action space contains every earlier method’s action space. It isn’t a statement about learnability. A larger action space contains better policies, but it’s also harder to search.

The paper’s own data shows the capability dependence. Qwen3.5-9B edits rarely (1.4 times per task on TerminalBench, and not at all in half the tasks), lets its context run up to a median peak of 30.2K of the 32K limit, and relies on RL to catch up.

One wrinkle the paper doesn’t reconcile: in Appendix F, zero-shot 9B with CLM is slightly ahead of the summary harness on BrowseComp-Plus (39.9% vs 37.7%), while the RL section reports it six points behind before training (28.8% vs 34.7%). The two evaluations differ in budget (32K vs 28K), turn caps, and judge model, which is probably the explanation. But it means “the small model is weaker at context management” depends on the setup.

#Functions as a meta-capability

Earlier methods expose fixed operations: the harness author decides what “compact” means. A CLM writes code, so it can define new operations and reuse them, like the compact_turns helper from Figure 3. In the edit-operator view, the model isn’t just choosing an $\mathcal{E}$ from a menu; it’s writing new ones. That’s what the paper means by context management as a meta-capability.

#State, memory, and multiple agents

The context file isn’t the agent’s whole state. Offloaded data lives in other files in the workspace $W_t$, so the true state is $(c_t, W_t)$, and the model only sees $c_t$. Formally that makes the agent’s problem partially observable: what it knows depends on what it chose to keep in view. The paper draws the line by role, not storage: the context file is the input to the next model call, while other files act as external memory until something is read back in.

Multiple agents follow naturally: one context file per agent, $c^{(1)}, \dots, c^{(m)}$, each synced to its own model server. Spawning a subagent means creating a file; ending one means deleting it. One question the paper leaves open: can an orchestrator edit a worker’s context file directly? Nothing in the mechanism prevents it, and it would be both a powerful coordination tool and a new way for one agent to corrupt another.

#The cost metric, revisited

The paper closes §4.1 by introducing prefix-reuse FLOPs. I covered the derivation and the code in “The serving problem, sentence by sentence,” so here are just two accounting details that matter when comparing numbers across the paper.

First, what counts as a cache hit differs between experiments. For evaluation (Appendix C), the server reuses the longest prefix of whole messages that appeared in any earlier prompt. For RL (Appendix E), costs use exact token-level matching against the previous turn only. The two will disagree whenever an edit lands inside a message. Second, as noted before, it’s theoretical compute: decode is memory-bound in practice, so wall-clock costs will rank methods somewhat differently.

#§4.2 Learning context management in context and in weights

#Two knobs: the text and the weights

Once context management is model behavior, there are two things you can change to improve it: the text the model reads, and the model’s weights. Both optimize the same objective, expected task reward over a distribution of tasks, with respect to different variables:

Here $\theta$ are the weights and $s$ is an instruction or skill document that enters the context, as in Eq. 4, $c_{t+1} = f_\theta^{\text{CLM}}(c_t; s)$. Steering and skill evolution optimize $s$ with $\theta$ frozen; RL optimizes $\theta$. They behave very differently in practice:

For most practitioners on hosted models, only the left column is available. That makes the text-side results more practically important than the RL headline.

#Eq. 5 as noisy black-box search

Skill evolution solves

without gradients. Proposer models read rollouts and write new candidate skills, and each candidate’s value is estimated by running it. With binary task scores on $n$ instances, an estimate of accuracy $p$ has standard error $\sqrt{p(1-p)/n}$, about 4.5 points at $p = 0.7$ on the paper’s 102-instance development split.

The paper’s staged evaluation (6 instances, then 12 more, then 102) is a racing scheme: it rejects weak candidates cheaply and spends the full budget only on promising ones. Its weakness is common to every search of this kind: the more candidates you compare on the same dev split, the more the winner’s score is inflated by luck. In the simulation below, every candidate has the same true accuracy:

# Why dev-split numbers from a selection loop are optimistic (the "winner's curse").
# Every candidate below has the SAME true accuracy; we pick the best-scoring one on 102 dev instances.
import numpy as np

rng = np.random.default_rng(0)
TRUE_ACC, N_DEV, TRIALS = 0.70, 102, 20_000

for k in [1, 5, 20, 50]:                       # number of candidate skills compared on dev
    dev_scores = rng.binomial(N_DEV, TRUE_ACC, size=(TRIALS, k)) / N_DEV
    picked = dev_scores.max(axis=1)            # the selected skill's dev accuracy
    print(f"{k:3d} candidates: selected dev accuracy {picked.mean():.3f}  "
          f"(true {TRUE_ACC:.3f}, optimism +{picked.mean() - TRUE_ACC:.3f})")

se = np.sqrt(TRUE_ACC * (1 - TRUE_ACC) / N_DEV)
print(f"\none standard error on {N_DEV} instances: {se:.3f};  "
      f"rule of thumb for k candidates: about se * sqrt(2 ln k)")

  1 candidates: selected dev accuracy 0.700  (true 0.700, optimism +0.000)
  5 candidates: selected dev accuracy 0.752  (true 0.700, optimism +0.052)
 20 candidates: selected dev accuracy 0.783  (true 0.700, optimism +0.083)
 50 candidates: selected dev accuracy 0.799  (true 0.700, optimism +0.099)

one standard error on 102 instances: 0.045;  rule of thumb for k candidates: about se * sqrt(2 ln k)

With 20 equally good candidates, the “winner” looks 8 points better than it really is. The optimism grows roughly like $\sqrt{2 \ln k}$ standard errors as the number of candidates $k$ increases.

This is why the paper’s held-out test number matters most, and why it evaluates the test split once, which is the right protocol. On KV Store, the development gain was 61.5 points (22.3% to 83.8%) while the test gain was 35.9 points (38.3% to 74.2%). Different starting points make a clean comparison impossible, but the gap is the size selection optimism predicts. The paper doesn’t report how many candidates were compared, so readers can’t correct for it.

#Stepwise GRPO, written out

The paper describes its RL method in prose rather than printing a full objective. Here’s my reconstruction from §4.2 and Appendix E. Treat it as a reading aid, not the paper’s equation.

For each prompt, sample $G = 32$ trajectories. Trajectory $i$ consists of segments, one per model call: segment $j$ has the input $c_{i,j}$ the model actually saw and the output tokens $y_{i,j,1}, y_{i,j,2}, \dots$ it produced. The objective is

with these pieces:

• $\rho_{i,j,k} = \pi_\theta(y_{i,j,k} \mid c_{i,j}, y_{i,j,1:k-1}) \,/\, \pi_{\theta_{\text{old}}}(\cdots)$ is the usual PPO ratio, evaluated under the segment’s own input $c_{i,j}$. That’s the “stepwise” part.
• $A_{i,k} = A_i^{\text{out}} + w_{\text{eff}}\, m_k\, A_i^{\text{eff}}$, where $m_k = 1$ if token $k$ is a context-management token and 0 otherwise, with $w_{\text{eff}} = 0.25$.
• $\epsilon_{\text{low}} = 0.2$, $\epsilon_{\text{high}} = 0.28$ (clip-higher), and $\beta = 0.01$ with the low-variance KL estimator.
• Truncated importance sampling additionally reweights each token by $\min\big(\pi_{\theta_{\text{old}}} / \mu,\, C\big)$, where $\mu$ is the rollout engine’s (SGLang’s) probability, which differs slightly from the trainer’s.
• $Z_i$ is a normalizer the paper doesn’t specify: total tokens in the trajectory, tokens per segment, or something else. It decides whether long trajectories with many calls get more gradient weight.

Applied to the harness trajectory from §4.1, here are the training segments it produces:

# Turning one CLM trajectory into stepwise-GRPO training segments.
from clm_harness import run, tokens

segments, final = run(verbose=False)

# Suppose this trajectory succeeded in its group, with these group-level advantages:
A_out, A_eff, W_EFF = 0.71, 0.30, 0.25        # A_eff applies only to context-management tokens

print("seg  input tok  input recoverable from final transcript?  CM tokens?  advantage")
for j, s in enumerate(segments, 1):
    recoverable = s["input"] in final           # could we rebuild this input from the final context?
    adv = A_out + (W_EFF * A_eff if s["edit"] else 0.0)
    print(f"{j:3d}  {tokens(s['input']):9,d}  {str(recoverable):40s}  {str(s['edit']):10s}  {adv:+.3f}")

print("\nEvery segment keeps its OWN input. Training on the final transcript alone would")
print("condition the model's earlier outputs on contexts it never actually saw.")

seg  input tok  input recoverable from final transcript?  CM tokens?  advantage
  1         39  False                                     False       +0.710
  2        673  False                                     False       +0.710
  3      1,308  False                                     True        +0.785
  4        710  False                                     False       +0.710
  5      1,345  False                                     True        +0.785
  6      1,362  False                                     True        +0.785
  7        701  True                                      False       +0.710

Every segment keeps its OWN input. Training on the final transcript alone would
condition the model's earlier outputs on contexts it never actually saw.

Only the last segment’s input survives in the final transcript, and that’s Property 2’s loss from the training side. Segments that perform edits get the slightly larger advantage because of the efficiency term.

What the token mask implies. Because the efficiency advantage touches only context-management tokens, it can directly reshape how and where the model edits, but not, say, how many searches it runs. Everything else is shaped only by the outcome reward. If CLM’s lower cost after RL came mostly from fewer turns, that would be an indirect effect, not the efficiency reward at work. An ablation separating the two would be informative; Appendix F’s reward ablations are the place to look.

Credit assignment. Every segment of a trajectory gets the same advantage, so a wasteful edit inside a successful run is reinforced along with the edit that saved it. With dozens of segments per trajectory, that’s a high-variance signal. Per-segment credit, for example from a learned value of each edit, is a natural next step.

#The training run in numbers

Appendix E gives a complete recipe:

Two numbers follow from the table. Seventy steps of 8 prompts touch at most 560 prompts, less than a fifth of the 3,040 available: this is a short run, and it’s fair to wonder what a full epoch would do. And it generates $70 \times 8 \times 32 = 17{,}920$ trajectories, plus whatever dynamic sampling discards.

Note also the asymmetries: different turn caps, CLM’s uncounted editing turns, and a FLOPs reward that only CLM receives in the headline comparison.

The results (Table 2) are discussed in Sentence 7: 28.8% to 42.5% for CLM, at 1.34 instead of 2.19 PFLOPs per question compared with the RL-trained summary harness. Figure 21 adds the training curves, including runs of both harnesses with and without the FLOPs reward. That’s where to check how the efficiency reward behaves for the summary harness, which the text summarizes as costing “no clear loss” in accuracy.

#My verdict

What I’d take away. Making context an editable file is a small mechanism with large consequences, and the paper explores those consequences unusually well: across serving cost, in-context learning, and RL, not just accuracy. For practitioners, the most useful idea isn’t the RL result. It’s that a context-management policy can live in a short, versionable skill document that one sentence can steer and an evolution loop can improve, and that works on any model, including closed APIs.

What I’d want before relying on it.

• Statistics. Single runs for the math and swarm results; a 0.4-point RL margin inside the noise; dev-split numbers selected on the same split.
• Internal consistency. Five vs six subagents; a 32K vs 23,560-token budget; and a 9B model that is ahead of the summary harness in one evaluation and six points behind in another.
• The minimal harness. Budget readouts, reminders, rollback, and uncounted editing turns are human design that the “no harness” framing understates.
• Security. An agent that can rewrite its own instructions, and a serving system that keeps deleted content in reused cache states, both need threat modeling before they touch sensitive or regulated data.
• Proof that the efficiency reward is doing the work. In my own $5 replication, Tiny CLM, stepwise GRPO taught a 1.7B model to edit at the cache-friendly tail without the efficiency reward, because in that task the cheap habit was also the safe one. I’d want to see Eq. 6 tested on a task where cheap and accurate disagree before counting on it.

Would I build on it? Yes, starting with the cheap parts: a context file the agent can edit, a backup-before-edit rule, size readouts, and an evolved skill document, with edits kept near the tail where caches survive them.

#References

• Shao, R., Shen, S. Z., Yin, J. O., et al. (2026). Context Language Models. arXiv:2609.37725. CC BY 4.0.
• Sutton, R. (2019). The Bitter Lesson. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
• Shao, Z., Wang, P., Zhu, Q., et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300.
• Agrawal, L. A., et al. (2026). GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. ICLR 2026.
• Zheng, L., et al. (2024). SGLang: Efficient Execution of Structured Language Model Programs. NeurIPS 2024.
• Arora, D., and Zanette, A. (2026). Training Language Models to Reason Efficiently. NeurIPS.