The agent loop and message lists

#A model call is a function of one list

An LLM API has no memory between calls. On every turn, the harness sends the entire conversation so far, and the model returns the next message. Whatever the agent “remembers” is exactly the list the harness keeps and resends; there is nothing else.

That list is made of messages, each tagged with a role:

[
  { "role": "system",    "content": "You are a research agent. Tools: search, read_doc." },
  { "role": "user",      "content": "Who founded the award launched in 2011 ...?" },
  { "role": "assistant", "content": "", "tool_calls": [{ "name": "search", "args": { "q": "award launched 2011" } }] },
  { "role": "tool",      "content": "1. ... 2. ... (10 snippets)" },
  { "role": "assistant", "content": "", "tool_calls": [{ "name": "read_doc", "args": { "id": 58939 } }] },
  { "role": "tool",      "content": "(up to 8K tokens of document text)" }
]

#Tool calls close the loop

The model’s reply can contain structured tool calls. The harness executes them, appends each result as a tool message, and calls the model again. The loop ends when the model replies without a tool call (its final answer), or when the harness hits a limit on turns, time, or tokens:

messages = [system_prompt, task]
for turn in range(MAX_TURNS):
    reply = llm(messages)            # the whole list, every time
    messages.append(reply)
    if not reply.tool_calls:
        break                        # final answer
    for call in reply.tool_calls:
        messages.append(tool_result(run(call)))

#Where the tokens go

Before each call, a chat template renders the list into a single token sequence with role markers, and that sequence has to fit the context window. Tool output dominates it. In the CLM paper’s deep-research setup, one search returns ten 512-token snippets, which is 5,120 tokens, so six searches alone nearly fill a 32K budget. Nothing in the loop above ever removes anything: the list grows monotonically until some rule intervenes.

One subtlety matters later in the paper. Some chat templates quietly drop the reasoning blocks from earlier assistant turns when rendering. So even a “plain” chat isn’t strictly append-only at the token level. This comes back when we get to serving in §4.3.

#The takeaway

Three facts carry the rest of the review:

1. The context is whatever the harness sends. There is no other memory.
2. It only grows, unless something edits the list.
3. Traditionally, only the harness edits it, typically by summarizing everything once a threshold is crossed. Context Language Models change who gets to do the editing.