Below the Ice — The Loop Debate: What AIEWF Got Right (and Left Open)

The AI Engineer World's Fair closed Thursday. The headline from the closing sessions, as we covered this morning, was a debate about loops. Two camps: practitioners who call the loop the natural atomic unit of autonomous work, and practitioners who say loops without checkpoints fail silently and expensively when it matters most.

That 3-minute cold open was the surface. Tonight we go below it.

---

What it is

An agentic loop is the core pattern behind almost every autonomous AI system in production today. Strip it to its essentials and you get three moving parts:

1. A model — a language model that can read a situation and decide what to do next.
2. A tool schema — a defined set of actions the model is allowed to take (call an API, read a file, search the web, write code and execute it).
3. An observation channel — the mechanism that feeds the result of each action back into the model's context.

The loop is this: model reasons → model calls a tool → observation lands in context → model reasons again. It repeats until one of two things happens: the model decides it's done (it emits a final answer), or an external governor halts it (step budget exceeded, error threshold crossed, human interrupt).

That's it. Everything you've seen under the labels "ReAct agent," "tool-use agent," "code-interpreter agent," and most orchestration frameworks is a variant on this pattern. The sophistication lives in the scaffolding around it, not in the loop shape itself.

---

How it actually works

Start from scratch, with a concrete example.

You ask an agent: "Find the three most-cited papers on retrieval-augmented generation and summarize their key findings."

The model doesn't answer in one shot. It doesn't hallucinate three papers and hope you won't check. Instead, the loop runs:

• Step 1 — the model reasons: "I should search for RAG papers by citation count." It calls web_search("retrieval-augmented generation highly cited papers site:arxiv.org").
• Step 2 — the tool returns a list of URLs and snippets. The observation lands in context.
• Step 3 — the model reasons: "I have three candidates. I should fetch the abstracts." It calls fetch_url(url) three times across three loop iterations.
• Step 4 — with three abstracts in hand, the model reasons: "I now have enough to answer." It emits a final summary and exits.

The analogy that helps: think of a chef following a recipe for the first time. They don't prep the entire dish in one move. They read the next step, make that move, taste or observe, decide whether to continue or adjust. The dish emerges one step at a time, with observation threading the steps together.

The seminal formalization of this pattern is the ReAct paper (Yao et al., 2022): Synergizing Reasoning and Acting in Language Models. ReAct interleaves "thought" traces (the model reasoning aloud) with "action" traces (tool calls) and "observation" traces (results). The key empirical finding: the interleaving of reasoning and action dramatically outperforms either alone. Thinking without acting gets stuck; acting without thinking makes costly mistakes.

What the loop does not do by default: it does not checkpoint its state, it does not detect when it is going in circles, it does not know when to stop and escalate. Those behaviors require scaffolding — external code that wraps the loop and enforces policy on it.

---

Why it matters now

The AIEWF closing sessions surfaced a data frame that teams in production already recognize: they are spending a growing fraction of their time not building new agents, but debugging and constraining existing ones. The problem is almost always the loop.

Three failure patterns that keep showing up:

Silent loops. The model calls a tool, the tool returns an ambiguous result, the model calls the same tool again with slightly different parameters, the result is still ambiguous — and this continues until the step budget runs out or the token window fills. No error is thrown. The agent just fails to reach a conclusion, and the user sees a timeout.

State amnesia. The loop accumulates context across steps, but context grows. After enough iterations, the early steps fall out of the window. An agent working a complex, multi-stage task can effectively forget what it was trying to do — mid-task — and start improvising.

Cascade exits. One tool call fails (network error, bad API response). The model reasons about the failure, decides to retry, retries, fails again — and the cost multiplies fast. The rebilled-invoice incident we covered in June was a real-world version of this.

Anthropic's Building Effective Agents guide addresses this directly: the pattern they recommend is investing heavily in the edges of the loop — clear tool descriptions, well-defined exit conditions, checkpoints where the model is explicitly asked "should you continue or escalate?" — rather than relying on model capability alone to navigate ambiguity.

The AIEWF debate maps cleanly onto this. The "loops are fine" camp is usually talking about narrow, well-bounded tasks with predictable tool call patterns and short horizons. The "loops break" camp is usually talking about long-horizon, open-ended tasks where the edges of the loop are under-specified. Both are right about their domain.

---

What is overhyped

The idea that better foundation models solve the loop reliability problem.

It is tempting to assume that a smarter model makes better decisions at each step and therefore loops break less often. There is some truth to it — a more capable model does make fewer reasoning errors. But the reliability failures in production are almost never about reasoning quality. They are about:

• Missing governors. No step budget. No retry cap. No escalation path.
• Ambiguous exit conditions. The model has no clear signal that tells it "you are done." It reasons on, uselessly.
• Unobservable intermediate state. You cannot inspect what the loop decided at step 7 without digging through raw token traces.

These are infrastructure problems. A more intelligent model running inside a loop with none of these guardrails fails faster — because it generates more plausible-looking chains of reasoning on the way to a wrong or circular conclusion.

The teams building reliable production agents are not waiting for the next model release. They are building observability tooling, loop governors, and escalation patterns. That is the more durable investment.

---

What to watch

Three concrete things.

1. Loop observability tooling. The next meaningful advance in agentic reliability is probably not a new model — it is a standardized trace format that exposes what the loop "decided" at each step, in a form that a human can inspect at 2am during an incident. Watch for frameworks that emit structured reasoning traces alongside tool calls. The AI Engineer World's Fair sessions this week surfaced several early efforts; none of them have a clear winner yet.

2. Step budgets as a first-class construct. Right now, most teams hardcode a step cap in their orchestrator and hope. Watch for loop frameworks that treat the step budget as a configurable policy — one that can respond to task complexity, resource cost, and deadline pressure, and that fires a deterministic escalation when the budget is hit rather than a graceful-exit hallucination.

3. Human-in-the-loop checkpoints becoming a design pattern, not an afterthought. The most productive thing from the AIEWF debate was the framing that a loop checkpoint — a moment where the agent pauses, summarizes its state, and surfaces a go/no-go question to a human — is not a failure mode. It is a feature. Teams that instrument their loops with explicit checkpoints are reporting both better outcomes and more user trust. That framing is starting to spread.

The surface story from AIEWF was: practitioners disagree about loops. The story below it is more interesting: the infrastructure to make loops trustworthy is still being built, and the teams that build it well are not going to advertise it as a product launch. They are going to be the ones whose agents work.

Listen to today's episode of Below the Ice for the audio deep-dive on the same topic.

---

Sources: ReAct: Synergizing Reasoning and Acting in Language Models — arXiv 2210.03629 · Anthropic — Building Effective Agents · Latent Space — AIEWF Daily Dispatch · AI Engineer World's Fair