AI Agent Loop Engineering

Content updated: September 28, 2026

An AI agent loop repeatedly calls a model, executes permitted tool calls, and feeds the results back until a task is verified or a limit is reached. In this guide, loop engineering means designing that control flow: stopping conditions, budgets, verification, and recovery. It is part of harness engineering, which also covers tools, permissions, and the surrounding execution environment.

What the Loop Actually Is

The following is Python-style pseudocode, not a drop-in SDK implementation. It shows message ordering and exit paths. The helpers represent runtime responsibilities described below; their exact implementation depends on the model and tool APIs.

def run(state, ledger, deadline):
    recent_failures = deque(maxlen=3)  # per-task state, not shared globally
    try:
        for step in range(MAX_STEPS):
            reply = metered_model(state, ledger, deadline)
            state.append(reply)  # preserve calls before their results
            if reply.tool_calls:
                for call in reply.tool_calls:
                    result = authorized_tool(call, ledger, deadline)
                    state.append(tool_result(call.id, result))
                    if guard(call, result, recent_failures):
                        return incomplete(state, reason="repeated_failure")
            else:
                check = metered_verify(state, reply, ledger, deadline)
                if check.passed:
                    return completed(reply, check, ledger.spent)
                state.append(verification_feedback(check))
            state = compact_preserving_call_pairs(state)
    except LimitReached as limit:
        return incomplete(state, reason=limit.reason)
    return incomplete(state, reason="step_limit")

metered_model, authorized_tool, and metered_verify must reserve a bounded allowance before starting, enforce per-call limits and the shared deadline, and settle actual usage before returning, including a successful final call. A rejected reservation or deadline expiry raises LimitReached with a reason; the loop returns incomplete without another paid attempt. Uncertain usage remains reserved until reconciled. Every tool result must reference its matching call ID, and compaction must preserve unresolved call/result pairs. Authorization is checked before tool execution. See tool calling and MCP for the interface boundary.

What the Loop Controls

Model choice and loop design both affect results. A useful comparison holds the model and task set fixed, then changes one loop decision: stopping rules, verification, retry limits, or context handling. Measure task completion, incorrect success claims, latency, and cost. This separates an engineering hypothesis from an unsupported claim that all frontier models behave alike.

The Control Flow: Plan, Act, Observe

The canonical loop body is a three-beat cycle, often called ReAct (reason + act): the model reasons about what to do next, emits an action (a tool call), and the harness returns an observation that goes back into context for the next reason step. The power is in the feedback: the agent isn't predicting a whole solution up front, it's taking one grounded step at a time, adapting to what each action reveals.

The first real design choice lives here: how much to do per turn. A single tool call per step is simplest and easiest to recover from, but slow and chatty. Allowing the model to request several tool calls at once (parallel reads, say) cuts latency and token cost — at the price of harder error handling when one of them fails. A conservative starting point is to batch independent reads and serialize dependent writes. Parallel mutations need explicit ownership or conflict handling; the right choice depends on the tools and their side effects.

Stopping Conditions — the Hard Part

The deceptively hard question in any loop is "are we done?" Stop too early and the agent hands back a half-finished task; never stop and it loops forever, or keeps "improving" a fine answer until it breaks it. There is no single right answer — you compose several signals:

• Model-declared completion — the model stops calling tools and emits a final answer. Necessary but not sufficient: models declare victory prematurely all the time.
• Verification-gated — "done" only counts if a check passes (tests green, build succeeds, output matches a schema). This is the strongest signal because it's grounded in the environment, not the model's opinion.
• Goal/exit criteria — an explicit predicate ("the function exists and all tests pass") the loop evaluates, rather than trusting prose.
• Budget exhaustion — a hard stop regardless of state; the backstop, not the goal.

Budgets: Steps, Tokens, Time, Cost

An LLM in a loop is an unbounded spender by default — a confused agent will happily take 200 steps and burn dollars going nowhere. Budgets are the hard limits that make a loop safe to run unattended, and a serious loop tracks several at once:

The subtlety isn't having a budget, it's what to do as you approach it. A good loop degrades gracefully: as the step budget runs low it can switch from exploring to wrapping up ("you have 2 steps left — summarize what you found and stop"), rather than getting cut off mid-thought. Budgets aren't just a kill switch; they're a signal the loop can reason about.

Verification: Closing the Loop

Verification gives the loop external evidence: test results, build output, or a check of the requested artifact. It improves feedback but does not guarantee convergence or correctness. Define checks from the task’s acceptance criteria; a weak check can approve the wrong result. A browser feature may need an interaction check in addition to unit tests.

Feed a failed check back with a concise error and the evidence needed to act on it. Retain full logs outside the prompt when useful. Evaluate whether another attempt improves completion enough to justify the extra latency and cost; LLM evals should measure both successful work and false claims of success.

Error Recovery and Doom Loops

Agents fail in ways a single call never does, and most of the engineering in a production loop is the recovery paths. The big ones:

• Tool errors — a command fails or a file is missing. The fix is to return a readable, actionable error as the observation ("file not found: did you mean X?"), not a raw stack trace — the error text is feedback the model acts on.
• Malformed output — the model emits invalid JSON for a tool call. Re-prompt with the parse error, optionally with constrained decoding, rather than crashing the loop.
• The doom loop — the agent repeats the same failing action over and over, convinced it'll work this time. This is the signature agent failure, and it needs explicit loop detection.

For a minimal guard, track the last three consecutive failed attempts in a buffer owned by the current task. Normalize arguments and use a stable error category. Clear the buffer after success so legitimate repeated reads are not classified as failure. Three identical failures are a reason to change strategy or stop, not proof that every retry is unsafe.

from collections import deque
import json

def guard(action, result, recent):
    if result.ok:
        recent.clear()  # successful repetitions are not failures
        return False
    args = json.dumps(action.args, sort_keys=True, separators=(",", ":"))
    signature = (action.name, args, result.error_code)
    recent.append(signature)  # use a stable error category, not a raw log
    return len(recent) == 3 and len(set(recent)) == 1

recent_failures = deque(maxlen=3)  # create one buffer per task

Loop Topologies

The plain while-loop is the baseline, but "loop engineering" increasingly means choosing the right shape of loop for the task. Four common topologies:

• Single loop — the default ReAct cycle. Best for open-ended tasks where the next step depends on the last observation; simplest to build and debug.
• Reflection — after producing an answer, a separate critique step looks for flaws and triggers a revision. Adds a self-review beat that catches errors the first pass missed, at the cost of extra model calls.
• Plan-then-execute — the agent drafts a plan first, then executes the steps (each potentially its own small loop). Keeps long tasks coherent and gives you a checkpoint to inspect the plan before spending on execution.
• Orchestrator + sub-agents — a main loop delegates scoped jobs (search the codebase, review a diff) to fresh sub-agents with their own clean context, which return just a conclusion. This bounds how much any one context window must hold and enables parallelism ; shared writes still need coordination.

Context Across Iterations

Every loop iteration appends observations to the state, so a long task inevitably bloats toward the context-window ceiling. Managing that growth is a core loop-engineering job — the loop has to keep the goal and the key facts in view while shedding noise. The standard move is compaction: when the state approaches the window, summarize or drop older turns, preserving the goal, the plan, and recent results. Related tactics include offloading detail to an external scratchpad the agent can re-read on demand, and the sub-agent pattern above, which keeps the main loop's context lean by sending heavy work to a child. Get this wrong and the agent "forgets" what it was doing mid-task; get it right and it can work on problems far longer than any single window.

Tuning the Loop Empirically

You cannot tune a loop by intuition, because the changes that matter — a different stop condition, a tighter budget, when to compact, whether to add a reflection beat — interact in ways that swing real-world success rates surprisingly far. Serious loop work is empirical: run the agent against benchmark tasks, measure success rate, steps, and cost, and inspect trajectories (the full sequence of calls and observations) to see where runs go wrong — premature stops, doom loops, budget blowouts. Then change one thing and re-measure. The loop is a system you tune against data, the same way you'd tune any control system.

Pitfalls and Tradeoffs

• Trusting model-declared "done." Without a verification gate, the agent stops on its own say-so and ships half-finished work. Gate on a real check.
• No loop detection. Relying on the budget alone means a doom loop wastes the entire budget on one mistake; detect repetition early.
• Budgets with no graceful degradation. A hard cut-off mid-thought wastes the work done so far; let the loop wind down when it's near the limit.
• Over-engineering the topology. Reflection and multi-agent add model calls, latency, and cost; many tasks are better served by a tight single loop. Add structure only when a simpler loop demonstrably fails.
• Context rot. Skipping compaction lets the window fill with stale observations until quality silently degrades; manage state every iteration.
• Tuning blind. Without trajectory analysis you fix the wrong thing — you see a low success rate but not whether it's early stops, loops, or bad recovery.

Frequently Asked Questions

What is loop engineering for AI agents?

It is the design of repeated model and tool execution: how state is updated, actions are authorized, results are verified, budgets are enforced, and the run stops. It is one part of the surrounding agent harness.

When should an AI agent stop?

Stop successfully when task-specific verification passes. Otherwise return an explicit incomplete result for exhausted budgets, repeated failures, cancellation, or missing required input. A model saying “done” is not sufficient evidence.

How do you prevent an agent doom loop?

Track repeated failed attempts using normalized arguments and stable error categories. A successful action should reset a consecutive-failure guard. Combine this signal with step, time, and cost limits.

Can an agent safely retry a timed-out write?

A timeout does not prove the write failed. Check operation status or use a stable idempotency key before retrying. Replaying a mutation with a fresh key can duplicate its side effects.

Worked Example: Fixing a Failing Test

Give a coding agent a bug report and a failing test. Require the final result to include the patch and the relevant test output. The following outcomes need different exit reasons:

The guard above intentionally does not detect alternating failures, successful polling with no progress, or semantically equivalent calls. Add a progress signal for the task, such as newly passing checks, and bound polling separately. Log the task ID, step, tool name, outcome, and exit reason; omit secrets and raw tool arguments.

Sources and Next Steps

Anthropic’s Building Effective Agents describes simple agent patterns and environmental feedback. The pseudocode and failure cases on this page are our explanatory examples. Continue with harness design, tool calling and MCP, and agent evaluation.