Custom agent loops

A custom in-house loop (a while loop over model turns and tool dispatch) instruments the same way as any app: initialize() once, wrap each run in a session, optionally mark units of work with task(). The usual gap is coverage: a missed call site produces no error, just missing data.

Inventory every LLM call site first

Before wiring anything, enumerate every place the codebase talks to a model. They are rarely all behind one shared client factory. Look for:

Official SDK clients (openai, anthropic, @anthropic-ai/sdk), including ones pointed at another base URL (OpenRouter, vLLM, a proxy). These are auto-captured regardless of base URL: instrumentation patches the client, not the destination.
Raw HTTP calls to chat/completions (common in agent frameworks and loops with their own HTTP layer).
Side paths that skip the shared client: a one-off titling or classification call, embeddings, provider batch-API usage, operator scripts and cron jobs.
Background workers and queue consumers: separate processes, each with its own initialize() requirement. See Background workers.

Quick greps that find most of them:

grep -rn "OpenAI(\|Anthropic(\|chat/completions\|/v1/messages" src/
grep -rn "api.openai.com\|api.anthropic.com\|openrouter.ai\|embeddings" src/

Check what is captured

Not every call style is captured; the table on Providers says which are. A client-SDK call site has exactly one capture path, so if initialize() didn’t run before that library was imported there is no HTTP-level fallback (init-order rule).

Place the session, then the tasks

Open one session per agent run, at the loop’s entry point, with an id that is stable for the whole run (your run id or conversation id). Everything inside (every model turn, every tool span) links under it. task() is optional and marks sharper task boundaries:

def run_agent(question: str, run_id: str, user_id: str) -> str:
    with loop.session(session_id=run_id, user_id=user_id):
        for turn in range(MAX_TURNS):
            response = client.chat.completions.create(...)  # traced automatically
            calls = response.choices[0].message.tool_calls
            if not calls:
                break
            for call in calls:
                with loop.task(call.function.name):  # optional boundary hint
                    dispatch(call)
    loop.flush()  # short-lived process: flush before exit
    return response.choices[0].message.content

initialize() runs once at process start, as in the Python quickstart. The Node shape is identical with withSession({ sessionId, userId }, fn) and task(name, fn); see SDK configuration. Two session rules that bite custom loops specifically:

Don’t nest sessions. Opening a session inside another replaces the outer one’s context rather than merging with it. One session per run, opened once at the top.
metadata keys become span attributes, nothing more. Only session_id and user_id are promoted to first-class fields; avoid metadata keys named session_id or user_id, which would clobber the real ones.

When a call site can’t be captured

For a call style that isn’t captured (an unsupported HTTP client, a language without an SDK, a hop through infrastructure you don’t control), emit spans yourself with any OpenTelemetry exporter; see Agents without the SDK.

Get started

Node SDK

Guides

Inference

Training

Benchmarks

Resources

Custom agent loops

Inventory every LLM call site first

Check what is captured

Place the session, then the tasks

When a call site can’t be captured

​Inventory every LLM call site first

​Check what is captured

​Place the session, then the tasks

​When a call site can’t be captured

Inventory every LLM call site first

Check what is captured

Place the session, then the tasks

When a call site can’t be captured