Instrumenting Custom Agent Loops: Coverage, Sessions, and Tasks

You don’t need a framework for Belvedir to trace an agent. A custom in-house loop — a while loop over model turns and tool dispatch — instruments the same way as any app: initialize() once, wrap each run in a session, optionally mark units of work with task(). What actually goes wrong in real integrations isn’t the loop; it’s coverage. Codebases of any age accumulate LLM call sites outside the main loop, and a missed one produces no error — just silently missing data.

Inventory every LLM call site first

Before wiring anything, enumerate every place the codebase talks to a model. They are rarely all behind one shared client factory. Look for:

Official SDK clients (openai, anthropic, @anthropic-ai/sdk) — including ones pointed at another base URL (OpenRouter, vLLM, a proxy). These are auto-captured regardless of base URL: instrumentation patches the client, not the destination.
Raw HTTP calls to chat/completions (common in agent frameworks and loops with their own HTTP layer).
Side paths that skip the shared client: a one-off titling or classification call, embeddings, provider batch-API usage, operator scripts and cron jobs.
Background workers and queue consumers — separate processes, each with its own initialize() requirement. See Background workers.

Quick greps that find most of them:

grep -rn "OpenAI(\|Anthropic(\|chat/completions\|/v1/messages" src/
grep -rn "api.openai.com\|api.anthropic.com\|openrouter.ai\|embeddings" src/

What auto-instrumentation captures — and what it doesn’t

Call style	Captured?
`openai` client SDK (any base URL: OpenAI, OpenRouter, vLLM, a proxy)	Yes
`anthropic` client SDK	Yes
Frameworks OpenLLMetry instruments (LangChain, LlamaIndex, …)	Yes
Raw HTTP `POST .../chat/completions` — Node `fetch`, Python `requests`/`httpx`	Yes (Node `belvedir@0.4.0+`, Python `belvedir==0.6.0+`), streamed responses included
Raw HTTP to Anthropic’s `/v1/messages` wire shape	No — use the `anthropic` client SDK for those call sites
Raw HTTP to `/embeddings`, `/responses`, or other non-chat paths	No — the raw-HTTP layer watches only `chat/completions`
Python `aiohttp`	No — use `httpx` or a client SDK

Two details worth knowing. The raw-HTTP layer skips requests made by the official OpenAI/Anthropic clients so nothing is captured twice — which also means a client-SDK call site has exactly one capture path: if initialize() didn’t run before that client’s library was patched into place, there is no HTTP-level fallback. And in Python there is no instrumentModules: auto-instrumentation always sees the installed packages, so init order is only about running initialize() before LLM calls are made.

Place the session, then the tasks

Open one session per agent run, at the loop’s entry point, with an id that is stable for the whole run (your run id or conversation id). Everything inside — every model turn, every tool span — links under it. task() is optional and marks sharper task boundaries for segmentation:

import belvedir as loop

loop.initialize(api_key=os.environ["BELVEDIR_API_KEY"], app_name="my-agent")

def run_agent(question: str, run_id: str, user_id: str) -> str:
    with loop.session(session_id=run_id, user_id=user_id):
        for turn in range(MAX_TURNS):
            response = client.chat.completions.create(...)  # traced automatically
            calls = response.choices[0].message.tool_calls
            if not calls:
                break
            for call in calls:
                with loop.task(call.function.name):  # optional boundary hint
                    dispatch(call)
    loop.flush()  # short-lived process: flush before exit
    return response.choices[0].message.content

The Node shape is identical with withSession({ sessionId, userId }, fn) and task(name, fn) — see SDK configuration. Two session rules that bite custom loops specifically:

Don’t nest sessions. Opening a session inside another replaces the outer one’s context rather than merging with it. One session per run, opened once at the top.
metadata keys become span attributes, nothing more. Only session_id and user_id are promoted to first-class fields; avoid metadata keys named session_id or user_id, which would clobber the real ones.

When a call site can’t be captured

For a call style outside the table above (an unsupported HTTP client, a language without an SDK, a hop through infrastructure you don’t control), emit spans yourself: the ingest endpoint at POST /api/v1/traces accepts standard OTLP (JSON or protobuf), and any span carrying a session.id or gen_ai.conversation.id attribute joins its session like SDK traffic. The OTel-native agents guide documents that contract end to end.

Next steps

Background workers — when the loop’s work crosses a process, thread, or queue boundary.
Model routing — running the loop’s inference through Belvedir.
Common issues — the symptoms, if something above didn’t take.

Get Started

SDK

Guides

Inference

Training

Benchmarks

Resources

Instrumenting Custom Agent Loops: Coverage, Sessions, and Tasks

Inventory every LLM call site first

What auto-instrumentation captures — and what it doesn’t

Place the session, then the tasks

When a call site can’t be captured

Next steps

​Inventory every LLM call site first

​What auto-instrumentation captures — and what it doesn’t

​Place the session, then the tasks

​When a call site can’t be captured

​Next steps

Inventory every LLM call site first

What auto-instrumentation captures — and what it doesn’t

Place the session, then the tasks

When a call site can’t be captured

Next steps