Background workers

Work that crosses a process, thread, or queue boundary (a Celery or BullMQ worker, a Temporal activity, a thread pool, a fork) can silently drop spans or arrive without a session. Three rules keep worker spans arriving and linked.

1. Initialize in every process

initialize() is per process. A queue worker, forked child, or subprocess never inherits the parent’s initialization: its LLM calls produce nothing, and flush() in that process silently no-ops instead of erroring. Call initialize() at each worker process’s startup. For prefork servers (gunicorn with --preload, multiprocessing with the fork start method), initialize after the fork, in the worker’s own startup hook.

2. Pass the session id as data, and re-open the session

The session context follows await within the same task, but it never crosses a thread, a process, or a queue hop: an LLM call made inside a Celery task, Temporal activity, or ThreadPoolExecutor job arrives session-less even when the job was enqueued from inside session(). The fix is one line on each side. A session id is just a string: carry it in the job payload (or the activity’s arguments), and re-open the session inside the worker:

# Producer: the id travels with the job, not with the context
queue.enqueue(handle_job, payload, session_id=chat_id)

# Worker: its own process, so initialize() ran at worker startup
def handle_job(payload, session_id):
    with loop.session(session_id=session_id):
        run_agent(payload)  # LLM calls here join the same session
    loop.flush()

Spans reported under the same id merge into one session, so the producer’s turns and the worker’s turns read as one conversation. If a workload genuinely has no id to carry (a third-party agent you can’t modify), the ingest header x-belvedir-session-from: trace promotes each trace’s id to a session id, but that makes one session per trace and fragments multi-step work, so pass the real id whenever you control the code.

3. Flush before the job ends

Call flush() at the end of each job (long-lived workers) and before exit (short-lived ones): os._exit, process.exit(), and SIGKILL skip exit hooks, so an unflushed batch is silently lost. In Python flush() is synchronous, so calling it right before os._exit() is safe; in Node it is async, so await it. Details on Flush on serverless.

Reporting outcomes from workers

The job is where you learn whether the work succeeded, so call report_outcome(session_id, ...) there, after flush(); see Outcomes.

Get started

Node SDK

Guides

Inference

Training

Benchmarks

Resources

Background workers

1. Initialize in every process

2. Pass the session id as data, and re-open the session

3. Flush before the job ends

Reporting outcomes from workers

​1. Initialize in every process

​2. Pass the session id as data, and re-open the session

​3. Flush before the job ends

​Reporting outcomes from workers

1. Initialize in every process

2. Pass the session id as data, and re-open the session

3. Flush before the job ends

Reporting outcomes from workers