Skip to main content
Asks which model should serve a call, decided from the project’s Routing config — size-tier defaults (Big for hard requests, Medium for the bulk, Small for quick simple calls) plus per-task assignments — by matching the request against your task groups and scoring the complexity of the instruction (the ask itself; pasted documents and payload text never escalate a request on size alone, though pasted code still counts as hard). Fast and replayable: no LLM sits in the decision path (group recognition uses one cached embedding lookup), and the same input decides the same way. Same API-key auth as ingest.
  • Authorization: Bearer token with your API key, or use the x-api-key header.
  • Body: { "messages": [...] } or { "task": "summarize this doc" }, plus an optional "session_id" (body field only on this endpoint, up to 200 characters) and an optional "model": naming a model here decides through that model’s anchored router, exactly like naming it on chat completions. Bodies cap at 128 KB.

​
Response

tier is "group" when an assigned task matched, "group-record" when Automatic updates picked the model from the group’s production record (below), "strong" when the complexity score crossed the escalation threshold (the Big model), "small" when the request is a quick machine task (a short instruction, single-shot, no code, with a clear task shape; a bounded payload rides along fine) and a Small model is set, "pinned" when an earlier decision for the same session_id was replayed, "fixed" when the project turned Smart routing off (below) and the named model — or the Default model, for auto — was served with no substitution, else "default" (the Medium model). router_id names the router that decided and anchor_model is set when it is a router anchored on a model your code named. The cheaper tiers only ever serve machine-shaped tasks: calls that classify, extract, summarize, or demand a strict output format, template-style prompts, and tool loops. When a person is talking to the model (a dialogue, or a message with no task markers) the response comes from the top of the router; ambiguous requests count as conversation, so they route up, not down. Every new project starts with a Default router — Medium GLM 5.2 serves the bulk, clearly hard requests take Big (Claude Fable 5), quick machine tasks take Small (Qwen 3.5 9B) — so a fresh project routes from day one. An unconfigured project (an older one, or one whose routers were deleted) gets configured: false (never an error), so the call is safe to ship before any dashboard config exists; keep a local fallback model.

​
Session pinning

Pass your SDK session id as session_id and the session’s first decision sticks: repeat calls return the same model as tier: "pinned" for up to 24 hours, keeping a conversation consistent and prompt caches warm. Changing the routing config releases stale pins (a pinned model no longer in the config is re-decided). Fresh decisions are logged, which is what powers the Routers page’s observed per-group success rates and recommendations.

​
POST /api/v1/route/chat/completions: let the router make the call

The routing decision, executed. This sibling endpoint is OpenAI-compatible: point any OpenAI client at baseURL: https://platform.belvedir.ai/api/v1/route with your bv_live_ key and chat.completions.create lands here. Belvedir picks the model (same decision and pinning rules as above), runs the call on its own provider account, and returns the provider’s response unchanged; streaming works. Request bodies cap at 1 MB (413 beyond it). OpenAI ids that only run on the Responses API (the gpt-*-pro and codex families) are refused with a clear 400 naming the base chat model (or a chat-latest variant) to use instead.
  • model works two ways. Keep the model you used before, on every call site: a real model id (x-ai/grok-4.6, anthropic/claude-fable-5, … — the full list with identifiers is on Available Models): the call goes through a router anchored on that model, created the first time you name it. The named model is the ceiling (Big) and it answers every conversational call: when a person is talking to your agent, the response comes from the model your code named. Your auto router’s cheaper Medium and Small tiers serve only the easier machine-shaped tasks underneath it (classification, extraction, formatting, tool loops). Code that names Grok in one place and Fable in another therefore keeps working and ends up with one router per model, listed on the Routers page as “From your code” and editable like any other. Deleting one just means the next call naming that model recreates it. Projects that deleted all their routers forward an explicit model unchanged (a plain metered proxy; x-belvedir-tier reports it as client). "auto" (or omitting model) hands the call to your auto router outright; on a router-less project it returns a clear 400.
  • Hosted models execute on Belvedir’s provider accounts. A route to a local model (Ollama-style name:tag ids) or to one of your fine-tuned models returns a clear 400 here, unless you registered your own deployment for that model id under Cloud Inference in the dashboard: then the call executes on your endpoint with your provider key, metered but never billed. Without a registered endpoint, use the decision endpoint above and execute the call yourself.
  • Fastest serving by default: open models execute on the multi-provider marketplace (everything not served by Anthropic, OpenAI, xAI, or Sail directly — including the hosted open models GLM 5.2, Kimi K2.6, DeepSeek V4 Flash, gpt-oss 120B, and Gemma 4 31B; the NVFP4 Gemma variant stays on Sail, which is the only host serving it) and run on the fastest provider currently serving that model, not the cheapest — the difference is real (gpt-oss-120b measures ~1,600 tokens/second on the fastest host against ~40–110 on the cheapest). The data-retention policy on Belvedir’s account filters the provider pool before speed is considered, so this never trades privacy for speed; it can pick a pricier host, and usage.cost reflects what actually served. To choose differently, pass an OpenRouter-style provider object on the body — it is forwarded untouched and replaces the default.
  • Pass your session id in the x-session-id header or a session_id body field (stripped before forwarding) to pin the conversation to one model.
  • Which model served the call comes back in model on the response body and the x-belvedir-model / x-belvedir-tier / x-belvedir-group response headers; x-belvedir-router carries the id of the router that decided.
  • Token counts come back on every response in the standard OpenAI usage block: prompt_tokens, completion_tokens, total_tokens, plus whatever detail the provider reports (for example prompt_tokens_details.cached_tokens). Non-streamed calls carry it on the body; streamed calls get it on the final chunk before [DONE] (Belvedir turns on stream_options.include_usage upstream, so you don’t have to). When prompt compression ran, the counts are what the model actually received; x-belvedir-tokens-saved is the difference.
  • What the call cost comes back on every response: usage.cost (USD, a number) on the JSON body and the x-belvedir-cost header, and on streamed calls usage.cost on the final usage chunk (the same place OpenRouter puts it, so clients that already read it need no change). It is the amount billed to your organization for that call, computed by the same pricing code the billing sweep runs: the model’s per-token price (or the provider’s reported cost) after the inference discount, so summing usage.cost across calls reproduces your usage page. One exception: when a server-side retry fired (the response carries x-belvedir-retry), the metered empty first attempt also bills, and usage.cost covers only the answer you received. Cached input is priced at the cache-read rate. Calls served from your own registered endpoint report 0. A model with no known price omits the field, and the usage page shows it unpriced.
  • Prompt compression: before the model call, your messages pass through an extractive token compressor that drops low-signal tokens, cutting cost and time to first token. Short prompts (under ~1,500 characters) skip it, since there is nothing to save and the round trip would only add latency. The routing decision always reads your original text, and the model’s response is never compressed. Tokens saved come back in the x-belvedir-tokens-saved response header, and count toward the savings shown on your Home and Billing pages (the without-Belvedir baseline prices the uncompressed prompt). If the compressor is ever unavailable, your original prompt is sent unchanged; a routed call never fails because of compression. Compression is a per-project toggle under Project Permissions. Turn it off when every word in your prompts carries weight: lab protocols, legal or medical text, dense technical specs. The compressor only drops what it scores as filler, but in material like that there is no filler, and no token saving is worth a changed meaning.
  • Chinese models: routing can serve traffic with strong open-weights models from Chinese labs (Qwen, GLM, DeepSeek, Kimi), always hosted on American compute. If your organization’s policy or compliance rules keep Chinese-lab models out of the stack (common in government, defense, and other security-sensitive work), turn off Chinese models under Project Permissions: the router will never pick one for this project (a routing config left with only Chinese-lab models answers configured: false with the reason), they disappear from the dashboard’s model pickers and can’t be saved into a router, training will never use one, and a call that explicitly requests one returns a clear error instead of silently substituting. The models Belvedir itself runs over your data — task labelling, cleaning, signal scans, the optimizer — switch to non-Chinese models as well.
  • Billing: routed calls bill your organization per token at the rates on the platform’s Pricing page (Inference → Pricing), attributed to the calling API key. Compression savings are yours automatically, since billing meters the tokens the model actually received. Calls draw down your organization’s prepaid balance. With a card saved and auto reload turned on (threshold and amount are set under Organization Settings → Billing), the balance tops up by your reload amount whenever it falls under your threshold, before more usage runs, so traffic never stops. Pay as you go (run past zero, charged hourly for exactly what was used) is enabled per organization on request: use Speak to sales on the Billing page. An organization with no balance and no card gets 402 before any model is called. Streamed responses that the client aborts are billed on an estimate of what was streamed.
  • Thinking models: when a call lands on a reasoning model (GLM 5.2, Qwen 3.5) and its max_tokens is under 2,048, or it routed to the Small tier, Belvedir turns thinking off for that call so the budget goes to the answer instead of an unfinished reasoning trace. Set reasoning, reasoning_effort, or chat_template_kwargs yourself to override. Your setting is honored in translated form — Belvedir maps it to whatever the serving model accepts (each provider family speaks a different reasoning dialect and rejects the others’), and an off setting becomes that backend’s working off-switch. One backstop: if the model then spends the whole completion budget on reasoning and returns an empty answer, Belvedir retries the call once with thinking off rather than bill you for nothing — the response carries x-belvedir-retry: thinking-disabled when that happened (and no retry happens if your own setting already had thinking off).
  • Content-filter fallback: some models’ safety filters deterministically return an empty answer (finish_reason: "content_filter") on prompts that are actually innocuous. When the router picked the model that did this, Belvedir retries the call on your routing config’s other tiers, cheapest suitable first, and the response carries x-belvedir-retry: content-filter-fallback. When your code named the model explicitly, you get that model’s verdict unretried. The empty first attempt is still metered; the retry is what turns it into an answer instead of a refund case.
  • Forced tool choice: Claude Fable 5.1 rejects tool_choice: "required" and named-function choices (Anthropic returns 400). Belvedir sends those calls with tool_choice: "auto" instead, on the sync router and in batches, and marks the response x-belvedir-retry: tool-choice-auto. Name the tool in your prompt and handle a text answer. The Messages passthrough is untranslated: send {"type": "auto"} yourself there.
  • Smart routing (per-project toggle under Project Permissions, on by default): whether the router may pick the serving model at all. On is everything this page describes — tier escalation, the Small tier, group routes, record picks, the cascade, session pins, and routing through a named model’s anchored router. Turn it off and you only ever get responses from the model you select: a request that names a model is served exactly that model, auto is served by your router’s Default model deterministically (tier fixed), and the content-filter fallback above never switches models. Your routers, records, and recommendations stay intact for when you turn it back on.
  • Automatic updates (per-project toggle under Project Permissions, on by default): the Routers page has always shown each group’s observed success rate per model and recommended the cheapest one still winning. With the toggle on, that recommendation becomes the live decision for recognized task groups you haven’t assigned a model to: machine-shaped requests serve the cheapest model within a few points of the group’s best observed success rate over recent judged tasks (minimum 10), as tier group-record. Conversation never routes this way (a person always gets the top of the router), groups with thin records keep their configured tier, and a model you assigned to a group always wins over the record. With the toggle off, nothing switches by itself: the router editor lists each pending recommendation with its observed record, and an Approve button assigns that model to the task; it serves traffic only after your approval. Your fine-tuned models earn traffic either way once registered under Cloud Inference.
  • Contract cascade (same toggle): a non-streaming machine-shaped request with a strict output contract (response_format of json_object or json_schema) is served by your router’s Small model first; the answer is validated mechanically (JSON parse, top-level type, required keys) and a violation escalates to the decided model. The response says which happened in x-belvedir-cascade: served-small or escalated. Both attempts meter, the same honesty as the retry paths, and a per-group guardrail stops cascading anywhere Small keeps losing (over ~30% escalation). Conversation and streaming never cascade.
  • Rate limit: 25 requests/second sustained per API key with a burst of 300 by default; beyond that the endpoint returns 429. Retry after a short backoff. The limit is adjustable per organization, up to and including no limit at all: use Speak to sales on the Billing page if the default is in your way.

​
POST /api/v1/messages: Anthropic-native passthrough

Claude traffic that leans on Anthropic-native features does not have to flatten into the OpenAI shape. POST /api/v1/messages serves the Messages API, untranslated: point the official Anthropic SDK at Belvedir and requests pass through to Anthropic byte-faithfully.
What passes through intact:
  • Prompt caching: cache_control breakpoints (per-block and top-level) reach Anthropic unmodified, and the cache-read discount flows through to your bill. Verify fidelity yourself by reading usage.cache_read_input_tokens off responses, exactly as you would against Anthropic directly. Cache writes bill at Anthropic’s published 1.25× (5-minute) / 2× (1-hour) write premiums.
  • Thinking: adaptive thinking, display settings, thinking blocks in responses and streams — Belvedir never injects or strips a thinking config on this surface.
  • anthropic-beta headers: forwarded verbatim, except oauth-* betas (a client-auth artifact), which are dropped (fast mode, compaction, context management, and future betas work without waiting on Belvedir). Fast-mode calls bill at Anthropic’s published fast rate.
  • Streaming: Anthropic’s native event shape (message_start, content_block_delta, message_delta), byte-for-byte. A frontend that parses Anthropic SSE keeps working unchanged.
  • The Claude Agent SDK: it honors the same env vars for all its traffic, subagents included — set ANTHROPIC_BASE_URL=https://platform.belvedir.ai/api and ANTHROPIC_API_KEY=<bv_live_ key> and its inference routes through Belvedir. ANTHROPIC_AUTH_TOKEN works as the key carrier too.
Rules of the surface: it is model-pinned by design — pass a real Claude model id (bare claude-* or anthropic/*; Sail-served open models work here too via Sail’s Anthropic-compatible API); "auto" and routing live on the OpenAI-compatible endpoint above. Errors come back in Anthropic’s error envelope so the SDK’s typed errors work. On non-streamed calls, what the call cost rides the x-belvedir-cost response header (the body is never rewritten); streamed passthrough calls carry no cost header — read spend off the usage page. Every token is metered from the native usage object, cache writes and fast mode included, at the rates on the Pricing page. Request bodies cap at 20 MB. Same key auth, spend gate, and rate limit (25 req/s default, adjustable per organization) as the router. What no router can carry: Anthropic’s Managed Agents / Agent API sessions run their inference inside Anthropic’s own orchestration — there is no base URL to override, for Belvedir or anyone else. Instrument those workloads with the Belvedir SDK for observability instead.

​
POST /api/v1/route/embeddings: OpenAI-compatible embeddings

embeddings.create against the router base URL passes through to OpenAI on Belvedir’s key: text-embedding-3-small, text-embedding-3-large, and text-embedding-ada-002 (with or without the openai/ prefix). Input tokens are metered like every routed call; the cost rides the x-belvedir-cost header. Embeddings are model-pinned — there is nothing to route. Request bodies cap at 4 MB.

​
POST /api/v1/route/batches: batch inference at half price

For offline work (classification, evals, backfills, summarizing a corpus) Anthropic and OpenAI both sell an asynchronous batch tier at 50% of list price, with results within 24 hours, and Sail Research runs a deferred flex completion window at a lower rate. Belvedir fronts all three behind one endpoint, in the same OpenAI chat-completions shape the router speaks — at the providers’ own scale: up to 100,000 requests per batch on Anthropic models, 50,000 on OpenAI (their limit), 1,000 on Sail-served models.
  • Models: Anthropic (anthropic/*), OpenAI (gpt-*, openai/*), and the Sail-served models with a flex window — zai-org/GLM-5.2-FP8, moonshotai/Kimi-K2.6, google/gemma-4-31B-it; the other Sail models are interactive-only, and the OpenRouter-served catalog has no batch tier. Batches are not routed; a request naming "auto" or a model without a batch tier is rejected up front with a clear message. A batch mixing providers is split into one provider batch per backend and returned as one result list. Sail requests run on Sail’s flex window and are worked through a slice at a time on each status poll and by the hourly sweep — which is why the Sail portion of a batch caps at 1,000 requests and ~2 MB serialized, whichever comes first (Belvedir drives them itself; the caps for Anthropic and OpenAI are the providers’ own). Batch submissions are also checked against the project’s Chinese models permission: a batch naming a Chinese-lab model on a project with it off is refused with 403 at submit.
  • Anthropic SDK batch methods work directly: with the Anthropic client pointed at baseURL: "https://platform.belvedir.ai/api", client.messages.batches.create / retrieve / results / cancel / list / delete all run against Belvedir unchanged (cursor pagination included; delete removes an ended batch and its stored results outright, so later retrieves 404 — unlike the 29-day sweep, which purges results but keeps the batch’s counts and spend) — /api/v1/messages/batches speaks Anthropic’s Message Batches wire shapes (batch objects with processing_status and request_counts, results as their exact JSONL) over the same machinery: same reservations, pricing, cancellation semantics, and 29-day retention. Inline SDK submissions against platform.belvedir.ai cap at ~4 MB (the platform edge limit). For larger SDK batches, point the client at the batch gateway instead — baseURL: "https://web-18927-96902dd0-7nu4yp6c.onporter.run" — which serves the identical API (sync and batch) and accepts batch creates up to 256 MB (Anthropic’s own per-batch limit) by converting them to the upload-by-reference flow server-side, under your own key. Raw-HTTP callers can use the upload flow directly at any size.
  • Results format: JSONL by default — one result object per line, parseable in constant memory at any batch size; batch id/status ride x-belvedir-batch-id / x-belvedir-batch-status headers. Code migrated from a provider’s own batch API keeps its results parser verbatim: ?format=anthropic returns Anthropic’s exact results structure (batches submitted entirely as native params), ?format=openai returns OpenAI’s batch output structure (batches submitted entirely as OpenAI bodies); a mixed batch gets a clear 400 for provider formats. ?format=json wraps everything in a single JSON envelope for small batches. The response stamps x-belvedir-results-format.
  • Cancellation: POST /api/v1/route/batches/{id}/cancel forwards the cancel to the providers’ real cancel endpoints (and immediately drops Sail’s still-pending work). Provider semantics are preserved: requests that already completed still return results and still bill; everything not yet run comes back canceled in the results list. Idempotent — canceling twice or canceling an ended batch returns the batch’s current state.
  • Failure semantics: there is no silent fallback to synchronous calls — a failed batch fails loudly at batch price or not at all. A submission whose every provider portion is refused returns 502 with the provider’s error and releases the reservation in full; a partial failure runs the surviving portions and reports the failed provider portion, with its error, on the batch status. Nothing is metered until results are durably stored, so an interrupted poll can’t double-bill: re-polling settles the batch exactly once. A submission that never fanned out (a crash mid-submit) is refunded by the hourly sweep, and its status says to resubmit with a new idempotency_key.
  • Shape: request bodies are OpenAI chat-completions bodies (messages, tools, response_format json_schema, images). Anthropic requests are translated to the Messages API on the way in — cache_control markers and a thinking config on the body survive the translation — and results translated back, so every result is an OpenAI chat.completion regardless of provider. max_tokens defaults to 4096 on Anthropic requests that omit it. custom_id is required, 1 to 64 characters, and must be unique within the batch. Streaming isn’t supported.
  • Large submissions: inline request bodies over ~4 MB are rejected at the platform edge with 413, so big batches submit by reference: POST /api/v1/route/batches/uploads returns 201 with {object, upload_url, expires_in_seconds}; PUT the full submission JSON ({"requests": [...]}) to upload_url with Content-Type: application/json, then POST /api/v1/route/batches with {"input_object": "<object>"} (plus idempotency_key if you use one). Validation, reservation, and the response are identical to inline submission. The signed upload_url is valid for 2 hours (expires_in_seconds: 7200); objects uploaded but never submitted are purged after 48 hours.
  • Anthropic-native shape: a request may send {"custom_id", "params"} instead of {"custom_id", "body"} — the same wire shape Anthropic’s own Message Batches API takes, forwarded verbatim (cache_control, thinking, native content blocks intact), and that request’s result carries the native Anthropic message instead of a translated chat.completion. The two shapes can mix in one batch. Native params are accepted for Anthropic and Sail-served models (not OpenAI ids).
  • Timing: most batches finish in minutes; the providers guarantee 24 hours. Polling the status endpoint completes the batch as soon as the providers are done; an hourly sweep finishes any batch you stop polling. The status object carries succeeded_count / errored_count (plus canceled_count / expired_count when relevant), created_at, ended_at, and a results_url once results are ready.
  • Retention: results are retained for 29 days after the batch ends (the providers’ own window), then purged; the results endpoint answers 410 after that, while the batch’s counts and spend stay on the status endpoint. Fetch results you want to keep into your own storage.
  • Billing: batches bill at the batch rate — half of the router’s per-token price for Anthropic and OpenAI, Sail’s flex rate for Sail; the Pricing page lists them under Batch inference — when they end, attributed to the calling API key and shown as “Router (batch)” on the usage page. Every model in a batch must be a priced model (the Pricing page lists them; an unpriced id is refused with 400). Submission reserves the batch’s estimated cost (request text plus max_tokens at the batch rate) against the organization’s credits; the reservation is released in full when the batch ends and actual usage bills as always, so concurrent batches can never overspend one balance. A pay-as-you-go organization with credits is reserved against like any other; only when its balance can’t cover the estimate does the batch proceed unreserved. A balance that doesn’t cover the reservation returns 402 with the estimate: add credits or split the batch. Submissions are limited to a burst of 30 and 30 per minute per API key (429); status polls at 60/minute per key; results reads are not limited. When the platform’s shared provider batch capacity is momentarily full, submission returns a retryable 503 with Retry-After.
  • Idempotency: on POST /api/v1/route/batches, pass a top-level idempotency_key (up to 128 characters, unique per project) with the submission. A retried submit — a network timeout, a crashed script — returns the existing batch instead of creating and paying for a second one. The Anthropic SDK facade’s batches.create does not carry an idempotency key; use the endpoint directly if you need one.
  • Sub-batches: large batches are submitted upstream as provider sub-batches of up to 25,000 requests automatically; the providers array on the batch status reflects them. Purely an internal mechanic — one batch id, one result list, and cancellation covers all of them.
See Model Routing for the full integration pattern, and Capabilities and Limits for the one-page summary of what every inference surface can and cannot do.