Skip to main content
Open models too big for your hardware run the same way from a hosted OpenAI-compatible endpoint; the integration is identical to local models, just with a different base URL and a real API key.

​
1. Pick an endpoint

Providers like Together, Fireworks, Groq and OpenRouter serve open models behind OpenAI-compatible APIs. If you’d rather run your own weights, self-host vLLM on a GPU cloud; it exposes the same API.

​
2. Point your agent at the hosted endpoint

Same OpenAI SDK, different base URL:

​
3. Trace it as usual

Initialize the SDK with instrumentModules: { openAI: openai.OpenAI } and wrap agent runs in withSession. Spans arrive with the hosted model name, so local, hosted and cloud-provider calls stay distinguishable in the dashboard.

​
4. Register the endpoint for the router (optional)

If your project uses model routing, register the deployment’s base URL and provider key under Cloud Inference in the dashboard. Calls the router sends to that model then execute on your endpoint with your key: this is how the router serves your fine-tuned and self-hosted models, and those calls are metered for the usage graphs but never billed.

​
Default gateway (optional)

Project settings → Integrations can register one OpenAI-compatible gateway for every routed model at once: every call the router executes then runs against your gateway with your key, instead of Belvedir’s provider accounts. Use it when you already aggregate AI spend somewhere, such as Vercel AI Gateway or an OpenRouter account: your gateway’s dashboard sees all traffic and spend, while Belvedir keeps routing, tracing, and metering the calls. See the Vercel AI Gateway guide for the end-to-end setup. Three things to know:
  • A per-model registration (section 4) wins over the default gateway for that model, and fine-tuned or local model ids never fall through to it; register those exactly if your gateway serves them.
  • Routed model ids are sent to the gateway in their public spellings. Keep your routers on ids the gateway serves (for Vercel AI Gateway, provider/model slugs such as openai/gpt-5.2).
  • Calls on your gateway bill on your gateway’s account, not your Belvedir balance; Belvedir’s usage graphs show their tokens at zero cost.
Note: unlike local models, the hosted provider sees your prompts; the usual data-handling considerations for a cloud provider apply.

​
Testing a fine-tuned model without a provider

Models fine-tuned on Belvedir need no provider account while you test: every playground session on the Models page is itself an OpenAI-compatible server. Start a session (“Run inference” on the model’s row), open Use as an API endpoint next to the chat, and copy the snippet: a base URL, a session key, and the model name adapter. Point any OpenAI SDK or curl at it from your own code. The endpoint lives for the session’s 60 minutes and the key stops working when the session ends. Two things to keep in the request (the snippet includes both): the model name must be adapter (the base model name would skip your LoRA), and chat_template_kwargs: {"enable_thinking": false} (thinking bases like Qwen3 otherwise spend the token budget on reasoning).

​
Deploying a fine-tuned model

Models trained on Belvedir are standard PEFT LoRA adapters. Click Deploy on the model’s row (Models page) and paste a Together API key: the platform merges the adapter into the base weights and uploads the merged model to a managed, scale-to-zero endpoint in your Together account (Together doesn’t accept raw LoRA adapters on dedicated endpoints, so the deploy ships a full merged copy; expect roughly 10 to 40 minutes depending on the base size). The key is used for the deploy and never stored. Then point your agent at that endpoint exactly as above. Two preconditions, each refused with a clear message: the model needs a recoverable training checkpoint to merge from (models trained on the managed Tinker backend have one), and the base model must be on Together’s supported dedicated-endpoint list. A model that can’t deploy this way still serves in the playground session above and self-hosts on vLLM. Prefer somewhere else? The panel also has prefilled steps for Fireworks and self-hosted vLLM, and the adapter downloads as a plain PEFT zip that loads anywhere. Belvedir is never in the request path.