Local Models

1. Run a model locally

Download the Belvedir Local desktop app; it bundles the llama.cpp engine (with Ollama and vLLM as switchable alternatives) and walks you through picking a model that fits your machine’s memory; the weights download on first launch.

macOS blocks the app on first open (it isn’t notarized yet). Click Done, not Move to Trash, then run xattr -cr "/Applications/Belvedir Local.app" and reopen it. Approving it under System Settings → Privacy & Security → Open Anyway also works.

The app serves on 127.0.0.1:8080, llama.cpp’s standard port, adopting a llama-server you already run there. On the Ollama engine the endpoint is 127.0.0.1:11434 instead; use whichever address matches your engine in the baseURL below.

2. Point your agent at the local endpoint

Ollama exposes an OpenAI-compatible API, so your agent talks to it through the OpenAI SDK, which the Belvedir SDK already instruments:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://127.0.0.1:8080/v1",
  apiKey: "local", // required by the SDK, ignored locally
});

const res = await client.chat.completions.create({
  model: "qwen3:4b",
  messages: [{ role: "user", content: "hello" }],
});

3. Trace it as usual

Nothing changes on the Belvedir side: initialize the SDK with instrumentModules: { openAI: openai.OpenAI } and wrap agent runs in withSession (see Quickstart). Spans arrive with the local model name (e.g. qwen3:4b) so you can tell local and cloud calls apart in the dashboard.

Note: inference stays on your machine, but traces do not: prompts and completions are exported to Belvedir like any instrumented app. Only enable tracing for local workloads you’re comfortable sending.

Get Started

SDK

Guides

Inference

Training

Benchmarks

Resources

1. Run a model locally

2. Point your agent at the local endpoint

3. Trace it as usual

​1. Run a model locally

​2. Point your agent at the local endpoint

​3. Trace it as usual

1. Run a model locally

2. Point your agent at the local endpoint

3. Trace it as usual