What it trains on
Successful tasks that captured both an input and an output, shown as the example count on the Training page. One in five tasks is held out for evaluation and never trained on; generated environments score against that holdout. Train on everything the agent does well, or pick a single task group to distill one kind of work into a specialist. The Training Sets page (Data section, below Task Groups) shows exactly what a run on each task group would see: the group’s successful tasks minus the holdout, tasks that errored, had a tool call fail, or showed user frustration or confusion, exact duplicates, and tasks unusually short or long for the group. Open a set to browse its examples. Training needs at least 200 examples. Datasets are capped at 1,000 examples; the newest eligible tasks win. Sessions you report as"fail" through outcomes are excluded.
Training methods
Pick one per run, from the dashboard, the CLI (belvedir train <project> --method sft|opsd|rl), or auto-training:
- Supervised fine-tuning (
sft): the default and the cheapest; trains on your successful outputs directly. - On-policy self-distillation (
opsd): the model generates its own answers and is trained toward the known-good output. - Reinforcement learning (
rl): trains on a score. The reward is an LLM judge grading each sample against your reference outputs, or a cheaper similarity reward (available for Qwen 2.5 7B Instruct and Llama 3.1 8B Instruct; Qwen 3.5 9B uses the judge reward). The algorithm is GRPO, PPO, or Auto, which picks for you.
Base models
Qwen 2.5 7B Instruct, Llama 3.1 8B Instruct, and Qwen 3.5 9B. Llama is gated on HuggingFace, so a run on it asks for an HF token; the token is used only for that run and never stored.Auto-training
Enable auto-training in the Training setup wizard to launch runs on their own: when enough new traces have arrived since the last run, or on a fixed cadence, with a cooldown between runs. It is off by default, repeats the configuration of your last completed run, and its runs are treated exactly like manual ones. Turn it off from the same wizard or withbelvedir autotrain <project> --off.
Auto-training cannot repeat a run that used a gated base (Llama), because the HF token is never stored; a project that wants auto-training should train on an ungated base.
What you get back
Metrics recorded on the run (train and eval loss for supervised fine-tuning, distillation loss for self-distillation, mean reward for RL) and the LoRA adapter, a zip of standard PEFT files. A run whose model did not beat the plain base model on the held-out data is marked failed instead of ready, with the comparison recorded on the run. Load a ready adapter on its base withpeft, vLLM’s --lora-modules, or any runtime that speaks LoRA, or use the test session and the Deploy button below, which handle any merge step the base needs.
Test a fine-tuned model
Every finished model has a Run inference button on the Models page. It serves the base model with your adapter attached and opens a chat panel against it. A test session costs about $2 and runs for up to 60 minutes; warm-up takes from a few minutes to about 25 for bases whose adapters must be merged first. The session is also an OpenAI-compatible server, so no provider account is needed to test from code. Open Use as an API endpoint next to the chat and copy the snippet: a base URL, a session key, and the model nameadapter. Point any OpenAI SDK or curl at it. The endpoint lives for the session’s 60 minutes and the key stops working when the session ends.
Keep two things in the request (the snippet includes both): the model name must be adapter (the base model name would skip your LoRA), and chat_template_kwargs: {"enable_thinking": false} (thinking bases like Qwen3 otherwise spend the token budget on reasoning).