Models

An Agent needs a language model to do work. OpenBot does not include a model. You connect one. A model provider is a model service, its API key, and the list of its models. You add providers through the API. OpenBot stores them in the database, so every instance uses the same ones.

There are two kinds of providers:

Kind	Who adds it	Who can use it
Organization provider	An organization, with its own key	Only that organization. This is “bring your own key”.
Shared provider	A platform admin, with the platform’s key	The organizations in its `availability`. See Platform.

When you run OpenBot from code, you can also give built-in models. See Built-in models.

Add a provider

POST /v1/organization/model_providers adds a provider to your organization. It needs model_providers.write.

curl -H "Authorization: Bearer $OPENBOT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "openai-compat",
    "name": "Local vLLM",
    "endpoint": "https://llm.example.com/v1",
    "apiKey": "sk-...",
    "models": [
      { "id": "qwen3-32b", "name": "Qwen3 32B", "reasoningEfforts": [], "serviceTiers": [], "contextWindow": 131072 }
    ]
  }' \
  https://api.example.com/v1/organization/model_providers

Field	Rules
`kind`	Required. The API that the service speaks: `openai`, `anthropic` or `openai-compat`. See the table below.
`name`	Required. 1 to 120 characters.
`endpoint`	Required for `openai-compat`. Optional for the others. An `http` or `https` base URL, often ending in `/v1`. No user name, password, query, or `#` part.
`apiKey`	Required. OpenBot stores it encrypted and sends it to the endpoint only.
`models`	Required. 1 to 200 models.

`kind`	API
`openai`	OpenAI Responses API.
`anthropic`	Anthropic Messages API.
`openai-compat`	OpenAI Chat Completions API, and services that copy it.

Each model has these fields:

Field	Rules
`id`	Required. The model name that the service expects, such as `gpt-4.1`.
`name`	Required. A name for people.
`reasoningEfforts`	Required. The `reasoningEffort` values that the model accepts. Can be empty.
`serviceTiers`	Required. The `serviceTier` values that the model accepts. Can be empty.
`contextWindow`	Optional. The size of the context window, in tokens.
`maxOutputTokens`	Optional. The most tokens that the model can write in one answer.

OpenBot answers with the provider. Its id starts with prov_. A provider ID is unique in the whole platform. A model selection names it as provider. The API key never comes back. The answer shows apiKey: { set, hint } instead. See Platform.

Change or remove a provider

Route	What it does
`GET /v1/organization/model_providers`	Lists the providers.
`GET /v1/organization/model_providers/:providerId`	Reads one provider.
`PATCH /v1/organization/model_providers/:providerId`	Changes a provider.
`DELETE /v1/organization/model_providers/:providerId`	Removes a provider.

Reading needs model_providers.read. Changes need model_providers.write.

The body of PATCH needs expectedVersion, the version that you read. If someone changed the provider after you read it, OpenBot returns 409 with ModelProviderConflict. Leave out apiKey to keep the current key.

curl -X PATCH -H "Authorization: Bearer $OPENBOT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"expectedVersion":0,"apiKey":"sk-new..."}' \
  https://api.example.com/v1/organization/model_providers/prov_4c1e...

The next model call after a change uses the new key or endpoint. OpenBot builds each model once, and builds it again when its provider changes.

You cannot remove a provider while defaultModel or helperModel in the organization settings names it. OpenBot returns 409 with ModelProviderInUse. Change the setting first.

Choose a model

GET /v1/organization/models lists the models that your organization can use now. It needs models.use. The list has your own providers, and the shared providers that the platform makes available to you.

curl -H "Authorization: Bearer $OPENBOT_TOKEN" \
  https://api.example.com/v1/organization/models

{
  "providers": [
    {
      "id": "prov_4c1e...",
      "owner": "organization",
      "kind": "openai-compat",
      "name": "Local vLLM"
    }
  ],
  "models": [
    {
      "provider": "prov_4c1e...",
      "id": "qwen3-32b",
      "name": "Qwen3 32B",
      "reasoningEfforts": [],
      "serviceTiers": [],
      "contextWindow": 131072
    }
  ],
  "defaultModel": {
    "_tag": "Config.Selection",
    "provider": "prov_4c1e...",
    "model": "qwen3-32b"
  }
}

defaultModel is the model of a new Session. Set it with defaultModel in the organization settings. When it is null, OpenBot uses the first model in the list. helperModel in the same settings sets the model that OpenBot uses for its own work: the judge of Rules and Agent memory.

A Session can save its own choice. Send the current Session revision as expected:

curl -X PUT -H "Authorization: Bearer $OPENBOT_TOKEN" -H 'Content-Type: application/json' \
  -d '{"expected":3,"selection":{"_tag":"Config.Selection","provider":"prov_4c1e...","model":"qwen3-32b"}}' \
  https://api.example.com/v1/sessions/$SESSION_ID/model

GET /v1/sessions/:id/models shows the same models for one Session. See the Sessions API reference for all fields.

To limit which providers a member may choose, use models.use on openbot.org.<org>.model.<providerId>. See Permissions.

Run without a model

An organization without any provider can still make, read, rename, fork, and delete Sessions.

Work that needs a model fails with status 503 and the error SessionExecutionUnavailable. This includes:

starting a Turn with POST /v1/sessions/:id/turns,
choosing a model with PUT /v1/sessions/:id/model,
continuing a Turn after a decision or after delegated work.

{ "_tag": "SessionExecutionUnavailable", "session": "..." }

Built-in models

When you run OpenBot from your own program, you can give models in code with the builtinModels option. See Run from code.

Every organization can use them.
The API cannot change them.
They show in the model list next to the stored providers.

builtinModels has three fields:

Field	What it is
`providers`	The providers to show, such as `{ id, name }`.
`models`	The models to show, as `Config.Model` values.
`resolve`	A function that gets a selection and returns an Effect with a `LanguageModel`.

This example gives one Anthropic model with a key from the environment of your program:

import { NodeRuntime } from "@effect/platform-node"
import { Config, Model, Provider } from "@openbot-build/core"
import {
  AnthropicProvider,
  EnvironmentConfiguration,
  Server,
} from "@openbot-build/openbot"
import { Context, Effect, Layer, Redacted } from "effect"
import { LanguageModel } from "effect/ai"
import { FetchHttpClient } from "effect/http"

const selection = Config.Selection.make({
  provider: Provider.Id.make("house"),
  model: Model.Id.make("claude-sonnet-4-5"),
})

NodeRuntime.runMain(
  Effect.scoped(
    Effect.gen(function* () {
      const settings = yield* EnvironmentConfiguration.load
      // Build the model once. It lives as long as the program.
      const layer = Layer.unwrap(
        AnthropicProvider.make({ max_tokens: 4096 }).model(selection),
      ).pipe(
        Layer.provide(
          AnthropicProvider.layerClient({
            apiKey: Redacted.make(process.env.ANTHROPIC_API_KEY ?? ""),
          }),
        ),
        Layer.provide(FetchHttpClient.layer),
      )
      const claude = Context.get(
        yield* Layer.build(layer),
        LanguageModel.LanguageModel,
      )
      const builtinModels = {
        providers: [{ id: selection.provider, name: "House models" }],
        models: [
          Config.Model.make({
            provider: selection.provider,
            id: selection.model,
            name: "Claude Sonnet 4.5",
            reasoningEfforts: [],
            serviceTiers: [],
          }),
        ],
        resolve: (requested: Config.Selection) =>
          requested.model === selection.model
            ? Effect.succeed(claude)
            : Effect.fail(
                new Config.ModelConfigurationError({
                  reason: "model-not-found",
                  message: "This model is not available",
                }),
              ),
      }
      return yield* Layer.launch(
        Server.layer({ ...settings, builtinModels }).pipe(
          Layer.provide(FetchHttpClient.layer),
        ),
      )
    }),
  ),
)

When resolve fails, starting a Turn returns 503. The development entry point uses builtinModels for its test models. See packages/openbot/src/app/DevelopmentModel.ts for a full example.

Provider modules

A provider module turns a model name into a LanguageModel for one kind of API. @openbot-build/openbot exports three of them. Each one has the same parts:

Part	What it does
`make(defaults?)`	Makes a provider. `defaults` are request options for every call.
`provider.model(selection)`	Returns the model Layer for one selection. It sends no requests.
`layerClient(options)`	Makes the HTTP client. Pass `apiKey`, and `apiUrl` for another address.
`layer(tag, defaults?)`	Puts the provider in a service that you name.

Module	Provider ID	API
`OpenAiProvider`	`openai`	OpenAI Responses API. `ResponsesCompaction` adds native context compaction.
`OpenAiCompatProvider`	`openai-compat`	OpenAI Chat Completions API, and services that copy it.
`AnthropicProvider`	`anthropic`	Anthropic Messages API.

OpenBot uses these modules for the stored providers too. Context compaction makes a long history shorter so that it fits in the model’s context window. See Packages for a short example of each provider.

Development models

The development server (devenv shell -- pnpm dev) does not call a real model. It uses test models instead. They give the same answer every time, so you can test clients without cost.

The provider ID is development. The model IDs are:

Model	What it does
`mock-text`	The default. Returns fixed text at once.
`mock-slow`	Returns text slowly over ten seconds. Use it to test streams and interrupts.
`mock-tool`	Calls a harmless test tool that waits for a request.
`mock-app`	Calls a tool of the test App and returns its real content.
`mock-failure`	Sends some text, then fails on purpose with a model error.

The token counts in their answers are not real. To pick one, save it on a Session:

curl -X PUT -H "Authorization: Bearer $OPENBOT_TOKEN" -H 'Content-Type: application/json' \
  -d '{"expected":1,"selection":{"_tag":"Config.Selection","provider":"development","model":"mock-slow"}}' \
  https://localhost:8443/v1/sessions/$SESSION_ID/model

The development entry point gives these models with builtinModels, so every organization of the development platform can use them. The production program never uses them.