Configuration

This page explains how a running gateway finds its configuration: which files it reads, where those files live, and which layer wins when more than one names the same file. It is the conceptual companion to Adding a New Model (the field-by-field model registry reference) and Routing (what the routing engine does with the result).

For distribution manifests, branding, UI modules and backend extensions, see Distribution customization.

Optional Tencent SES email delivery

EMAIL_PROVIDER defaults to smtp. Select tencent_ses in Admin > Configuration to use Tencent Cloud transactional email instead. The selection is explicit: a failed SES request or missing SES template does not fall back to SMTP. The existing verification, password-reset and notification callers use the selected transport.

Configure TENCENT_CLOUD_SECRET_ID, TENCENT_CLOUD_SECRET_KEY, optional TENCENT_CLOUD_TOKEN, TENCENT_SES_REGION, TENCENT_SES_FROM_EMAIL and optional TENCENT_SES_FROM_NAME. The sender must be approved in the selected region. Secret values remain server-side and are redacted in the configuration API. Keep production credentials and sender identities in deployment configuration.

Set positive IDs for TENCENT_SES_TEMPLATE_VERIFICATION and TENCENT_SES_TEMPLATE_PASSWORD_RESET before enabling SES account mail. Configure TENCENT_SES_TEMPLATE_APPROVAL, TENCENT_SES_TEMPLATE_REJECTION, TENCENT_SES_TEMPLATE_ADMIN_SIGNUP and TENCENT_SES_TEMPLATE_NOTICE for the notifications the deployment uses. Unset IDs disable those templates. Configuration health reports missing SES account-mail fields when public signup requires email verification, instead of asking for SMTP credentials.

Reference HTML lives in config/examples/tencent-ses/. Before submitting templates for provider approval, replace the example domain with the deployment console domain and choose the desired language and wording. Verification and reset templates contain a fixed HTTPS origin and path; only the encoded token query value varies. Other templates use site_name and, where applicable, reason, identifier or body. Keep template links consistent with FRONTEND_URL; the gateway cannot inspect approved template contents.

SES receives approved templates with escaped scalar values, never arbitrary HTML variables. Serialized template data is limited to 800 UTF-8 bytes; oversized notices and HTML-only notices fail rather than being truncated. Successful submission means provider acceptance, not inbox delivery. Calls are not automatically retried because a timeout can follow acceptance. Deployments that require unrestricted HTML mail or long notices can continue to use SMTP.

What the gateway reads at startup

Kind

Holds

models

The model registry: every model id the gateway serves and the upstream routes behind it

routing

Optional deployment-wide settings: health probing and a local/remote weight split

alerts

Alert rules and thresholds

Settings

Credentials, SMTP, proxy trust, logging, routing tuning and every other application setting, stored in the database and edited in the admin console

Environment

The database connection, the paths to the files above, and how the containers are wired

Only the model registry is required to serve traffic. Without it the gateway starts, serves /health, and answers GET /v1/models with an empty list. Without a routing file each route keeps the weight the registry gives it, and without an alerts file the built-in thresholds apply.

The files are not the whole story once a database is configured. The admin console writes providers, keys, routes, weights and per-model overrides to the operational store, and the gateway re-applies that state on top of the loaded registry at every start. Runtime configuration from the admin console covers that layer, and Settings stored in the database covers the settings.

Where configuration lives

config/examples/ contains reference gateway configuration, including models.openrouter.yaml and routing.minimal.yaml. A deployment keeps its model registry, routing rules and alerts in its own configuration directory, such as distributions/<name>/config/, and selects them through explicit environment variables or an active distribution manifest. Distribution customization explains the manifest and overlay layout.

A provider, key, route or whole model can also be added from the admin console while the gateway runs. That state lives in Postgres and is described below.

How a gateway finds its config

Precedence is environment, then manifest, then built-in default.

Each kind of file is looked up on its own, in this order:

  1. An explicit environment variable. MODELS_CONFIG_PATH, ROUTING_CONFIG_PATH, ALERTS_CONFIG_PATH. (The older names MODELS_CONFIG and ROUTING_CONFIG are still accepted; the canonical *_CONFIG_PATH name wins when both are set.)

  2. The distribution manifest’s paths: section — only when DISTRIBUTION_CONFIG_MODE=active. By default a manifest is only checked, not applied; see Activating a manifest.

  3. The built-in default, which points at the reference examples: config/examples/models.openrouter.yaml and config/examples/routing.minimal.yaml. The alerts default is config/alerts.yaml, a path this repository does not ship — a missing alerts file means “use the built-in thresholds”.

Two things follow:

  • An environment variable beats the manifest. If you set MODELS_CONFIG_PATH in a deployment that also has a manifest, the manifest’s models: path is ignored, and the gateway logs explicit env override ... wins over manifest value .... Pick one mechanism.

  • Paths that come from the environment are resolved relative to the working directory of the process. Relative paths inside a manifest are resolved against the manifest file’s own directory.

A path that resolves from layer 1 or 2 but does not exist is a warning, not a failure: the gateway logs Models config not found: <path> (source=env) and continues with no models registered.

What a fresh clone does with no configuration at all

Start the gateway in a clean checkout with no MODELS_CONFIG_PATH, no manifest and no overlay, and the built-in default applies: it serves config/examples/models.openrouter.yaml. That file registers three models — two straight OpenRouter routes and one that puts a local OpenAI-compatible server first with OpenRouter as automatic fallback — and every one of them needs the same single credential.

export OPENROUTER_API_KEY=sk-or-...
uv run uvicorn serving.servers.app:app --no-proxy-headers --port 8080

Without that variable every model in the file is skipped — its route’s only API key expands to empty — and the gateway says so before serving anything:

No models are available: every model in config/examples/models.openrouter.yaml was
skipped because its credential is unset. Set OPENROUTER_API_KEY (Admin > Configuration)
and restart. /v1/models will stay empty until then, and requests will report the
model as not found.

With a database, set it on the admin console’s Configuration tab instead, which lists it as missing, and restart the backend.

This is the fastest way to a gateway that routes real traffic. Replace the file with your own registry once you know what you want to serve.

The model registry

Adding a New Model is the field reference. Three things belong here because they are properties of configuration loading rather than of any one field:

A route’s kind: picks the adapter; the model’s provider: is its default. When a route omits kind:, the model’s top-level provider: is used; when a model omits route: entirely, a single route is built from the top-level provider:, base_url and api_key. provider is also the label written to api_logs.provider and shown in metrics, which is why a route may override it independently with provider:.

These are the built-in kinds. A deployment can add more with a backend extension; those are checked first.

Kind

Adapter

openai_compat, staging, vllm, sglang, ollama, chutes, featherless, cliproxy, deepseek, zai, kimi, minimax

OpenAICompatAdapter

openrouter, openrouter[<slug>]

OpenRouterAdapter

claude

ClaudeAdapter

gemini

GeminiAdapter

anthropic

AnthropicAdapter

Local inference servers have no dedicated adapter: vllm, sglang and ollama are OpenAI-compatible kinds that differ only in their provider label and usage handling. Any other kind fails registry loading with ValueError: Unknown adapter kind: <kind>.

${VAR} interpolation in the registry is whole-value only. In models.yaml, only base_url, api_key, api_keys, provider_model_id and a route’s embeddings_path are expanded, and only when the entire value is exactly ${VAR}. The value is the setting of that name: its database value, else the environment variable. There is no ${VAR:-default} and no embedded substitution:

base_url: ${LOCAL_BASE_URL}                  # expanded
base_url: ${LOCAL_BASE_URL:-http://x/v1}     # NOT expanded — treated as a var
                                             #   named "LOCAL_BASE_URL:-http://x/v1",
                                             #   resolves empty, model is skipped
base_url: http://${LOCAL_HOST}/v1            # NOT expanded — the literal string,
                                             #   including "${LOCAL_HOST}", is used

The routing and alerts files are more permissive (see below), so do not carry habits from one file to the other.

The routing file

routing.yaml is optional, and most deployments can leave it out. Read this section before relying on it: on a gateway with a database, which includes the standard Docker stack, its weight split has no effect, and its health probe never changes routing. Where traffic goes is decided by the weights in the model registry, any weight overrides set in the admin console, and the router described in Routing.

Every key, with its default:

Key

Default

Meaning

default_router

fixed

The router for models that do not name one with router:. The weight split below also runs only when this is fixed.

timeout

2

Seconds a health probe waits for a connection.

health_check

0

Seconds between health probes; 0 disables probing.

local_deployment

[]

Endpoints treated as local.

remote_deployment

[]

Endpoints treated as remote.

logging

{}

Free-form mapping.

Each deployment entry needs both an endpoint: (which must start with http:// or https://) and a non-empty models: list; an entry with an empty models: fails validation for the whole file, and the gateway logs RoutingManager failed to initialize and carries on with the registry’s own weights. An entry whose endpoint: expands to empty is dropped individually, with a warning naming the orphaned models, so one unset variable cannot invalidate every other endpoint.

default_router: fixed
timeout: 2
health_check: 30

local_deployment:
  - endpoint: ${LOCAL_BASE_URL:-http://localhost:8000}
    models: [<model-id>]

remote_deployment:
  - endpoint: https://api.your-provider.example/v1
    models: [<model-id>]

The weight split applies only to a gateway without a database. At startup, the gateway sorts each model’s routes into local and remote: a route joins a group when its base_url matches that entry’s endpoint: exactly and the model is named in that entry’s models:. It then gives each group half the traffic (unless the deprecated routing_parameter.local_fraction sets another share), spreads that evenly within the group, and scales a model’s weights to sum to 1.0. When one group is empty for a model, the other gets everything. A model that matches nothing keeps the weights from its route: list. On a gateway with a database, routing reads the registry’s weights plus any admin overrides and never sees this split.

Health probing covers local_deployment endpoints only — remote providers do not serve the gateway’s /health path and would be marked unhealthy for it. Each probe is a GET to the endpoint’s origin root plus /health.

The probe only reports; it never changes routing. An endpoint that fails every probe keeps its weight and keeps receiving traffic. Each change of state is logged at WARNING, and GET /routing shows the latest results under manager_status.endpoint_health, next to endpoint_health_enforced: false. The circuit breaker is what takes a failing endpoint out of rotation.

Unlike the model registry, this file’s expander handles ${VAR}, ${VAR:-default}, and variables embedded in longer strings, at any depth.

routing_strategy: and routing_parameter: are the deprecated spellings of default_router: and per-model router_params:. They still load, and log a deprecation warning. Per-model router selection (router: / router_params: in the model registry) overrides default_router and is documented in Routing.

Settings that currently have no effect

The gateway accepts these when it loads its configuration, so setting them does not stop it from starting, but today they do nothing — or, in the case of the routing file’s weight split, nothing on a gateway with a database.

Setting

Where

What happens

local_deployment / remote_deployment weight split

routing file

Applied only on a gateway without a database; see The routing file

health_check probe results

routing file

Logged and reported on GET /routing, never used for routing

router_params.local_fraction

model registry, router: fixed

Accepted and validated, not read

features.routers

distribution manifest

Published in /site-config, does not select or restrict a router

paths.mcp

distribution manifest

Accepted, not read by any component

site.terms_document, site.privacy_document

distribution manifest

Accepted, not loaded into the terms page; use a Site UI module

deployment.target

distribution manifest

A label only; does not select a host or build anything

Runtime configuration from the admin console

The files above are read once, at startup. Everything else an operator changes about routing goes through the admin console — or the /admin/* endpoints behind it — and is written to the operational store, so it takes effect without a restart and survives one. This is the layer to use when a model, a provider or a key has to exist now, on a running gateway, without editing the overlay and redeploying.

It needs a database. DB_ENABLED defaults to true (Database covers the connection); with DB_ENABLED=false there is no operational store, every endpoint below answers 500 Database not configured, and the console tabs that call them have nothing to write to. Every endpoint requires an administrator’s JWT or ADMIN_TOKEN in the Authorization: Bearer ... header.

Providers tab

What

Console

Endpoint

Stored in

Add a custom OpenAI-compatible provider, with its first key

Overview → Add provider

POST /admin/provider-definitions; POST /admin/provider-definitions/verify probes the upstream first, without saving

provider_definitions

Edit or remove a custom provider

Overview → Edit provider

PATCH / DELETE /admin/provider-definitions/{provider}

provider_definitions

Add a provider API key

Keys → Add a new key

POST /admin/provider-keys; POST /admin/provider-keys/verify checks it first

provider_api_keys

Disable, re-enable or delete a key

Keys

POST /admin/provider-keys/{key_id}/disable, .../enable, DELETE /admin/provider-keys/{key_id}; for a key that came from the environment, POST /admin/provider-keys/disable-env and .../enable-env

provider_api_keys, disabled_provider_env_keys

Reserve a key for a tier

Keys → Reserved for

POST /admin/provider-keys/{key_id}/min-role, .../min-role-env — see Reserving upstream keys for a tier

provider_api_keys.min_role, provider_env_key_min_roles

Take a provider out of rotation

Availability → Enabled

PATCH /admin/providers/{provider}/disabled; GET /admin/providers/routable lists what is in the routing table

disabled_providers

The registry on the Overview tab lists every provider the gateway can route to, but only custom providers are editable there. A provider that code or the model registry already owns — the built-in adapter kinds, and any provider: label a route declares — is read-only in this table, and a stored definition that reuses one of those slugs is skipped at boot rather than allowed to shadow it. Custom providers are openai_compat only; a protocol the generic adapter cannot speak needs an adapter, which is a code change (Writing a Provider Adapter).

A key added here joins the same pool as the keys the registry names through ${VAR} and rotates with them. A key the registry names — an environment key on this tab, whose value comes from the settings — has no row in this table, so this tab can disable it or reserve it for a tier but cannot delete it. To remove it, clear the variable on the Configuration tab and restart the backend.

Routing tab

What

Console

Endpoint

Stored in

Create a model that is not in the registry

Create model

POST /admin/routing/provider-route-models; POST /admin/routing/provider-route-model-verifications tries the route without registering it

provider_route_candidates, plus a strategy and required-role marker in site_settings

Add a route to an existing model

Add provider route

POST /admin/routing/provider-route-candidates/{model_id}; .../provider-route-candidate-verifications/{model_id} to check first; PATCH / DELETE /admin/routing/provider-route-candidates/{model_id}/{route_id}

provider_route_candidates

Point a registry route somewhere else

edit the route’s Target

PUT /admin/routing/provider-routes/{model_id}/{route_id}; .../provider-route-verifications/{model_id}/{route_id} to check first; DELETE removes the override and restores the YAML route

provider_route_configs

Change a route’s weight

the weight field on each route

PUT / DELETE /admin/routing/weights/{model_id}/{endpoint_id}; GET shows the YAML, override and effective values

provider_weight_overrides

Switch a model between fixed and routewise

the strategy selector

PATCH /admin/routing/provider-route-strategies/{model_id}

site_settings

Reserve a route for requests left waiting in the outbound queue or for an engine’s first token

Queue offload

PUT / DELETE /admin/routing/offload-routes/{model_id}; GET /admin/routing/offload-routes lists every model’s, and whether routing applies it — see Queue-wait offload

site_settings

Tune RouteWise for one model

RouteWise settings

/admin/routewise/model-settings — see RouteWise

site_settings

GET /admin/routing/provider-routes (or .../{model_id}) returns every route the gateway is serving, each tagged source: yaml, override or runtime, and is the quickest way to see what the two layers add up to.

The provider selector on this tab offers every custom provider, plus each built-in kind that the registry, the live route table or a configured credential already names. Which route types a provider may be added as — on_demand, quota or concurrency — is the deployment’s contract with that vendor and is declared with the PROVIDER_ROUTE_TYPES setting (see Environment variables); an unlisted provider may use any of the three.

A model created here is a runtime model: it exists only in the database, carries an id, a first route, a pricing table, a router strategy and a required_role (default admin, so a new model stays invisible to ordinary users until you lower it), and is restored at every start. It does not carry the catalog metadata a registry entry declares — context length, modalities, supported parameters, aliases — and takes ModelConfig’s defaults for those: context_length 8192, max_output_length 4096, text in and out, no tool support. When a model needs any of that, put it in the registry.

Settings tab

What

Console

Endpoint

Stored in

Change who can see a model

Model Visibility

PATCH /admin/models/{model_id}/visibility sets or clears a required_role override; GET /admin/models/visibility lists baseline, override and effective values

model_visibility_overrides

Exempt a model from the per-user concurrency limit

Model Concurrency Limit

PATCH /admin/models/{model_id}/concurrency

model_concurrency_exemptions

How the two layers combine

The registry is loaded first and the stored state is applied on top of it, in this order at boot; each admin change is also applied to the running router the moment it is saved.

  1. Custom provider definitions, skipping any slug the code or registry owns.

  2. Stored provider keys, seeded into each provider’s pool beside the environment keys.

  3. Per-model router strategy overrides.

  4. Runtime routes. A runtime model is restored only if its creation finished; a stored route whose model is no longer in the registry is skipped with a warning, so removing a model from YAML does not bring it back through a leftover row.

  5. Route overrides, retargeting the registry routes they name.

  6. Key tier reservations, re-read now that every provider is known; then weight overrides, disabled providers, offload routes and model visibility, loaded into resolvers the router consults at request time.

Two rules follow from that order. A stored change never edits the file it overrides: delete the override and the registry route, weight or router: value is back, and a runtime candidate sits beside the registry’s routes rather than replacing them. And the file still wins for anything the store has no row for, so a registry edit plus restart is how catalog metadata, aliases and new adapter kinds change.

Settings stored in the database

Everything the gateway reads that is not one of the files above, the database connection or container wiring is a setting: provider keys, SMTP, Slack webhooks, proxy trust, browser origins, logging, routing and timeout tuning, the cloud-agent contract, and the secrets that sign sessions and protect API keys. A gateway with a database keeps them in its app_config table, and an administrator edits them on the admin console’s Configuration tab. A gateway with DB_ENABLED=false has no stored settings and reads the environment and the built-in defaults only.

Where a value comes from

Each setting resolves in this order:

  1. its row in the database — even an empty one, which is how a setting is cleared;

  2. the environment variable of the same name;

  3. the built-in default.

The environment is a starting point. At every start, a setting that has a non-empty value in the environment and no row in the database is copied into the database, and from then on the row wins. That is how an existing .env carries over on an upgrade, and why editing the variable afterwards changes nothing: the tab marks it Environment ignored. Change the value on the tab instead, or Reset it, so that the environment, and then the default, applies again. A reset lasts: the database remembers it, so the next start does not copy the variable in again, and later edits to it take effect. The switches on the Settings tab — USER_AUTH_ENABLED, SIGNUP_ENABLED and the others — are copied from the environment the same way.

An environment value that is not valid on its own — a CIDR range that does not parse, a number that is not one — is not copied in. The backend then starts only if the database holds a valid value for that setting, which applies, with a warning in the log; otherwise it stops and names the variable. Fix or remove the variable, or store a valid value with python -m serving.config.manage set (see Without the console below), which works while the environment is invalid. The database connection settings (DB_HOST, DB_PORT and the others) are the exception: they are needed to read the database, so an invalid one always stops the backend.

Settings that depend on each other — TRUST_X_REAL_IP needs TRUST_PROXY_HEADERS — are judged together, the database’s values over the environment’s. A variable whose partner is already stored is copied in like any other, so it survives the variable’s removal. When the two sources together still break such a rule, neither side is copied in and the backend stops, naming the rule, until one side is corrected.

On the first start against an empty database the backend also generates the two secrets it cannot run without, JWT_SECRET_KEY and API_KEY_SECRET, unless the environment supplies them. API_KEY_SECRET and ERASURE_FENCE_SECRET cannot be changed once set, because records depend on them; see Database. The backend refuses to start rather than generate API_KEY_SECRET for a database that already holds API keys, since a new secret would invalidate every one of them.

The Configuration tab

The tab groups the settings by category, describes each one and shows where its value comes from. Besides every setting the gateway reads itself, it lists:

  • every ${VAR} the active model registry and routing file name, under Providers, with the models that use it, and those of the alert rules under Alerts;

  • numbered extra keys for a provider’s key pool, such as MINIMAX_API_KEY2, once one is set;

  • variables added with Add variable, for a ${VAR} reference the gateway cannot see yet, such as one in a registry you are about to switch to.

Secrets are write-only: the tab shows whether one is set, never its value, and the admin audit log records only that it changed. A save is checked as a whole, against everything it leaves in force, so settings that depend on each other — TRUST_CLOUDFLARE_HEADERS needs TRUST_PROXY_HEADERS — can change together. A save or a Reset that would leave another setting unable to apply is refused and names it: turning TRUST_PROXY_HEADERS off while TRUST_X_REAL_IP is on, for example, or clearing the Cloudflare networks while TRUST_CLOUDFLARE_HEADERS is on. Change both in one save. A Reset is also refused when the environment value it would fall back to is not valid. A setting that was already invalid never blocks an unrelated change. The names that stay in the environment, such as DB_PASSWORD or MODELS_CONFIG_PATH, cannot be set here. Behind the tab are GET and PATCH /admin/config and DELETE /admin/config/{key}.

When a change applies

Most settings apply as soon as they are saved: at once in the backend process that served the save, and within ten seconds in every other one. A setting marked Restart required is read only at startup, and the process keeps the value it started with until it restarts. Provider credentials and every variable the registry names are among them, as are the circuit-breaker and outbound-concurrency tuning, alerting and DB_STORE_FULL_CONTENT.

While such a change is waiting, the tab shows a Restart pending notice with a Restart backend button. The button (POST /admin/system/restart) makes the backend exit so that its supervisor starts it again: Docker, through the bundled Compose file’s restart: unless-stopped, or systemd, through the bundled units’ Restart=on-failure. Nothing inside the process can see that policy, so the deployment declares it: the bundled Compose file and units set SUPERVISOR_RESTARTS_BACKEND=1 next to it, and a deployment that starts the backend some other way sets it only if its supervisor restarts the process when it exits. Without it, and whenever several worker processes serve the backend (WEB_CONCURRENCY or UVICORN_WORKERS above 1, or uvicorn --workers, where the button would restart only one of them), the tab shows a command to run yourself instead, such as docker restart hybridinference-backend. A restart cuts off requests in flight, streams included. The console knows the restart is over once /health names a different process in its instance field, however quickly the backend came back.

Without the console

A setting can break the console that edits it — a value that stops sign-in from working, say. python -m serving.config.manage changes stored settings from the command line instead, with the console’s own validation:

docker exec -it hybridinference-backend python -m serving.config.manage list
docker exec -it hybridinference-backend python -m serving.config.manage reset JWT_ALGORITHM

list shows every setting, get KEY one of them, set KEY VALUE stores a value — add --secret when it creates a custom variable that holds one — and reset KEY resets it as the tab’s Reset does, so that the environment or the default applies again. It talks to the database directly with the DB_* connection settings alone, so it needs no sign-in and still runs when another setting in the environment is invalid, and it never prints a secret’s value. When the backend container is not running, run it in the repository root of a source checkout instead (uv run python -m serving.config.manage ...), where .env reaches Postgres on 127.0.0.1:5432. A running backend picks the change up within about ten seconds, or at its next restart for a setting marked Restart required. The command that resets an administrator’s password works the same way; see The first admin account.

Missing settings

Some settings are required: the two generated secrets, every variable a model route needs to load (unless the route is optional: true), and SMTP_USER and SMTP_PASSWORD while public signup requires email verification. When one of them has no value, the console shows a banner to signed-in users. Administrators get a link to the Configuration tab, where Missing only lists what to fill in; everyone else is told that some features may not work and to contact the administrator. A model whose credential is missing is not loaded, and a request for it is answered with a 404. When an administrator sends it with their own API key, the 404 says the deployment’s configuration is incomplete; anyone else gets the same “not found” as for a model the deployment does not define, so the difference reveals nothing beyond what they may use.

A deployment that leaves a model’s credential unset on purpose — a route it does not use yet, or a canary it enables elsewhere — should mark that route optional: true in the model registry. The route is then skipped quietly, and its variable is not reported as missing.

Warning

Settings are stored in plaintext. Anyone who can read the database — a pg_dump, a backup, a psql or pgAdmin session — can read every provider key, decrypt users’ stored API keys and sign administrator tokens. Protect database access and backups as you would the credentials themselves.

Environment variables

The environment keeps what the backend needs before it can reach its database and what describes the containers rather than the application:

  • the database connection: DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD, DB_ENABLED;

  • the configuration files: MODELS_CONFIG_PATH, ROUTING_CONFIG_PATH, ALERTS_CONFIG_PATH, DISTRIBUTION_CONFIG_PATH, DISTRIBUTION_CONFIG_MODE;

  • the backend process: LOG_FILE, BACKEND_EXTENSIONS, GEOIP_COUNTRY_DB, GEOIP_COUNTRY_PROVIDER, WEB_CONCURRENCY, SUPERVISOR_RESTARTS_BACKEND, and REFRESH_TOKEN_COOKIE_NAME, which the console reads too;

  • Compose and the console: host ports and bind addresses, the console’s build values and its runtime ones such as AGENT_WEB_INTERNAL_URL.

The gateway reads a .env file from its working directory and the process environment; the process environment wins. .env.example in the repository root is the annotated list — copy it to .env and edit. Installation lists the settings you are most likely to set. A value in the environment for any other setting is imported into the database at startup, as described above. Never write a secret into the model registry or the manifest: reference it there as ${VAR}.

Running with your configuration

uv run uvicorn serving.servers.app:app --no-proxy-headers --port 8080

Then check what actually loaded:

curl -s localhost:8080/health           # includes routes_configured
curl -s localhost:8080/v1/models        # generated from the registered adapters

The startup log says what was loaded. Registered N routes from <path> names the registry file that was used; [distribution dark mode] ... lines show what a manifest would change once activated; Skipping model '<id>' ... after env expansion names each model whose credentials were unset.

Warning

GET /routing requires no credentials and returns, for every published route, the upstream base_url, its provider label and its weight — that is, your complete upstream topology including any host and port embedded in a base URL. Block /routing at your reverse proxy on any gateway reachable from the internet unless you intend to publish this topology. GET /admin/routing additionally reports unpublished routes and requires an administrator’s JWT or ADMIN_TOKEN in the Authorization: Bearer ... header. Treat both endpoints’ output as sensitive.

Optional phone authentication

Phone signup, login and binding are disabled by default. Enable SMS_AUTH_ENABLED and select SMS_PROVIDER=tencent in Admin > Configuration only after delivery is configured. The gateway advertises features.phone_auth through /site-config; the shared forms show the phone option only when it is available. Existing email and first-run administrator logins continue to work.

The Tencent adapter requires TENCENT_CLOUD_SECRET_ID, TENCENT_CLOUD_SECRET_KEY, SMS_REGION, SMS_SDK_APP_ID, SMS_SIGN_NAME and SMS_TEMPLATE_ID. Temporary credentials also require TENCENT_CLOUD_TOKEN. Its approved template takes the code and its lifetime in minutes, in that order. This adapter currently sends to mainland China mobile numbers only. Keep real credentials, approved signatures and template IDs in deployment configuration, outside source control.

Account identifiers use international format, such as +8613812345678. SMS_DEFAULT_COUNTRY_CODE optionally supplies a calling code for local input; leave it empty to require international input. SMS_ALLOWED_PREFIXES optionally restricts destinations using comma-separated prefixes. For the Tencent domestic adapter, set the former to 86 if local input is desired and the latter to +86. Number policy belongs to the deployment, while provider delivery remains separate from account storage.

Codes expire after 300 seconds by default, allow at most five guesses, and can be consumed once. PostgreSQL serializes send and verification quotas across workers; the SMS_OTP_* settings control their bounds. Turnstile and trusted-proxy client IP handling remain shared with signup. Phone registration follows public-signup and approval policy, never automatically merges with an email account, and creates no password. Binding requires an authenticated session and a separate code for that account.

Startup adds phone identity columns and durable code tables and makes password hashes nullable. Upgrade every gateway worker before enabling phone signup. Back up the database first; once phone-only accounts exist, an older binary that assumes every account has a password is not a supported rollback. Disabling SMS hides phone login but does not remove accounts, so arrange another verified sign-in method before disabling it for phone-only users. Site UI v1 message overrides remain optional; distributions can translate the added phone message slots without owning authentication logic.