Model health

A roster pins each agent to a model_name on a provider. Providers retire ids, rename them, and sometimes advertise ids they will not serve — and a dead id fails every request the agent is ever given. An agent therefore judges its own model and benches itself, so the scheduler stops assigning it work.

There are two detectors, and they answer different questions.

Reactive detection alone means the first caller routed to a dead seat pays for the discovery. The proactive probe exists so nobody does.

What the proactive probe checks

One free request, throttled to once every five minutes: fetch the provider's OpenAI-compatible /models list and look for the id. A successful fetch that omits it is a verdict.

ModelAvailability::new(format!("{base}/models"), provider_id)

The probe never sends a completion. A catalog is an advertisement — a provider can list an id whose /chat/completions answers 404 — but confirming that costs a billed request per seat per interval, and a fleet doing so paid for a token on every tick. A listed-but-unserved model is instead caught by the reactive detector on its first real task.

Context length from the same catalog

The probe also reads each entry's stated context length — context_length (OpenRouter), then max_context_length, then context_window, then top_provider.context_length — and quorum serve pushes the agent's context_window to that value at startup, in both directions: an understated yaml wastes context the provider serves (the shrink-guard clamps output against the stated window), an overstated one ships requests the provider rejects. The catalog, not the yaml, is the authority on what the provider serves. When the catalog is unreachable or its entry states no context field, the configured value stands (fail-open); the change is logged as context_window set from provider catalog.

Fail-open is the invariant

Availability::Unavailable is returned only when the provider says the model is not there — omitted from a successfully fetched catalog. Everything else is Availability::Unknown, which never benches:

• a catalog fetch that failed
• a rate limit, an auth failure, a provider 5xx, a transport error
• an agent on a provider this probe does not cover
• a probe that has not run yet

The reason is blunt: every agent in a fleet shares a provider, so a rule that benched on any error would turn one provider hiccup into a total outage. A model that is briefly unreachable keeps serving; only one that is reported gone stops.

Benching

A verdict of Unavailable arms model_down for an escalating cooldown — repeat strikes bench progressively longer, capped — and the heartbeat reports model_down while the deadline stands. The scheduler excludes a down-model agent from assignment even while it is otherwise online, and the bench clears on expiry so a transient outage does not sideline an agent permanently.