Are you an LLM? Read llms.txt for a summary of the docs, or llms-full.txt for the full context.
Skip to content
Route model calls

Calls answered by an adapter

When a task in your project has an adapter version in a partial rollout or live, Mubit can answer a call with that adapter instead of your provider. Your app sends the same request, with the same model ID. This page says which calls it can apply to, how to see it in the console, and what you can do to stop it. It is for anyone whose app runs through Mubit and who needs to know exactly which model answers.

ℹ️Note

This is on by default in the workspaces on app.mubit.ai, for every project there. You don't opt in to it. A task has an adapter only after Mubit has trained one for it: see Train adapters.

What happens to a call

For each call, before it sends the call to the provider, Mubit checks whether the call belongs to a task that has an adapter ready to answer. It does this in order, and stops at the first step that fails. The provider then answers, as it always does.

  1. The call can be considered. Today, that means a Chat Completions call on /openai/v1/chat/completions, with an OpenAI key or a Neon key, that is sent at the default recording level, as a whole body of at most 256 KiB with no compression. Calls to Anthropic, Gemini and the OpenAI-compatible services always go to the provider.
  2. A task matches. Your project has a task with a version in its rollout, and the call has the model ID and the instructions that define that task: see Tasks and sessions. A version in a silent test only gets a copy of the call after your provider has answered, and one in a partial rollout takes only its share of conversations: see The rollout.
  3. The call fits the adapter. A call with an image, a document or audio, or one longer than the adapter's context (8,192 tokens today), goes to the provider.
  4. Mubit is sure. An automated service Mubit uses for this decision gets the call's instructions and its last user message, each cut to its first 2,000 characters, with a few example messages of each task it compares the call with, and says how well the call fits each task that has an adapter. The adapter answers only when the fit is high. A fixed threshold sets what high means; you can't change it.
  5. The adapter is ready. If the adapter isn't ready to answer just now, the provider answers, and the call doesn't wait.

If any step fails, or takes too long (the decision has about a second), the provider answers as if the adapter didn't exist. When the adapter's reply fails Mubit's checks, such as a malformed tool call, Mubit sets it aside and your provider answers.

When the adapter answers, Mubit doesn't send the call to your provider for its reply. A sample of these calls still goes to your provider, on your key, as spot-check copies, described below.

The answer has the shape of the API you called, and the model ID is the one your app sent. An answer that an adapter gave also carries the header x-middleware-served-by, and its value names the version that answered, as version=<n>. An answer from your provider doesn't carry it. Read it from the answer's headers, for example response.headers.get("x-middleware-served-by").

While a version answers calls, Mubit sends copies of a sample of them to your provider, on your key, to compare the replies: about 2% of the calls each version answers by default, at least 10 and at most 500 a day. In a silent test, a call the live version answered is also asked of your provider again, for the comparison. Your provider bills those calls to you. See Costs and data.

See which calls an adapter answered

In the console, Requests shows Served by for each call, and the Served by filter lists the calls one server answered. Calls an adapter answered are recorded: you can open one and read its request and reply, as with any call. Read who answered in Requests lists each value, and the copies Include copies adds.

Stop it

There are two ways. They differ in what else they change.

To stopHowWhat else changes
One callSend x-middleware-record: metadata or off, or x-middleware-store: false, with the call.Mubit also lowers what it records of that call: see What Mubit records.
One adapter, for every call of its taskIn the console, open the adapter's page and select Switch off…. Emergency stop… stops every version of it at once.The task's calls go to your provider. The adapter stays, and Switch on… brings it back.

Nothing turns routing off for a whole project, and no API operation does it. The management API's operation for changing a task's serving isn't served today. If you need the provider to answer every call, send each call at metadata or off.

Next