Runtime calibration

Runtime calibration lets your app keep what it has learned. Your app stores knowledge it has checked, such as a lesson from a run whose answer a reviewer confirmed. On a later run, it asks Mubit for the stored items that fit the new question, uses them, and reports whether the run worked.

How it fits with your model calls

Runtime calibration uses the same Mubit key and the same host as your model calls: https://api.mubit.ai. Model calls go to a provider's prefix, such as /openai/v1. Runtime calibration calls go to paths that start with /v2. Your app can use it whether or not it sends model calls through Mubit.

• Key: your project's Mubit key, in the Authorization: Bearer header. Use an Inference key. Another kind of key, such as a read-only key, gets 403 wrong_lane: the key is of the wrong kind. To create a key, follow step 1 of the Quickstart.
• Records: Mubit keeps them per project. A key reads and writes the records of its own project only.

The calls

The API reference lists each call's fields, answers and errors. The examples below follow one pattern: check, store, recall, report.

The Python examples use requests: install it with pip install requests. The TypeScript examples use fetch and top-level await, so run them as ES modules: save the file with an .mts name, such as calibration.mts, and run npx tsx calibration.mts, or add "type": "module" to your package.json. The examples read the key from MUBIT_API_KEY.

A worked pattern

The example is a support triage app. A reviewer confirmed that a message about a double charge belongs in the billing category. The app stores that as a lesson, recalls it when a similar message comes in, and reports the outcome of the answer.

All the calls use one run_id, support-triage. A run groups the items you store together. Search with the same run_id to read them back.

1. Check your key

A good key gets 200 and the text OK.

import os
import requests
 
response = requests.get(
    "https://api.mubit.ai/v2/core/health",
    headers={"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"},
    timeout=30,
)
print(response.status_code, response.text)

2. Store a lesson

Store only what you have checked, for example after a reviewer's confirmation or a comparison with a trusted reference. Mubit stores the text you send and doesn't judge it.

Send each lesson as one item with intent set to lesson. item_id is your own id for the item. metadata_json is a JSON object written as a string: use it for fields you want to keep with the item.

import json
import os
import requests
 
lesson = (
    "A message about being charged twice belongs in the billing category, "
    "not the account category. A reviewer confirmed this on the October invoice case."
)
 
response = requests.post(
    "https://api.mubit.ai/v2/control/ingest",
    headers={"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"},
    json={
        "run_id": "support-triage",
        "agent_id": "triage-agent",
        "idempotency_key": "lesson-double-charge",
        "parallel": False,
        "items": [
            {
                "item_id": "lesson-double-charge",
                "content_type": "text",
                "text": lesson,
                "intent": "lesson",
                "lesson_type": "success",
                "metadata_json": json.dumps({"kind": "category-rule"}),
            }
        ],
    },
    timeout=30,
)
print(response.status_code, response.json())

The answer says the item is queued:

{
  "accepted": true,
  "deduplicated": false,
  "job_id": "…",
  "status": "queued"
}

Ingestion is asynchronous. The call returns when the item is queued, so a search sent at once may not find it yet. Write lessons from verified runs shows how to wait for the job.

To retry a store without storing the item twice, send the same idempotency_key again. The answer then says "deduplicated": true.

3. Recall on a later question

When a similar message comes in, search the run with the new message as the query. Each entry in the answer has a score, where a higher score is a closer match, and the stored text in metadata.content.

import os
import requests
 
response = requests.post(
    "https://api.mubit.ai/v2/core/search",
    headers={"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"},
    json={
        "query": "Why did my card get charged two times this month?",
        "run_id": "support-triage",
    },
    timeout=30,
)
for entry in response.json():
    print(round(entry["score"], 2), entry["metadata"]["content"])

Put the entries that apply in your prompt, then send the model call as usual, for example as Connect with the OpenAI SDK shows. Search returns the closest matches whatever they say, so check that an entry applies before you use it.

4. Report the outcome

When you know whether the run worked, report it. Set reference_id to an empty string to report on the whole run. Use outcome success with signal 1, or failure with signal -1, and a short rationale.

import os
import requests
 
response = requests.post(
    "https://api.mubit.ai/v2/control/outcome",
    headers={"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"},
    json={
        "run_id": "support-triage",
        "reference_id": "",
        "outcome": "success",
        "signal": 1,
        "rationale": "A reviewer confirmed the billing category.",
    },
    timeout=30,
)
print(response.status_code, response.json())

The answer holds "success": true and an outcome_id. A reference_id must be an empty string or the id of an entry that exists, or the call gets 404. Don't send entry_ids: the call gets 403 not_in_1_0.

Keep stored lessons correct

• Store what you checked. A lesson from an unchecked run can mislead every later run.
• Say where a lesson applies. A lesson about one database, schema or contract can stop applying when that changes. Put its scope in the text or in metadata_json, and check it against the current setting when you read the entry back. Scope lessons per database and version shows how.
• Retry with the same key. Send the same idempotency_key when you retry a store, so the item is stored once.

Other calls

• Read back a run: POST /v2/core/lookup with a run_id returns entries stored under that run, and can match the fields you stored.
• List lessons: POST /v2/control/lessons with a run_id lists the lessons held for the run.
• Read approved lessons: POST /v2/loop/context with a session_id serves only lessons a person approved. Until approved lessons are served, the answer holds no lesson.
• Checkpoint, archive and delete: POST /v2/control/checkpoint saves the context your agent holds, POST /v2/control/archive stores one exact block, and POST /v2/control/delete_run deletes a run's entries and lessons.

The API reference shows each one.

Limits and errors

Each key can send up to 60 POST calls and 600 GET calls a minute. Past that, the call gets 429 rate_limited, with the seconds to wait in Retry-After.

Errors have an error object with a code and a message, and each answer carries its request ID in an x-request-id header (it starts with req_). Runtime calibration errors lists the codes and what to do.

A template that uses it

mubit-neon-db-agent is a public template for a database agent on Neon. It uses this pattern, and its src folder shows it in code:

• it stores up to three short lessons from each run a reviewer or a trusted reference confirmed;
• it recalls lessons with a search for each new question, and checks each one against the current database and schema before it uses it;
• it reports each outcome with the outcome call, and retries a failed store with the same idempotency_key.

Go further

• Runtime calibration concepts: runs, entries, lessons, recall, outcomes and checkpoints.
• Write lessons from verified runs
• Recall lessons in a later run
• Report outcomes
• Scope lessons per database and version
• Use the SDKs, with the SDK helpers that work on the hosted service and the ones that don't.

Next

• API reference: runtime calibration
• Mubit keys
• Troubleshooting