Are you an LLM? Read llms.txt for a summary of the docs, or llms-full.txt for the full context.
Skip to content
Runtime calibration

Write lessons from verified runs

This guide is for developers whose app can tell a good run from a bad one, and who want a good run to improve later runs. It stores one lesson and waits until Mubit has stored it. Runtime calibration concepts explains the terms.

The examples read your key from MUBIT_API_KEY. Python uses requests (pip install requests). TypeScript uses fetch and top-level await: save the file as .mts, such as lesson.mts, and run npx tsx lesson.mts.

Decide what counts as verified

Write a lesson only when something outside the model has checked the run:

  • a person confirmed the answer;
  • a comparison with a trusted reference passed;
  • a test or a rule your app owns passed.

A lesson from a run nobody checked can mislead every run that recalls it. Keep failed and unchecked runs out of the lessons. Report them as outcomes instead.

Store a lesson

POST /v2/control/ingest stores items under a run_id. For a lesson, set intent to lesson and describe it with the lesson_* fields. Use one run for the lessons you want to recall later, such as support-triage-lessons, and reuse it on every run of your app.

Python
import json
import os
import requests
 
response = requests.post(
    "https://api.mubit.ai/v2/control/ingest",
    headers={"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"},
    json={
        "run_id": "support-triage-lessons",
        "agent_id": "triage-agent",
        "idempotency_key": "lesson-double-charge-v1",
        "parallel": False,
        "items": [
            {
                "item_id": "lesson-double-charge",
                "content_type": "text",
                "text": (
                    "A message about being charged twice belongs in the billing "
                    "category, not the account category. A reviewer confirmed this "
                    "on the October invoice case."
                ),
                "intent": "lesson",
                "lesson_type": "success",
                "lesson_importance": "medium",
                "metadata_json": json.dumps({"kind": "category-rule", "contract": "v1"}),
            }
        ],
    },
    timeout=30,
)
accepted = response.json()
print(response.status_code, accepted)

The answer says the item is queued:

{
  "accepted": true,
  "deduplicated": false,
  "job_id": "…",
  "status": "queued"
}

Send the same idempotency_key again to retry without storing the item twice. The answer then says "deduplicated": true. Choose a key that names the lesson and its version, so a corrected lesson gets a new key.

Wait until it is stored

Ingestion is asynchronous, so a search sent at once may not find the lesson. GET /v2/control/ingest/jobs/{job_id} reports the job. Pass the same run_id as a query parameter. The job is finished when done is true: status is then completed or failed, and a failed job carries its reason in error.

Python
import os
import time
import requests
 
job_id = "<job_id from the store answer>"
headers = {"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"}
 
ingest_url = "https://api.mubit.ai/v2/control/ingest"
 
for _ in range(20):
    response = requests.get(
        f"{ingest_url}/jobs/{job_id}",
        params={"run_id": "support-triage-lessons"},
        headers=headers,
        timeout=30,
    )
    job = response.json()
    if job.get("done"):
        print(job["status"], job.get("error"))
        break
    time.sleep(0.5)

A finished job also lists, for each item, the records it wrote, with a record_id for each. Report outcomes uses that id to credit one entry.

Check what was stored

POST /v2/control/lessons with the run_id lists the lessons Mubit holds for that run. List lessons shows the request and the object it returns. Use it to confirm that a lesson was stored and to see its validation_status.

Keep stored lessons correct

  • Store what you checked. One lesson per verified run is enough.
  • Keep each lesson short and specific. Say what to do and when it applies. A lesson that names a rule and the case it came from reads better on recall.
  • Write the scope into the lesson. A lesson about one database, schema or contract can stop applying when that changes. Scope lessons per database and version shows how.
  • Retry with the same key. A retry with the same idempotency_key stores the item once.
  • Delete a run you wrote by mistake. Delete a run removes the run's entries and lessons.

Next