Scope lessons per database and version

This guide is for developers whose lessons hold only for one setting, such as one database, one schema version or one business contract. A lesson that is right for schema version 1 can be wrong for version 2. Mubit stores and returns lessons and does not know which setting each one is for, so your app records the setting with each lesson and checks it on recall.

mubit-neon-db-agent is a public template that does this for an agent that answers questions from a Neon database. Everything it does with runtime calibration is in one file of its src folder, the one that holds its Mubit client. This guide follows it.

The pattern

1. Write the scope into the lesson. The first line of the lesson text carries a small JSON header with the scope. The lesson then carries its scope back when it is read.
2. Keep the scope in metadata_json too, so POST /v2/core/lookup can match on it.
3. Recall, then filter. Search by meaning, decode each entry's header, and keep only the entries whose scope matches the current setting.
4. Supersede, don't edit. A corrected lesson names the lesson it replaces in its header. The reader skips the replaced one.

What the template scopes by

Each lesson in the template has a kind, and each kind has its own rule:

A schema lesson is the one that sets itself aside after a schema change. The app computes a hash of the schema it sees and stores it in the header. After a migration the hash differs, so the old mappings stop matching and the agent inspects the new schema itself. The lessons are not deleted: a branch still on the old schema keeps using them.

Store the scope with the lesson

The template stores each lesson as a text entry whose first line is a header. Mubit keeps the text as it is, so the header comes back with the entry.

import json
import os
import requests
 
HEADER = "[my-app lesson]"
 
scope = {"db": "orders-prod", "contract": "2", "schema": "a41f09c2"}
lesson_text = "Revenue by month is the sum of orders.total_cents, grouped by month."
 
header = {"v": 1, "id": "lesson-revenue-by-month", "kind": "schema", **scope}
text = f"{HEADER} {json.dumps(header)}\n{lesson_text}"
 
response = requests.post(
    "https://api.mubit.ai/v2/control/ingest",
    headers={"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"},
    json={
        "run_id": "my-app-lessons-orders",
        "agent_id": "my-app",
        "idempotency_key": "lesson-revenue-by-month-a41f09c2",
        "parallel": False,
        "items": [
            {
                "item_id": "lesson-revenue-by-month",
                "content_type": "text",
                "text": text,
                "intent": "lesson",
                "metadata_json": json.dumps({"kind": "schema", **scope}),
            }
        ],
    },
    timeout=30,
)
print(response.status_code, response.json())

Use one run for each database and namespace (my-app-lessons-orders here; the template uses one run for each namespace), so a search over the run never meets another database's lessons. The scope in the header still guards against mistakes: the run is a convenience, the check is the guarantee.

Recall and keep the lessons that apply

Search the run, decode each entry's header, and keep the entries whose scope matches. This is the rule the template applies, written for the three kinds above.

import json
import os
import requests
 
HEADER = "[my-app lesson]"
 
 
def decode(text):
    at = text.find(HEADER)
    if at < 0:
        return None
    rest = text[at + len(HEADER):].lstrip()
    first, _, body = rest.partition("\n")
    try:
        header = json.loads(first)
    except ValueError:
        return None
    return {**header, "text": body.strip()}
 
 
def applies(lesson, current):
    if lesson.get("db") != current["db"] or lesson.get("contract") != current["contract"]:
        return False
    if lesson.get("kind") == "schema":
        return lesson.get("schema") == current["schema"]
    return True
 
 
current = {"db": "orders-prod", "contract": "2", "schema": "b77d31e0"}
 
response = requests.post(
    "https://api.mubit.ai/v2/core/search",
    headers={"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"},
    json={"query": "revenue by month", "run_id": "my-app-lessons-orders", "k": 20},
    timeout=30,
)
lessons = [decode(e["metadata"]["content"]) for e in response.json()]
usable = [l for l in lessons if l and applies(l, current)]
replaced = {l["sup"] for l in usable if l.get("sup")}
usable = [l for l in usable if l["id"] not in replaced]
print(len(lessons), "returned,", len(usable), "apply")

The lesson stored above has the schema hash a41f09c2. The app now sees b77d31e0, so the example drops it and keeps semantic lessons, which don't depend on the schema hash. The agent then works out the new mapping itself and stores a new schema lesson under the new hash, once a run is verified.

Two details from the template are worth copying:

• Only a lesson that applies can supersede another. A version 2 mapping that names a version 1 lesson in sup must not hide that lesson from a branch that is still on version 1. The example builds replaced from the usable lessons only.
• Show what was set aside. The template lists the lessons it excluded, with the reason, such as "schema v1 mapping; this database is v2". It makes a schema change visible instead of a silent loss.

What Mubit does and doesn't do here

• Mubit stores the text and metadata_json you send and returns them. It doesn't read the header, and it doesn't skip a lesson because its scope is wrong.
• POST /v2/core/lookup can match the fields you put in metadata_json, for example {"match": [{"schema": "a41f09c2"}]}, so your app can also read back one schema's lessons without scoring.
• Deleting is for mistakes. Delete a run removes a whole run. A schema change is not a reason to delete: the old lessons still fit the old schema.

Next

• Report outcomes
• The template's source