Runtime calibration concepts

This page is for developers who build an app that keeps what it has learned from its own runs. It names the parts of runtime calibration and says what each one does, so the guides and the API reference read clearly. Runtime calibration shows the whole loop in code.

The loop

1. Write. After a run your app has checked, it stores what it learned as a lesson.
2. Recall. On a later run, it asks for the stored entries that fit the new question, and keeps the ones that apply.
3. Report. It tells Mubit whether the run worked. That is an outcome.

Mubit stores and returns what your app sends. It does not decide for your app whether a lesson is true or whether it applies. Your app does both.

Runs

A run is the group an entry belongs to. Every write names a run_id, and a read names the same run_id to see that group. Mubit keeps runs per project: a key reads and writes the runs of its own project only.

Choose the run ids to match how your app reads:

• A calibration run is a long-lived run that holds lessons for many later runs, such as one run per app, or per database and version. Lessons you want to recall later belong in one.
• A session run holds the entries of a single task or conversation.

A run_id is required on every route that writes or lists a run's records. A blank one gets 403 not_in_1_0. The SDKs call a run a session: their session_id argument is sent as run_id.

Entries and kinds

Everything your app stores is an entry. The intent of the item you send sets the entry's kind. The kinds the store accepts:

With no intent, Mubit chooses a kind from the text. Set it yourself for anything your app will read back by kind.

Lessons

A lesson is an entry of kind lesson. A lesson item can carry:

A lesson that Mubit holds has a validation_status: pending, active or rejected. A new lesson starts pending. Outcomes you report against it count toward its validation: once at least three have come in, a pending lesson becomes active or rejected by its success rate, and an active lesson is rejected after sustained failures. A rejected lesson stays rejected. A project can also be set to propose each change for a person to confirm. List lessons shows the object.

Write lessons from runs you have checked, such as a run a reviewer confirmed or one compared with a trusted reference. A lesson from an unchecked run can mislead every run that recalls it. Write lessons shows how.

Scope

A lesson usually holds for one setting only: one database, one schema version, one business contract. Mubit has no field that makes a read skip lessons from another setting. Scope is something your app records and checks:

• Record it with the lesson, in metadata_json and in the lesson text, so it comes back with the entry.
• Check it when your app reads entries back, and set aside the lessons whose scope does not match the current setting.
• Supersede it when it turns out wrong: store the corrected lesson with a field that names the lesson it replaces, and skip the replaced one when your app reads back.

A lesson written for one schema version sets itself aside when the app moves to another version, because the app compares the recorded version with the current one and skips the mismatch. Scope lessons per database and version shows a worked example.

Recall

Recall reads entries back. Two routes do it:

• Search returns the entries closest in meaning to a question, with a score. It is by meaning, so it returns the closest entries whatever they say. Check that an entry applies before your app uses it. Search sees an entry only after its ingest job has finished.
• Look up returns the entries of a run in order, with no ranking, and can match exact metadata fields. It is the right route to read back an entry by a field you stored.

Neither route returns only lessons. Both can return entries of any kind in the run.

A third route, the approved lessons read for a session, serves only lessons a person approved. Until a lesson is approved, it answers with none.

Outcomes

An outcome tells Mubit whether a run, or one entry, worked: success, failure, partial or neutral, with a signal from -1 to 1 and a short rationale. Set reference_id to an empty string for the whole run, or to the id of an entry to credit or fault that entry. An outcome carries an idempotency_key so a retry applies once.

Report outcomes from a check your app trusts, not from the model's own opinion of its answer. Report outcomes shows how.

Ingest jobs

Storing is asynchronous. A write returns a job_id as soon as the items are queued, and the job runs in the background, in order within a run. Check an ingest job reads its status, so an app can wait until an entry is stored before it searches for it.

Checkpoints, archives and deleting a run

• A checkpoint saves the full context your agent holds at a point in a run, such as just before it compacts that context, with a label of your choice.
• An archive stores one exact block of content, such as a tool result, under a kind you name. It returns a reference_id for that block.
• Deleting a run removes the run's entries and lessons. It cannot be undone.

Next

• Write lessons from verified runs
• Recall lessons in a later run
• Report outcomes
• Scope lessons per database and version
• Use the SDKs