Are you an LLM? Read llms.txt for a summary of the docs, or llms-full.txt for the full context.
Skip to content
Runtime calibration

Report outcomes

This guide is for developers whose apps recall stored lessons and can tell afterwards whether the run worked. An outcome records that verdict against the run, or against one entry the run used.

Choose a verdict

outcomesignalUse it when
success1A trusted check passed.
failure-1A trusted check failed.
partiala value from -1 to 1Part of the result was right.
neutral0The check could not tell.

signal is a number from -1 to 1. Mubit clamps a value outside that range. Any other outcome gets 400. Take the verdict from a check your app trusts, such as a reviewer, a reference answer or a test, and not from the model's opinion of its own answer.

Report on the whole run

Set reference_id to an empty string. The verdict then applies to the run.

Python
import os
import requests
 
response = requests.post(
    "https://api.mubit.ai/v2/control/outcome",
    headers={"Authorization": f"Bearer {os.environ['MUBIT_API_KEY']}"},
    json={
        "run_id": "support-triage-lessons",
        "reference_id": "",
        "outcome": "success",
        "signal": 1,
        "rationale": "A reviewer confirmed the billing category.",
        "idempotency_key": "outcome-ticket-1042",
    },
    timeout=30,
)
print(response.status_code, response.json())

The answer holds "success": true and an outcome_id. Report an outcome lists the other fields.

Report on one entry

To credit or fault the entry your app used, set reference_id to that entry's id. Take the id from the metadata.id of a search result, or from the record_id of a finished ingest job. An id that names no entry of yours gets 404.

{
  "run_id": "support-triage-lessons",
  "reference_id": "<entry id>",
  "outcome": "failure",
  "signal": -1,
  "rationale": "The reviewer moved the ticket to the account category.",
  "idempotency_key": "outcome-ticket-1043"
}

Report on one entry for each entry the run used, with the verdict that entry's use earned. These are the outcomes that decide a lesson's validation_status: a new lesson is pending, and once at least three outcomes have been reported against it, it becomes active or rejected by its success rate. An active lesson that keeps failing is rejected, and a rejected lesson stays rejected. Don't send entry_ids: a body with entry_ids gets 403 not_in_1_0.

Retry safely

Send an idempotency_key that is unique to the outcome, such as one built from your ticket or request id. Mubit applies an outcome once for each key. A retry with the same key, for example after a timeout, answers "success": true and changes nothing. Don't reuse a key for a different outcome: Mubit will treat the second one as a retry and skip it.

Next