Are you an LLM? Read llms.txt for a summary of the docs, or llms-full.txt for the full context.
Skip to content
Concepts

Trust, confidence, and staleness

How an agent should decide whether to act on a memory — score vs knowledge_confidence, staleness and supersession, production verification, and memory health.

Retrieval tells you a memory is relevant; it does not tell you the memory is right. Mubit puts both signals on every evidence item so your agent can separate "this matches my question" from "this is safe to act on". This page explains each trust signal, where it comes from, and how to combine them into a decision.

Mental model: score answers "is this about my query?"; knowledge_confidence answers "how reliable is this knowledge in general?"; is_stale answers "is this still the current belief?". They are independent axes — read all three before acting.

Two different axes: score vs knowledge_confidence

scoreknowledge_confidence
MeasuresRetrieval relevance for this queryDurable reliability of the entry itself, in [0, 1]
Computed fromFusion of semantic, lexical, and recency components at query time (see Retrieval semantics)Reinforcement history from outcomes, lesson scope weight (run < session < global < org), the production-verification flag, and recency — minus a contradiction penalty when opposing lessons are known
Changes whenYou change the query, rank_by, or temporal boundsOutcomes are recorded, the lesson is promoted, or contradictions appear
Use it toOrder candidates for this questionWeight how much to trust each candidate, independent of topicality

A hot take from five minutes ago can have a high score and low knowledge_confidence; a battle-tested global lesson can have modest score on a tangential query and high knowledge_confidence. Neither substitutes for the other.

Staleness and supersession

Mubit's belief store is bi-temporal: it distinguishes when something happened (occurrence time) from when Mubit learned it (ingest time), and it never silently overwrites a belief — a new belief supersedes the old one.

On each evidence item:

  • is_stale: true — this entry has been superseded by a newer entry.
  • superseded_by — the ID of the entry that replaced it (empty when not stale).

Superseded beliefs are excluded from current-state answer synthesis (kept for history). When stale entries do appear in the evidence list, they are deprioritized in ranking (a 0.5x penalty, visible as stale_penalty_applied in the explain breakdown) and returned for transparency — prefer non-stale entries for answers, and follow superseded_by to the current belief. Time-bounded queries (min_timestamp/max_timestamp) filter on occurrence time, so history stays queryable.

Production verification

verified_in_production is the strongest trust signal an entry can carry. It is set by you, through the outcome path: record_outcome(..., verified_in_production=True) declares that the lesson was applied and verified in a live production environment — not a test or staging run. Production-verified entries receive a ranking boost and surface with higher knowledge_confidence. Reserve it for genuinely live verification; it is the signal other agents will lean on hardest. See The learning loop.

Response-level signals

  • degraded: true — answer synthesis fell back to a deterministic, no-LLM digest of the evidence (for example, the LLM was unavailable). The evidence list is still real retrieval output; treat final_answer as a grouped summary rather than reasoning, and prefer reading the evidence directly.
  • citations — 0-based indices into evidence that the synthesized answer is grounded in, validated server-side. An answer whose claims you can't trace to a citation should be treated as ungrounded.
  • confidence — the synthesizer's own confidence in the answer, distinct from any per-evidence field.

The decision table

SignalsRead it asAction
High score, high knowledge_confidence, not staleRelevant and reliableAct on it
High score, low knowledge_confidenceOn-topic but unprovenTreat as a hypothesis — verify before acting, then record_outcome
Low score, high knowledge_confidenceReliable but probably off-topicDon't force it into this answer
is_stale: trueSuperseded beliefFollow superseded_by to the current entry; use the stale one only for history
verified_in_production: true in metadata, high knowledge_confidenceBattle-testedSafe default when candidates conflict
Response degraded: trueSynthesis fallbackRely on the evidence list, not final_answer
Claim without a citation indexUngrounded synthesisVerify against evidence before repeating it

Memory health: the hygiene dashboard

memory_health() aggregates the same signals across a whole scope, so you can audit memory quality instead of discovering it one query at a time:

health = client.memory_health(
    session_id="support:acme",
    stale_threshold_days=30,
)
Response fieldWhat it tells you
entry_countsEntries per type — spot runaway trace growth or missing lessons
stale_entriesEntries older than the threshold or superseded
contradictionsPairs of similar lessons with opposing types — these drag knowledge_confidence down until resolved
low_confidence_countEntries below the confidence floor
promotion_candidatesLessons that have earned a move up the scope ladder
section_healthPer-section count, average confidence, and average age

Nimbus example: the Acme plan change

On May 3, customer Acme Corp upgrades from the Starter plan to the Scale plan. Nimbus's memory already holds the fact "Acme Corp is on the Starter plan"; ingesting the new plan fact supersedes it rather than deleting it. Afterwards, Ari's recall for "what plan is Acme on?" is synthesized from the current belief only, and the old fact comes back flagged:

{
  "id": "3d90c2f1-...",
  "content": "Acme Corp is on the Starter plan.",
  "entry_type": "fact",
  "score": 0.77,
  "knowledge_confidence": 0.31,
  "is_stale": true,
  "superseded_by": "b41e88a0-...",
  "retrieval_mode": "semantic"
}

Ari answers from the Scale-plan fact. Sana, the analytics agent, can still reconstruct April's state for her weekly digest by querying with a max_timestamp before May 3 — the superseded fact was excluded from current answers, not erased.

Related