Utility-weighted forgetting

Bounded memory growth without losing what matters — unused episodic evidence archives out reversibly while credited, retrieved memories stay hot.

A busy agent fleet writes episodic evidence far faster than it rereads it: traces, logs, tool outputs. Left alone, retrieval wades through an ever-growing candidate pool where the signal never got denser. Utility-weighted forgetting is the opt-in instance feature that moves aged, never-used episodic evidence to an archive tier — out of retrieval, still reachable by ID, and reversible. Use this pattern on long-lived, high-volume scopes where most episodes are routine.

How it works

The idle-time consolidation worker (the same one behind sleep-time consolidation) runs an archival pass per sweep. An entry moves to the archive tier only if all of these hold:

• Episodic evidence only. Only trace, log, tool_input, tool_output, and step_outcome entries qualify. Guidance (lessons, rules, mental models), beliefs (facts, observations), handoffs, and archive_block artifacts are never archived — beliefs age out via supersession, guidance via consolidation.
• Aged. Older than the minimum window (default 7 days). Entries with no parsable timestamp are never archived — no evidence of age is not evidence of disuse.
• Never credited. No positive outcome ever attributed to it (its success and reinforcement counters are zero) — see The learning loop.
• Not recently used. Not retrieved and not outcome-touched within the window. A retrieval resets the clock — being read is being useful.

Archived entries leave the retrieval candidate pool and all overlays, but remain reachable by ID (dereference / get-by-id), so stored references never dangle. Each sweep archives at most a bounded batch, so a first pass over a large backlog cannot flood the store.

Reversibility. Archival writes markers; it does not destroy anything. An un-archive lever (MUBIT_CL_UNFORGET=1) suppresses forgetting and progressively revives previously archived entries on each sweep until the archive tier drains — a standing rollback, not a one-shot restore.

Why utility beats TTL

A TTL asks one question — how old is it? — and deletes the answer. Utility-weighting asks has it ever mattered? using signals the store already tracks:

Because outcome attribution marks which entries contributed to results, value — not age — decides retention. And it's archival, not deletion, so a wrong call costs a revival, not data.

Walkthrough

Nimbus's support scope accrues thousands of traces from routine, resolved tickets. The incident run's evidence, though older, stays hot: its traces were credited when the failure lesson was distilled, and the lesson itself is guidance — never archivable.

# Watch the episodic tier shrink as archival sweeps run (opt-in feature).
health = client.memory_health(session_id="support:ari", stale_threshold_days=30)
print(health["entry_counts"])
# Routine ticket traces age out of retrieval; credited incident
# evidence and every lesson/rule/fact stays.
 
# Distinct operation: explicit, immediate deletion.
client.forget(lesson_id=bad_lesson_id)      # delete one lesson
client.forget(session_id="support:scratch") # delete a session's memory
# Exactly one of lesson_id / session_id — deletion is permanent,
# unlike archival.
 
# archive() artifacts are exact-storage, not the archive tier:
# dereference() keeps working regardless of forgetting.
exact = client.dereference(reference_id=sql_ref_id)

When not to use it

• Regulated or audit-bound evidence with hard retention rules. Archival is a retrieval-tier decision, not a records policy. Manage retention explicitly rather than assuming what archives or survives.
• Low-volume scopes. If retrieval isn't wading through noise, leave it off — the default is off for a reason.
• You actually need deletion. Poisoned entries, user data removal: forget() (or delete_run) deletes; archival deliberately doesn't.
• Evidence that is rarely read but must stay retrievable by content. Archived entries are ID-reachable, not searchable. If it must surface in future queries despite years of disuse, attach an outcome to it (credit pins it) or store it as guidance instead of a trace.

Related

• The learning loop — outcome attribution, the credit signal that keeps memories alive
• Sleep-time consolidation — the idle worker this pass runs in
• Bi-temporal memory — how beliefs age (supersession), as opposed to evidence
• The memory model — the episodic entry types this feature owns, and archive_block verbatim storage
• Trust, confidence, and staleness — memory_health() for watching scope hygiene