Are you an LLM? Read llms.txt for a summary of the docs, or llms-full.txt for the full context.
Skip to content
Help

Troubleshooting

Common issues, error messages, and solutions for Mubit SDK and API usage.

Start with mubit doctor

The CLI ships with the Python package (pipx install mubit-sdk). mubit doctor prints the endpoint, the masked key, the server mode, the resolved scope, the health probe and one line per problem; it exits 1 when it found any.

mubit doctor            # human-readable
mubit doctor --json     # the same fields as JSON
endpoint:      https://api.mubit.ai
key:           mbt_acme_k1_****
mode:          loop_v1 (server 0.4.0, policy starter@1)
scope:         project=first-project env=dev agent=(unset)
health:        ok (/v2/core/health)
deprecations:  0 distinct warnings emitted in this process

mode is loop_v1 (full loop), legacy (server without /v2/loop: run-level outcomes are ignored and run end reflects synchronously) or unreachable. mubit status --agent <id> adds events_received, reflections_scheduled/reflections_finished, lessons_stored, last_injection and last_outcome.

Authentication

Invalid API key

Symptom: AuthenticationError (HTTP 401) from Client, or, on the loop helpers, one line:

WARNING mubit: rejected by https://… (HTTP 401 Unauthorized: ACL error: Invalid API key). Memory is off for this process. Check MUBIT_API_KEY, or run `mubit doctor`.

The helpers then continue without memory; ContextBlock.reason is unauthorized.

CauseFix
Malformed keyKeys follow the format mbt_<instance>_<key_id>_<secret>. Verify all segments are present.
Wrong instanceThe instance tag in the key must match the endpoint's instance. A local key is accepted only by a local endpoint; a hosted key only by api.mubit.ai.
Revoked keyCreate a new key in the console under Settings → API keys → your project.
Missing env varSet MUBIT_API_KEY, or pass mubit.init(api_key=...) / Client(api_key=...).

To fail hard instead of warning, use mubit.init(on_error="raise") or MUBIT_ON_ERROR=raise.

Permission denied

Symptom: PermissionDeniedError (HTTP 403) with a body such as principal manager cannot set the kill switch; needs admin.

Control-plane writes need a principal of the named kind. The key's role sets the principal (Admin key → admin, User key → worker); x-mubit-principal can only lower it. See Loop console → principals.

Transport and connectivity

Connection refused

Symptom: APIConnectionError: connection refused by 127.0.0.1:3000 (gRPC and HTTP). Is Mubit running? MUBIT_ENDPOINT=http://127.0.0.1:3000 from Client, or WARNING mubit: cannot reach http://127.0.0.1:3000 (connection refused). Running without memory; will retry on the next call. Set MUBIT_ENDPOINT if this is the wrong server. from the helpers.

ContextFix
Local developmentEnsure Mubit is running. The SDK fails a refused connection within connect_timeout (2 s) and retries only 429/5xx; the loop helpers continue without memory and print one warning.
Hosted endpointCheck network connectivity to api.mubit.ai. Verify no firewall blocking HTTPS/443.
Wrong serverA default of 127.0.0.1:3000 in the message means MUBIT_ENDPOINT is unset for this process (the CLI and learn paths still default to the local address in 0.14). Set it to https://api.mubit.ai or copy the four .env lines from the console.
gRPC transportVerify MUBIT_GRPC_ENDPOINT points to the correct host
.

Transport selection issues

Symptom: Unexpected transport used, or auto-detection fails.

Set MUBIT_TRANSPORT explicitly to "http" or "grpc" instead of relying on "auto". The auto probe waits connect_timeout (2 s) for gRPC readiness before it falls back to HTTP; set MUBIT_CONNECT_TIMEOUT_MS if your network needs longer.

gRPC field names wrong (JS SDK)

Symptom: Response fields use camelCase instead of snake_case proto names, or vice versa.

Fix: Update @mubit-ai/sdk to the latest version. Current SDKs decode proto responses with keepCase: false by default, so fields surface as camelCase. Option objects are camelCase-only in 0.14; a snake_case key is accepted with one console.warn and rejected in 1.0.

The loop

No lesson injected on run two

CauseFix
Different envLessons are partitioned by env. Both runs must use the same MUBIT_ENV (or env= argument). The ContextBlock.scope of run two shows which project/env was used; MUBIT_LOG=info prints context: 0 units for env=prod project=default.
Run one had nothing to learn fromA run needs a correction, a tool failure or an outcome. Record one with run.outcome(good=False, label="...").
Reflection not finishedThe server distils a few seconds after run.end. mubit status --agent <id> shows reflections_finished and lessons_stored.
Run one never endedSee Lost turns.
Kill switch onThe SDK prints WARNING mubit: kill switch is on for env=dev (reason: "…"); injection disabled. once; reason is killed. Turn it off in the console (admin) or mubit kill off.
Legacy serverWARNING mubit: server at … has no /v2/loop (legacy mode). Upgrade the instance.

Lessons not appearing after reflect

CauseFix
Looking in the wrong listUnder the built-in starter@1 policy, lessons distilled at run end are stored active with scope=session (confidence 0.5) and become visible to other runs in the same env according to the overlay gate (same actor, or promotion, or gate.min_actors distinct actors). Explicit lessons written with remember(kind="lesson") or note(intent="lesson") are gated (gate_notes=true) and enter as pending; mubit lessons list --status pending shows them.
Gate mode is propose or externalThe lesson is waiting for review. Open Proposals in the console or mubit proposals list.
Run never endedA process that exits without ending its run (os._exit, process.exit(), a crash) loses the turns it captured but had not flushed; call mubit.shutdown() or use with mubit.run(): / await mubit.run(fn).
Not sure what happenedRun mubit doctor, then mubit status --agent <id> and mubit lessons transitions --run <run_id>.

Lost turns when the process exits

Symptom: 
WARNING mubit: 2 captured events were not sent before exit (run run-…); call mubit.shutdown() or end the run inside `with mubit.run():`.

The capture worker posts events in the background. with mubit.run() (Python) and await mubit.run(fn) (JS) flush on exit; mubit.shutdown(timeout=2.0) flushes when you manage runs yourself. os._exit() and process.exit() skip both.

OutcomeReceipt(sent=False)

reasonMeaning
nothing_to_recordNo good, score or label was given, or there is no current run
uninitializedNo mubit.init() and nothing resolvable from env
disabledMUBIT_DISABLED=1 or init(disabled=True)
unreachable, unauthorized, killed, legacySee the rows above

Memory and retrieval

Recall returns empty results

CauseFix
Data not yet ingestedclient.memory.remember() waits for the ingest job by default (wait=True). If you wrote through client.raw.invoke("control.ingest", ...), poll client.jobs.get(job_id, run_id=...) until done is true.
Wrong run_idrecall reads the current run. Pass run_id= to read another run, or store the item with visibility="global".
Entry-type filter too narrowBroaden entry_types (kinds on mubit.recall()) or omit it to search all types.
Lane mismatchIf using lanes, ensure the lane on recall matches the lane used at write time.

Context block is empty or too small

CauseFix
Zero units with degraded=FalseCheck ContextBlock.scope: the read used a different project/env than the writes.
degraded=TrueRead reason; the matching warning was printed once earlier in the process.
Token budget too lowbudget is enforced after assembly; the lowest-score units are dropped first. Raise it (inject.budget_tokens in the policy sets the default).
No relevant evidenceIngest more data or broaden the task text.

Runs and scoping

Run scope conflicts

Symptom: Data from one run appears in another, or data is missing.

  • Runs scope memory writes. Lessons additionally carry a scope (run, session, global) and are partitioned by env.
  • Linked runs (client.runs.link(...)) share data intentionally. Check whether runs are linked.
  • Inspect runs and their relationships with client.runs.history(...) or GET /v2/control/runs; the Runs tab of the console lists loop runs with their injections and outcomes.

Cannot delete a run

Symptom: client.runs.delete(run_id) fails or data persists.

  • Run deletion is permanent and removes all associated data.
  • Ensure no active ingest jobs are running for the run.
  • Both control and core planes can delete a run; client.runs.delete uses the control version, which handles both.

Lesson lifecycle

Lessons carry status (pending, active, rejected, retired) and scope (run, session, global). Every change is a transition with a reason, an actor and the policy_version in force; client.lessons.transitions(lesson_id=...) lists them and the console's lesson drawer renders the same timeline.

Promotion between scopes is governed by the effective policy (starter@1 by default) and by outcomes credited to the lesson; it is not tied to reflect(). Copying lessons between environments is an explicit action: client.policy.promote(from_env="dev", to_env="prod") or the Policy tab in the console.

Deprecation warnings

Symptom: mubit: client.query() is deprecated and will be removed in mubit-sdk 1.0; use client.raw.invoke("control.query") — https://docs.mubit.ai/sdk/migration#query

Every 0.13 name keeps working in 0.14 and prints one such line per call site. The link names the replacement; the full list is in Migration. To fail CI on any use: python -W error::mubit.MubitDeprecationWarning, MUBIT_DEPRECATIONS=error (JS) or #![deny(deprecated)] (Rust). mubit doctor prints the number of distinct deprecation warnings emitted in the current process.

Integration packages

Framework adapter not connecting

All integration packages (mubit-crewai, mubit-langgraph, etc.) use the core SDK internally. Verify:

  1. MUBIT_API_KEY is set
  2. MUBIT_ENDPOINT points to a reachable Mubit instance
  3. The adapter package version matches your SDK version (see Versioning)

Getting help