Train adapters

Tasks and ratings

An adapter learns from the calls of one task that are rated good. This page says how calls form a task, how they are rated, and how you rate a call yourself. It is for developers who want to see, and steer, what an adapter learns from.

Tasks

Calls form one task when they share their instructions, tools, tool choice, output format and model. The messages after the instructions and settings such as temperature don't count. Tasks and sessions gives the full rule, and how to group a conversation's calls into a session.

Tasks in the console lists your project's tasks, each under a short name made from the first words of its instructions. Open a task to see its status and its count of kept examples, such as 12 of 25 examples.

Kept examples

A kept example is a call of the task that is rated good. The adapter trains on kept examples, and its exam uses kept examples it never trained on.

A call can be a kept example only when Mubit recorded it in full. A call sent with x-middleware-record: metadata or off, or with x-middleware-store: false, is not rated and trains nothing. A call made in Try models is not stored and trains nothing either.

How calls are rated

Mubit rates each task's new calls automatically, a few minutes after they arrive. An automatic rater reads the call's request and reply, and marks the reply good or bad.

Mubit rates up to 200 calls of each task a day. The task's other calls that day stay unrated.
A rating of good makes the call a kept example. A rating of bad keeps it out of training.
A call can be rated more than once. The latest rating counts, whoever gave it, so your rating replaces Mubit's.

The rater runs on Mubit's own account, not on your provider key. Costs and data says where call content goes.

Rate a call yourself

Rate the calls you know are right or wrong: your ratings decide what the adapter learns.

In the console

Open Requests, select a call, and under Rate this reply select Good or Bad. The Rating column shows each call's rating. See Find calls in Requests.

From code

Rate a call with a management key whose role is member or admin. The call's ID comes from the list of calls (calls.list, filtered by task).

Python

from mubit import Mubit
 
client = Mubit()  # reads MUBIT_MANAGEMENT_KEY from the environment
 
calls = client.calls.list(task="<task id>", limit=20)
client.calls.feedback(calls.data[0]["id"], rating="down", comment="Missed the sales tax line.")

Field	Value
`rating`	`up` (good) or `down` (bad). Required.
`comment`	Why, in up to 500 characters. Optional.
`correction`	The reply that would have been right. Optional.

A call outside your key's organization, or outside its project for a key limited to one project, gets 404.

ℹ️Note

A management key starts with mwm_. An admin creates one in the console, under Workspace, Management keys. A Mubit key (mbt_) can't call the management API.