Skip to main content
A trial is one attempt of one task by one agent arm. Trial ids are global: every trial command takes the trial id alone, and the trial’s record carries its job id. Treat ids as opaque strings.
trial show prints the task, the agent and model, the status, the reward, the tokens, what the trial spent and, when it failed, the exception. Spend prints as a plain amount once the meter has settled it, as at least $X while it is still a lower bound, and as - when nobody has measured it yet. While a trial runs, attempt_phase says which step it is in (prepare, build, boot, install, agent, verify, persist), and live_spent_usd is a lower bound on its spend so far, refreshed about every 30 seconds and null before the first reading; usage carries the same reading with its tokens. On a settled trial read agent_result.cost_usd; spend_source says whether it is final (measured).

​
The trace

Every trial records a parsed event trace: the instruction, each agent turn, every tool call and its result. Print it, filtered on the server.
--type keeps events of exactly one type. --grep is a case-insensitive regex over type and content. --tail keeps the last N matching events. To search every trial of a job in one pass:

​
Artifacts

Beside the parsed trace, a trial keeps the raw record of its run. One vocabulary names the pieces everywhere; every file and format is described on trial outputs.
  • trace-parsed: the parsed event trace
  • trace-stdout: the agent process’s stdout, byte for byte
  • trace-stderr: the agent process’s stderr
  • trace-atif: the normalized trajectory, in the ATIF format
  • agent-home: the text view of the agent’s home folder, captured after the run
  • verifier: everything the scoring step printed
  • trajectory: reserved for the harness’s own native session file, not served yet
Stream one artifact to stdout:

​
Download the trial tree

Without --stream, the trial is saved whole under <dir>/<trial-id>/ in the trial tree layout, plus evolve.json.
An artifact the trial never recorded is an absent file, never an empty placeholder. --overwrite replaces an existing folder. The job archive from evolve job download holds the same tree per trial, plus what only the server can add: lock.json, trial.log, artifacts/, and the raw verifier/reward.txt.

​
Act on one trial

retry runs a settled trial again, regrade re-runs only its verifier, and each produces a new job holding the one new trial. stop ends in-flight trials without cancelling their job; each stopped trial settles CANCELLED and is charged its spend so far.

​
Statuses

A valid reward, 0 included, is SCORED; a failure is never reported as a fabricated zero.
  • QUEUED: waiting for a sandbox slot.
  • RUNNING: the agent phase; attempt_phase says which step.
  • SCORING: the agent finished; the verifier is running.
  • SCORED: a valid reward is in reward.
  • SCORING_ERROR: the verifier crashed or wrote a reward outside [0, 1]; read exception_info.
  • INFRASTRUCTURE_ERROR: the trial was lost before a result was recorded; retried automatically, then resume.
  • BUDGET: a budget above the trial’s own cap refused it. The exception type is ApiUsageLimitError and the message carries user: (your credits), team: (the organization’s monthly budget) or other: (the platform’s global stop). Never retried on its own; add credits or raise the budget, then resume.
  • INDETERMINATE: the platform cannot tell whether the trial completed.
  • CANCELLED: stopped before settling; the exception type is CancelledError.
SCORING_ERROR and INDETERMINATE carry the last 4000 characters of the verifier’s own output in exception_message. Job statuses are on jobs.

trial reference

Every flag of evolve trial.