Skip to main content
The Jobs page lists your jobs, newest first.
The Jobs page: a NEW JOB button, a search box, a status filter, and a table of jobs with status, datasets, agents, trials, cost and creation time.

The jobs list

Search by name, and filter by status: ALL, QUEUED, RUNNING, CANCELLING, COMPLETED, CANCELLED or FAILED. Each row shows the job’s datasets, its agents with their models, how many trials have settled out of the total, what it has spent, and when it started. Tick rows to cancel live jobs or delete finished ones.

​
New job

NEW JOB opens the form. It takes the same choices as evolve run.
The New job form: a Sources list of datasets to tick, an Agents section with agent, model and reasoning effort, an ADD AGENT button, and a Summary rail with LAUNCH and VALIDATE buttons.

The New job form

  • Sources: tick one or more datasets, cap the task count, and narrow the tasks with include and exclude patterns over their names.
  • Agents: one arm per entry, with its agent, its model (required) and its reasoning effort. Arm options hold the version pin, a preset, a config file and skills. ADD AGENT adds another arm.
  • Secrets: attach a secret from your vault, or save and attach a one-off value.
  • Job settings: job name, attempts per task, parallel trials, auto-retries, and the spend cap per trial.
  • Advanced, collapsed by default: the sandbox provider, the timeout multipliers, the judge overrides for the verifier, and the exceptions to retry on.
The Summary rail counts datasets × agents × attempts. VALIDATE checks the whole form without starting anything. LAUNCH starts the job and opens its page.

​
One job

A job's page: the job id and status in the header, CANCEL, RESUME, DOWNLOAD and RE-RUN JOB buttons, tiles for trials, errors, average reward, cost and max cost, and the OVERVIEW tab with a progress bar and a per-agent table.

A job's page, on its OVERVIEW tab

The header carries the job id and its status. CANCEL stops a running job. RESUME runs a finished job’s failed or stopped trials again as a new job. DOWNLOAD saves a finished job’s results as a .tar.gz in the standard job layout; while the job runs it is disabled, with the reason on hover. RE-RUN JOB opens the New job form filled in with this job’s choices. Five tiles sum the job up: trials settled out of the total, errors, average reward, cost so far, and the most the job can cost. The average reward shows a value for a job with one agent and one dataset; with more, it shows a dash and the OVERVIEW rows carry the means. Below them, five tabs.
  • OVERVIEW: a progress bar by trial status, and one row per agent and dataset with its scored trials, errors, mean reward and pass@k.
  • TRIALS: every trial, with ALL or ERRORS, its task, agent, status, reward, cost, duration and token counts. Click a row to open the trial viewer.
  • ANALYZE: the trace analyses of the job’s trials, below.
  • CHECK: the task quality checks of the job’s datasets. See checks.
  • CONFIG: the stored job record as JSON: its id, status, counts, stats and resolved settings.
The TRIALS tab: ALL and ERRORS filters and a table of trials with task, agent, status, reward, cost, duration, input tokens and output tokens.

The TRIALS tab

​
The ANALYZE tab

The ANALYZE tab: tiles for completed, failed and pending analyses and the analyzer's cost, the policy with model, effort and criteria, and a table of pass, fail, not applicable and unknown counts per criterion.

The ANALYZE tab after every trial was analyzed

The tab counts completed, failed and pending analyses and the analyzer’s own spend. The policy block names the analyzer’s model, its reasoning effort and the rubric’s criteria. The first table counts, per criterion, how many analyses ruled pass, fail, not applicable or unknown. Below it, one row per trial carries its label, its verdicts, the analyzer’s spend and its summary; click a row to open the analysis in the trial viewer. Start analyses with evolve analyze <job-id> or with --analyze on evolve run, see analyze.