Skip to main content
This page takes you from an empty terminal to a downloaded trial: one API key, any harness, any model, any sandbox provider, and every trial’s trace streamed live. A trial is one agent’s attempt at one task. A job is a set of trials: the tasks of one or more datasets, run by one or more agents.
1

Install the CLI and set your key

Create a key on the dashboard’s API keys page, then export it. See installation for the SDKs.
2

Give your coding agent the manual

Your agent then reads these docs with evolve skills get evals, and the create-task, rewardkit, create-adapter and publish skills by name. The five skills.
3

Pick a dataset

The catalog lists every dataset you can run, with its versions.
dataset show prints the version’s tasks and, for each task, which sandbox providers can run it.
4

Start a job and watch it

Name a dataset, an agent, and a model. --watch streams the job’s events until it finishes; -l 5 caps the run at five tasks and --max-trial-spend 1 caps what each trial may spend on the model, so the first job is small.
The final block prints the job id, how many trials landed in each status, and what the trials spent. Without --watch the command returns the id at once and the job keeps running.
5

Read the result

job show is the whole job: how many trials landed in each status, the rewards, the spend. job trials lists the trials one per line with their ids. trial show is one trial in full, with its reward and, when it failed, why.
6

Download the trial

A trial downloads as a trial tree: the trajectory, the agent’s stdout and stderr, the verifier log, and the reward.
To read one artifact without saving anything, stream it instead.

​
Next, by what you want to do

Port a benchmark

evolve skills get create-adapter turns it into Harbor-format tasks.

Write a task

evolve skills get create-task, and rewardkit for its verifier.

Publish a dataset

evolve skills get publish, then evolve dataset publish.

Upload a job you ran elsewhere

evolve upload <job dir> brings a Harbor job into the dashboard.