Skip to main content
Evolve is the engine for running reproducible agent evaluations at scale across tasks, sandboxes, harnesses, and models. It provides quality control during task creation, with full observability and trajectory analysis for every run. An evaluation has three parts: tasks to solve, an agent (a harness and a model) that attempts them, and a verifier that scores each attempt. To check agent traces at scale, use Edith, our decision model.

Task quality checks

Check that tasks are solvable, instructions are clear, and verifiers score correctly.

Agent optimization

Compare harnesses, models, and settings on the same tasks. Measure rewards and costs.

Trajectory analysis

Review the agent’s actions, failures, and the evidence behind its score.

Full observability

Inspect traces, files, verifier logs, timing, token usage, and costs.

Run one task

After installation and authentication, run one DeepSWE task:
The job runs in the cloud. Closing the terminal does not cancel it. The cap applies to one trial’s model spend; an in-flight call can exceed it.

First evaluation

Run one task, then read its score and trace.

Your own tasks

Define the instruction, environment, and verifier.

Python and TypeScript

Run evaluations from your application.

Use your coding agent

Install the skill and read the CLI manual.

Read the result

An evaluation produces a trial for each attempt. Read its reward, follow the agent’s trace, and open the saved files. Compare jobs to see how a different model or harness changes the result. Use task checks to review a task before running it. Use Edith to ask structured questions about a completed run, or Analyze for a written review with evidence.
Last modified on November 9, 2026