> ## Documentation Index
> Fetch the complete documentation index at: https://docs.evolvingmachines.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# jobs
> Start, follow, and derive jobs.
`jobs()` returns the jobs client. Every job method takes a job id, and every derived run returns a new `Job`.
## start
```ts theme={null}
start(
input: JobCreate,
options?: StartJobOptions,
): Promise
```
`JobCreate`: `datasets` (selectors: `name`, optional `version`, `task_names`, `exclude_task_names`, `n_tasks`), `agents` (arms: `name`, `model_name`, optional `version`, `reasoning_effort`, `kwargs`, `preset`, `skills`), and optional `job_name`, `n_attempts`, `n_concurrent_trials` (1 to 150), `max_trial_spend_usd`, `sandbox_provider`, `retry`, `analyze`, the five `*_timeout_multiplier` fields, `verifier_env` (`REWARDKIT_JUDGE` and `REWARDKIT_MODEL` only) and `secrets`. `agent_env` is in the shape but refused by the server.
The response is the [Job](/sdk-reference/types#job), with the resolved `max_trial_spend_usd`, `worst_case_spend_usd`, `retry`, `analyze` and multipliers. `options.idempotencyKey` makes a repeated call return the same job, with `idempotent_replay` true; a different request under a used key is refused with `idempotency_key_reused`.
```ts theme={null}
const job = await jobs().start({
datasets: [{ name: "terminal-bench-4", version: "4.0", n_tasks: 5 }],
agents: [{ name: "codex", model_name: "gpt-6-astra" }],
});
```
```python theme={null}
async def start(
*,
datasets: List[DatasetSelector | dict],
agents: List[AgentArm | dict],
job_name=None,
n_attempts=None,
n_concurrent_trials=None,
max_trial_spend_usd=None,
sandbox_provider=None,
retry=None,
analyze=None,
timeout_multiplier=None,
agent_timeout_multiplier=None,
verifier_timeout_multiplier=None,
agent_setup_timeout_multiplier=None,
environment_build_timeout_multiplier=None,
agent_env=None,
verifier_env=None,
secrets=None,
idempotency_key=None,
) -> Job
```
The same fields as TypeScript, as keyword arguments; `datasets` and `agents` take dicts or the `DatasetSelector` and `AgentArm` dataclasses, and `retry` is a `JobRetryConfigInput`. `idempotency_key` makes a repeated call return the same job, with `idempotent_replay` true; a different request under a used key is refused with `idempotency_key_reused`.
```python theme={null}
job = await jobs().start(
datasets=[{"name": "terminal-bench-4", "version": "4.0", "n_tasks": 5}],
agents=[{"name": "codex", "model_name": "gpt-6-astra"}],
)
```
## get
```ts theme={null}
get(id: string): Promise
```
One job in full.
```ts theme={null}
const job = await jobs().get("3e1f9a2c-…");
```
```python theme={null}
async def get(id: str) -> Job
```
One job in full.
```python theme={null}
job = await jobs().get("3e1f9a2c-…")
```
## list
```ts theme={null}
list(options?: { search?: string; scope?: "my" | "shared"; limit?: number; cursor?: string }): JobList
```
Your jobs, newest first. Await one page, or iterate every page.
```ts theme={null}
for await (const job of jobs().list({ search: "nightly" })) {
console.log(job.id, job.status);
}
```
```python theme={null}
def list(
*,
search: Optional[str] = None,
scope: Optional[JobListScope] = None,
limit: Optional[int] = None,
cursor: Optional[str] = None,
)
```
Your jobs, newest first. Await one page, or iterate every page.
```python theme={null}
async for job in jobs().list(search="nightly"): print(job.id, job.status)
```
## trials
```ts theme={null}
trials(
id: string,
options?: { status?: TrialStatus[]; dataset?: string; limit?: number; cursor?: string },
): TrialList
```
A job's trials. `status` and `dataset` filter.
```ts theme={null}
const failed = await jobs().trials(
job.id,
{ status: ["INFRASTRUCTURE_ERROR", "SCORING_ERROR"] },
);
```
```python theme={null}
def trials(
id: str,
*,
status: Optional[List[str]] = None,
dataset: Optional[str] = None,
limit: Optional[int] = None,
cursor: Optional[str] = None,
)
```
A job's trials. `status` and `dataset` filter.
```python theme={null}
failed = await jobs().trials(
job.id,
status=["INFRASTRUCTURE_ERROR", "SCORING_ERROR"],
)
```
## tasks
```ts theme={null}
tasks(
id: string,
options?: { limit?: number; cursor?: string },
): JobTaskRollupList
```
One row per task: its trial tally, mean reward and cost, the dataset it came from (`source`), and its latest quality check (`check`, null when never checked).
```ts theme={null}
for await (const row of jobs().tasks(job.id)) {
console.log(row.task_name, row.mean_reward);
}
```
```python theme={null}
def tasks(
id: str,
*,
limit: Optional[int] = None,
cursor: Optional[str] = None,
)
```
One row per task: its trial tally, mean reward and cost, the dataset it came from (`source`), and its latest quality check (`check`, None when never checked).
```python theme={null}
async for row in jobs().tasks(job.id): print(row.task_name, row.mean_reward)
```
## watch
```ts theme={null}
watch(id: string, options?: { onEvent?: (event: JobEvent) => void; signal?: AbortSignal; reconnectDelayMs?: number; maxReconnectDelayMs?: number }): JobWatch
```
The job's event stream. Await the handle for the final job, or iterate it for each event; one form per call, and `onEvent` fires in both. The stream replays from the beginning and reconnects on its own, backing off from `reconnectDelayMs` (default 1000) to `maxReconnectDelayMs` (default 30000). Every event type and payload is on [types](/sdk-reference/types#events); a `trial.settled` is not final while a `trial.retrying` can follow it.
```ts theme={null}
const final = await jobs().watch(job.id);
```
```python theme={null}
def watch(
id: str,
*,
on_event: Optional[Callable[[JobEvent], None]] = None,
timeout_s: Optional[float] = None,
reconnect_delay_s: float = 1.0,
max_reconnect_delay_s: float = MAX_WATCH_DELAY_SEC,
)
```
The job's event stream. Await the handle for the final job, or iterate it for each event; one form per call. The stream replays from the beginning and reconnects on its own, backing off from `reconnect_delay_s` to `max_reconnect_delay_s` (30 s). Every event type and payload is on [types](/sdk-reference/types#events); a `trial.settled` is not final while a `trial.retrying` can follow it.
```python theme={null}
final = await jobs().watch(job.id)
```
## cancel
```ts theme={null}
cancel(id: string): Promise
```
Request cancellation. A finished job is a no-op.
```ts theme={null}
await jobs().cancel(job.id);
```
```python theme={null}
async def cancel(id: str) -> Job
```
Request cancellation. A finished job is a no-op.
```python theme={null}
await jobs().cancel(job.id)
```
## resume
```ts theme={null}
resume(
id: string,
request?: { filter_error_types?: string[] },
options?: StartJobOptions,
): Promise
```
A new job over a finished job's failed or stopped trials; `source_jobs` records `action: "resume"`. The default set and the refusals are on [jobs](/core-concepts/jobs#derive-a-new-job). Supports `idempotencyKey`.
```ts theme={null}
const next = await jobs().resume(
job.id,
{ filter_error_types: ["InfrastructureError"] },
);
```
```python theme={null}
async def resume(
id: str,
*,
filter_error_types: Optional[List[str]] = None,
idempotency_key: Optional[str] = None,
) -> Job
```
A new job over a finished job's failed or stopped trials.
```python theme={null}
next_job = await jobs().resume(
job.id,
filter_error_types=["InfrastructureError"],
)
```
## retry
```ts theme={null}
retry(
id: string,
request?: { trial_ids?: string[]; failed_only?: boolean },
options?: StartJobOptions,
): Promise
```
A new job re-running selected trials: named ids, the failed ones, or all of them; `trial_ids` and `failed_only` together is refused. The named form works on a running job once every named trial has settled; the other two need a finished source. The rules are on [jobs](/core-concepts/jobs#derive-a-new-job).
```ts theme={null}
const again = await jobs().retry(
job.id,
{ failed_only: true },
);
```
```python theme={null}
async def retry(
id: str,
*,
trial_ids: Optional[List[str]] = None,
failed_only: Optional[bool] = None,
idempotency_key: Optional[str] = None,
) -> Job
```
A new job re-running selected trials: named ids, the failed ones, or all of them.
```python theme={null}
again = await jobs().retry(
job.id,
failed_only=True,
)
```
## regrade
```ts theme={null}
regrade(
id: string,
request?: { statuses?: TrialStatus[]; task_name?: string },
): Promise
```
Verifier-only re-run of a finished job, in fresh separate verifier boxes under the recorded network policy. The result is a job with `is_regrade` true. Only settled separate-mode trials are eligible; `no_regradable_trials` when none are. See [jobs](/core-concepts/jobs#derive-a-new-job).
```ts theme={null}
const regraded = await jobs().regrade(
job.id,
{ task_name: "tricky-task" },
);
```
```python theme={null}
async def regrade(
id: str,
*,
statuses: Optional[List[str]] = None,
task_name: Optional[str] = None,
) -> Job
```
Verifier-only re-run of a finished job. The result is a job.
```python theme={null}
regraded = await jobs().regrade(
job.id,
task_name="tricky-task",
)
```
## analyze
```ts theme={null}
analyze(
id: string,
request?: AnalyzeConfigInput,
): Promise
```
Enqueue one trace analysis per trial and return at once with the job, its `stats.analysis.n_pending` counting the batch. `AnalyzeConfigInput`: `model_name`, `rubric`, `prompt`, `sandbox_provider`, `reasoning_effort`, `n_concurrent`, `passing`, `failing`, `n_trials`, all optional; `{}` is every default. Calling it again is the re-analysis path, one wave at a time. The same object on `start()` as `analyze` arms the embedded trigger.
```ts theme={null}
await jobs().analyze(
job.id,
{
failing: true,
n_trials: 20,
},
);
```
```python theme={null}
async def analyze(
id: str,
*,
model_name=None,
rubric: Optional[Rubric] = None,
prompt=None,
sandbox_provider=None,
reasoning_effort=None,
n_concurrent=None,
passing=None,
failing=None,
n_trials=None,
) -> Job
```
Enqueue one trace analysis per trial and return at once.
```python theme={null}
await jobs().analyze(
job.id,
failing=True,
n_trials=20,
)
```
## watchAnalysis
```ts theme={null}
watchAnalysis(id: string, options?: { onStats?: (job: Job) => void; signal?: AbortSignal; pollIntervalMs?: number }): Promise
```
Poll until no analysis is pending, from `pollIntervalMs` (default 2000) doubling to 30 s while the tally stands still, `onStats` firing on every change. Resolves with the final job. A job that was never analyzed polls forever, so call it after `analyze()`. On a still-running job created with `analyze`, `n_pending` can touch 0 between trial settles, so the watch can return early.
```ts theme={null}
const analyzed = await jobs().watchAnalysis(job.id);
```
```python theme={null}
async def watch_analysis(
id: str,
*,
on_stats: Optional[Callable[[Job], None]] = None,
poll_interval_s: float = 2.0,
timeout_s: Optional[float] = None,
) -> Job
```
Poll until no analysis is pending. Resolves with the final job.
```python theme={null}
analyzed = await jobs().watch_analysis(job.id)
```
## compare
```ts theme={null}
compare(ids: string[]): Promise
```
Two to ten jobs side by side: per-job aggregates and a per-task matrix, disagreement rows first. Means cover SCORED trials only, with `coverage` beside them. The shape is on [types](/sdk-reference/types#smaller-shapes).
```ts theme={null}
const cmp = await jobs().compare([jobA.id, jobB.id]);
```
```python theme={null}
async def compare(ids: List[str]) -> CompareResponse
```
Two to ten jobs side by side: per-job aggregates and a per-task matrix.
```python theme={null}
cmp = await jobs().compare([job_a.id, job_b.id])
```
## download
```ts theme={null}
download(id: string): Promise
download(
id: string,
options: { to: string },
): Promise
download(
id: string,
options: { stream: true },
): Promise>
```
A finished job's results as one `.tar.gz` in the standard job layout: the bytes, the saved file's path, or a stream. The first two shapes are verified against the response's `Content-Length` and, when the server states one, its digest; `{ stream: true }` hands you the raw bytes to verify yourself.
```ts theme={null}
const path = await jobs().download(
job.id,
{ to: "./results" },
);
```
```python theme={null}
async def download(
id: str,
*,
to: Optional[str] = None,
)
```
A finished job's results as one `.tar.gz` in the standard job layout: the bytes, or with `to` the saved file's path.
```python theme={null}
path = await jobs().download(
job.id,
to="./results",
)
```
## upload
```ts theme={null}
upload(source: string | { archive_url: string }, options?: { dataset?: string; onUploadProgress?: (sentBytes: number, totalBytes: number) => void; onRegistered?: (importId: string) => void }): Promise
```
Upload a job directory, its `.tar.gz`, or a public archive URL as a finished job. Returns the import record; follow it with `watchImport`. A large upload resumes after a dropped connection, and `onRegistered` hands you the import id as soon as it starts. A directory without `result.json` and `config.json` at its root is refused client-side.
```ts theme={null}
const imp = await jobs().upload(
"./job-2026-08-27__12-00-00",
{ dataset: "terminal-bench-4@4.0" },
);
```
```python theme={null}
async def upload(
dir_or_archive: Optional[str] = None,
*,
archive_url: Optional[str] = None,
dataset: Optional[str] = None,
on_upload_progress=None,
on_registered=None,
) -> JobImport
```
Upload a job directory, its `.tar.gz`, or a public archive URL as a finished job. Returns the import record.
```python theme={null}
imp = await jobs().upload(
"./job-2026-08-27__12-00-00",
dataset="terminal-bench-4@4.0",
)
```
## getImport, watchImport, listImports
```ts theme={null}
getImport(id: string): Promise
watchImport(id: string, options?: { onStatus?: (jobImport: JobImport) => void; onProgress?: (progress: JobImportProgress, jobImport: JobImport) => void; pollIntervalMs?: number; signal?: AbortSignal }): Promise
listImports(options?: { status?: DatasetImportStatus; limit?: number; cursor?: string }): JobImportList
```
Read one upload, follow it until it settles, or list your uploads.
```ts theme={null}
const done = await jobs().watchImport(imp.id);
```
```python theme={null}
async def get_import(id: str) -> JobImport
async def watch_import(
id: str,
*,
on_status=None,
on_progress=None,
poll_interval_s: float = 2.0,
timeout_s: Optional[float] = None,
) -> JobImport
def list_imports(
*,
status: Optional[str] = None,
limit: Optional[int] = None,
cursor: Optional[str] = None,
)
```
Read one upload, follow it until it settles, or list your uploads.
```python theme={null}
done = await jobs().watch_import(imp.id)
```
## delete
```ts theme={null}
delete(id: string): Promise
```
Permanently delete a job you created: its trials, traces, analyses and stored files. The receipt is `{ job_id, trials_deleted, analyses_deleted }`. The refusals are on [jobs](/core-concepts/jobs#stop-cancel-delete).
```ts theme={null}
await jobs().delete(job.id);
```
```python theme={null}
async def delete(id: str) -> JobDeleteResult
```
Permanently delete a job you created: its trials, traces, analyses and stored files.
```python theme={null}
await jobs().delete(job.id)
```
## grep
Search every trial's parsed trace in one pass. `q` is a case-insensitive regex over each event's type and content; a plain string is a plain substring. Items are per-trial groups with the exact `match_count` and the first five matching events, ordered by trial id; `limit` defaults to 50, at most 200. Keep paging while `hasMore` (`has_more` in Python) is true; a pattern too expensive to evaluate is refused with `invalid_input`.
```ts theme={null}
grep(
id: string,
q: string,
options?: { type?: string; limit?: number; cursor?: string },
): Promise
```
```ts theme={null}
const hits = await jobs().grep(job.id, "permission denied");
```
```python theme={null}
async def grep(
id: str,
q: str,
*,
type: Optional[str] = None,
cursor: Optional[str] = None,
limit: Optional[int] = None,
) -> JobGrepPage
```
```python theme={null}
hits = await jobs().grep(job.id, "permission denied")
```
## passAtK
```ts theme={null}
import { passAtK } from "@evolvingmachines/evolve";
function passAtK(job: Job): { evals_key: string; points: { k: number; value: number }[] }[]
```
Reads `stats.evals[key].pass_at_k` off a job you already hold, as sorted numbers; no request is made. Groups that cannot answer are left out, so an empty array means the job has no pass\@k to show.
```ts theme={null}
for (const group of passAtK(job)) {
for (const point of group.points) console.log(group.evals_key, `pass@${point.k}`, point.value.toFixed(3));
}
```
```python theme={null}
from evolve import pass_at_k
def pass_at_k(job: Job) -> List[PassAtKGroup]
```
Reads `stats["evals"][key]["pass_at_k"]` off a job you already hold, as sorted `PassAtKGroup(evals_key, points)` with `PassAtKPoint(k, value)`; no request is made. Groups that cannot answer are left out, so an empty list means the job has no pass\@k to show.
```python theme={null}
for group in pass_at_k(job):
for point in group.points:
print(group.evals_key, f"pass@{point.k}", f"{point.value:.3f}")
```