Checks methods

Create client with checks(). A check id identifies the group. Each results[] entry has a task check id identifying one checker run. Examples reuse the SDK setup. Use IDs returned by the relevant list or create call. Run Python snippets inside the async function shown in the Python guide.

Check: check.id
└── Task check: check.results[].id
    ├── Result and evidence
    ├── Transcript and logs
    └── Sandbox files

Python returns check and task-check records as dictionaries. Use check["id"] and check["results"].

Work	Methods
Create and follow	create, get, list, defaults, watch
Read one checker	task, transcript, artifact — TypeScript only
Files, sharing, and removal	download, taskFilesystem, share, unshare, shares, delete

create

Check a local task directory, a directory of tasks, or a published dataset. Returns the accepted Check immediately.

Signature

create(input: CreateCheckInput): Promise<Check>;

async def create(
    directory: Optional[str] = None,
    *,
    dataset: Optional[str] = None,
    name: Optional[str] = None,
    agent: Optional[str] = None,
    model_name: Optional[str] = None,
    rubric: Optional[Rubric] = None,
    prompt: Optional[str] = None,
    reasoning_effort: Optional[str] = None,
    sandbox_provider: Optional[EvalSandboxProvider] = None,
    n_concurrent: Optional[int] = None,
    include_task_names: Optional[List[str]] = None,
    exclude_task_names: Optional[List[str]] = None,
    n_tasks: Optional[int] = None,
    on_upload_progress: Optional[Callable[[int, int], None]] = None,
) -> Check: ...

import { checks } from "@evolvingmachines/evolve";

const client = checks();
const check = await client.create({
  source: {
    directory: "./tasks"
  },
  agent: "codex",
  model_name: "gpt-6-luna",
  n_tasks: 1,
});

from evolve import checks

client = checks()
check = await client.create(
    './tasks',
    agent="codex",
    model_name="gpt-6-luna",
    n_tasks=1,
)

Source	Inputs
Local directory	TypeScript: `source.directory`. Python: `directory` or the first positional argument.
Published version	TypeScript: `source.dataset`. Python: `dataset`. Use `name@version`.

Supply exactly one source. A local source must be a directory, not an archive. include_task_names and exclude_task_names are glob lists; n_tasks caps the selected set after filtering. n_concurrent is bounded by organization capacity. Optional onUploadProgress(sent, total) / on_upload_progress(sent, total) reports local transfer bytes. All omitted policy settings use defaults. The accepted check records the resolved policy. Follow with watch(check.id) / watch(check["id"]).

Input fields

interface CheckConfigInput {
  name?: string;
  agent?: string;
  model_name?: string;
  rubric?: Rubric;
  prompt?: string;
  reasoning_effort?: string;
  sandbox_provider?: EvalSandboxProvider;
  n_concurrent?: number;
  include_task_names?: string[];
  exclude_task_names?: string[];
  n_tasks?: number;
}

interface CreateCheckInput extends CheckConfigInput {
  source: { directory: string } | { dataset: string };
  onUploadProgress?: (sentBytes: number, totalBytes: number) => void;
}

get

Read a check at any status. Returns Check, including one task-check result per selected task.

Signature

get(checkId: string): Promise<Check>;

async def get(check_id: str) -> Check: ...

const check = await client.get(checkId);

check = await client.get(check_id)

Use the group’s check id. See all check fields.

list

List visible checks, newest first. Returns a page or an iterable of Check.

Signature

list(options?: ListChecksOptions): CheckList;

def list(
    *,
    scope: Optional[JobListScope] = None,
    status: Optional[List[CheckStatus]] = None,
    dataset: Optional[str] = None,
    limit: Optional[int] = None,
    cursor: Optional[str] = None,
) -> _PaginatedList: ...

const page = await client.list({
  dataset: "my-tasks@1.0",
  status: ["completed"]
});

page = await client.list(
    dataset='my-tasks@1.0',
    status=['completed'],
)

Optional scope: my (default), shared, or org; status: a list of queued, running, or completed; dataset: a bare name or pinned name@version. limit defaults to 50 (maximum 200); cursor continues a page.

defaults

Read CheckDefaults: current agent, model, reasoning effort, sandbox provider, rubric, and unrendered prompt template. Pass agent to read its defaults when no model is specified.

Signature

defaults(options?: { agent?: string }): Promise<CheckDefaults>;

async def defaults(*, agent: Optional[str] = None) -> CheckDefaults: ...

const defaults = await client.defaults();
const codex = await client.defaults({ agent: "codex" });

defaults = await client.defaults()
codex = await client.defaults(agent="codex")

Python returns a dictionary. The check’s prompt uses {task_path}, {file_tree}, and {criteria_guidance}; the output contract is appended after your template. An agent the platform does not offer is refused with invalid_input.

watch

Wait until every task check has settled. Returns the final Check, which may contain failed task checks.

Signature

watch(
  checkId: string,
  options?: WatchCheckOptions
): Promise<Check>;

async def watch(
    check_id: str,
    *,
    on_progress: Optional[Callable[[Check], None]] = None,
    poll_interval_s: float = 2.0,
    timeout_s: Optional[float] = None,
) -> Check: ...

const check = await client.watch(checkId, {
  onProgress: c => console.log(c.status)
});

check = await client.watch(
    check_id,
    on_progress=lambda c: print(c['status']),
)

Optional onProgress(check) / on_progress(check) runs when per-task statuses change. pollIntervalMs / poll_interval_s defaults to 2 seconds, doubles while unchanged to 30 seconds, and resets on change. TypeScript accepts signal; Python accepts timeout_s.

task

TypeScript only. Read one checker’s TaskCheck result at any status.

Signature

task(taskCheckId: string): Promise<TaskCheck>;

const result = await client.task(taskCheckId);

Use check.results[].id, not the group’s id. Python reads the same result within check["results"]. The result contains checks, attempts, measured cost, and typed failure.

transcript

TypeScript only. Read one checker’s own activity. Returns TaskCheckTranscript.

Signature

transcript(
  taskCheckId: string,
  options?: AnalysisTranscriptOptions,
): Promise<TaskCheckTranscript>;

const transcript = await client.transcript(taskCheckId, {
  since: 0
});

Optional since skips that many events; default 0. No server pagination. total counts all stored events; gateway_calls is separate and returned whole. Transcript fields include the owning check_id and dataset.

artifact

TypeScript only. Read one checker’s stored stdout, stderr, or captured home.

Signature

artifact(
  taskCheckId: string,
  stream: Exclude<AnalysisArtifactStream, "agent-home">,
): Promise<string | null>;
artifact(
  taskCheckId: string,
  stream: "agent-home",
): Promise<Record<string, string> | null>;

const stderr = await client.artifact(
  taskCheckId,
  "trace-stderr"
);

Use the task check id. trace-stdout and trace-stderr return string | null; agent-home returns a path-to-text map or null. Null means not stored. Python can read stored evidence from download(task_check_id).

download

Download a settled whole check or one task check as a .tar.gz. Both id forms use the same method.

Signature

download(id: string): Promise<Buffer>;
download(
  id: string,
  options: { to: string }
): Promise<string>;
download(
  id: string,
  options: { stream: true }
): Promise<ReadableStream<Uint8Array>>;
download(
  id: string,
  options?: DownloadJobOptions
): Promise<Buffer | string | ReadableStream<Uint8Array>>;

async def download(
    id: str,
    *,
    to: Optional[str] = None,
) -> bytes | str: ...

const path = await client.download(checkId, {
  to: "./results"
});

path = await client.download(
    check_id,
    to='./results',
)

Omit options for Buffer / bytes; use to for a saved file path. TypeScript also accepts { stream: true }, a raw stream whose integrity the caller verifies. Python has no stream option. A live group or task check returns check_not_terminal. It returns the same code for a short time after they settle, while a worker is still finishing; retry in a few minutes. The group archive includes check_report.json and each checker’s wrapper trial.

taskFilesystem

Python: task_filesystem. Get one checker’s RunFilesystem. Requires both the group id and task check id.

Signature

taskFilesystem(
  checkId: string,
  taskCheckId: string
): RunFilesystem;

def task_filesystem(
    check_id: str,
    task_check_id: str,
) -> 'RunFilesystem': ...

const fs = client.taskFilesystem(checkId, taskCheckId);

fs = client.task_filesystem(
    check_id,
    task_check_id,
)

See filesystem methods for files, sandbox logs, and processes. Grant read access to a check you created. Returns JobShares, the same share-state shape used by jobs. A check can have up to 50 email grants.

Signature

share(
  id: string,
  request: JobShareRequest
): Promise<JobShares>;

async def share(
    id: str,
    *,
    link: bool = False,
    emails: Optional[List[str]] = None,
) -> JobShares: ...

const shares = await client.share(checkId, {
  link: true
});

shares = await client.share(
    check_id,
    link=True,
)

Supply link: true / link=True, emails: string[] / emails=[...], or both. New email grants send invitations. Recipients can read the check, its task checks, and downloads. They cannot operate it. Share-state fields describe the result.

unshare

Revoke a check’s link or email grants. Returns JobShares. Creator-only and idempotent.

Signature

unshare(
  id: string,
  request: JobShareRequest
): Promise<JobShares>;

async def unshare(
    id: str,
    *,
    link: bool = False,
    emails: Optional[List[str]] = None,
) -> JobShares: ...

const shares = await client.unshare(checkId, {
  link: true
});

shares = await client.unshare(
    check_id,
    link=True,
)

Takes the same link and emails fields as share.

Read the check’s whole share state: visibility, link, and email grants. Creator-only.

Signature

shares(id: string): Promise<JobShares>;

async def shares(id: str) -> JobShares: ...

const shares = await client.shares(checkId);

shares = await client.shares(check_id)

delete

Permanently remove a settled check you created: its task checks with their results, transcripts, logs and captured files, any tasks uploaded for it, and its shares. Returns CheckDeleteResult.

Signature

delete(id: string): Promise<CheckDeleteResult>;

async def delete(id: str) -> CheckDeleteResult: ...

const receipt = await client.delete(checkId);

receipt = await client.delete(check_id)

The receipt has check_id and task_checks_deleted; in Python it is an object, not a dictionary. Creator-only: anyone else who can read the check is refused with org_forbidden. A check with a task check still queued or running returns check_not_terminal; wait until every task check settles. It returns the same code for a short time after they settle, while a worker is still finishing; retry in a few minutes.

HTTP API

Python and TypeScript

Run agents

Shared behavior

SDK workflows

SDK resources

SDK methods

Checks methods

create

get

list

defaults

watch

task

transcript

artifact

download

taskFilesystem

unshare

shares

delete