Datasets HTTP API

Use the Evolve dataset API to find tasks or publish your own task corpus. A publish returns an import record; wait for its result before starting jobs against the dataset. Examples reuse connection setup:

curl -sS "$API/api/datasets/deepswe?version=1.1" \
  -H "Authorization: Bearer $EVOLVE_API_KEY"

Success returns 200 Dataset, including versions and a page of the selected version’s tasks. See dataset methods for the complete response fields.

Catalog and version routes

Method and path	Input	Success
`GET /api/datasets`	Optional catalog filters below	`200 DatasetPage`
`GET /api/datasets/{name}`	Optional `version`, `limit`, `cursor`, `sortField`, `sortDirection`	`200 Dataset`
`GET /api/datasets/{name}/download`	Optional `version`	`200`, `.tar.gz` archive
`PATCH /api/datasets/{name}`	Required JSON `upstream_auto_import` boolean	`200 Dataset`
`POST /api/datasets/{name}/versions/{version}/activate`	Owned dataset name and version	`200 Dataset`
`DELETE /api/datasets/{name}`	Owned dataset name	`204`, no body

Catalog filters are search, visibility (public or private), org (slug/id), limit, cursor, sortField (name, updated, tasks), and sortDirection (asc, desc). Task-list sorting accepts only sortField=name. Dataset detail pages default to 200 tasks, maximum 500; catalog pages use 50/200 pagination. Omitting version selects the active version. Activation selects another READY version as the default for bare-name requests. A successful publish of an owned dataset normally activates its READY version without a second request. Enabling upstream_auto_import requires an active version with a watchable moving git reference. An uploaded archive, commit-only source, or missing active version returns 409 upstream_not_watchable; disabling it is allowed.

Preflight configuration

POST /api/datasets/preflight checks task configuration before publishing. Given a task directory named my-task, create a request from its actual configuration:

python3 - <<'PY' > preflight.json
import json
from pathlib import Path

task = Path("my-task")
print(json.dumps({
    "tasks": [{
        "name": task.name,
        "task_toml": (task / "task.toml").read_text(),
    }]
}))
PY

tasks must be nonempty. Each entry requires a unique name of 1–256 characters and task_toml text; optional dataset_toml is the corpus manifest text. Preflight requires Python 3 only for this request-building example; the endpoint accepts JSON from any language.

curl -sS "$API/api/datasets/preflight" \
  -H "Authorization: Bearer $EVOLVE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data-binary @preflight.json

Success returns 200 DatasetPreflight with per-task verdicts and an optional manifest verdict. Preflight does not build task images. See preflight methods for the returned checks.

Publish a corpus

POST /api/datasets/publish accepts multipart data. For a prepared tasks.tar.gz archive:

curl -sS "$API/api/datasets/publish" \
  -H "Authorization: Bearer $EVOLVE_API_KEY" \
  -F 'name=my-dataset' \
  -F 'version=1.0' \
  -F 'archive=@tasks.tar.gz'

Success returns 202 DatasetImport. Store its id as IMPORT_ID and read the import result.

Multipart field	Rules
`name`, `version`	Required for git and `archive_url` sources; uploaded archives can supply them through `dataset.toml`, and hub packages default them from the package name and resolved revision; explicit fields win
`org`	Optional owning organization slug/id; requires membership; cannot move an existing dataset to another organization
`archive`	Gzipped corpus archive
`archive_url`	Public HTTPS URL of a gzipped corpus archive; URL credentials and private hosts are refused
`git_url`	HTTPS repository URL; paired with `git_ref`
`git_ref`	Full 40-character commit hash or tag; branch names are refused
`git_path`	Optional relative repository folder, valid only with a git source
`hub_package`	Public Harbor package reference: `org/name[@ref]`

Choose exactly one source: archive, archive_url, git_url with git_ref, or hub_package. Tags and hub references are resolved to fixed content when accepted. git_path must be an unambiguous relative POSIX path, without parent/current-directory segments, whitespace, backslashes, or glob characters. Publishing an owned name@version again restarts that version’s import in place. Use a new version label when you need to preserve the older version’s contents. A manifest controls task selection and verifies pinned content digests. A custom metric.py is unsupported. See publish inputs for source-specific rules and complete import fields.

Follow an import

Method and path	Input	Success
`GET /api/datasets/imports/{importId}`	Import id	`200 DatasetImport`
`GET /api/datasets/imports`	Optional `status`, `dataset`, `limit`, `cursor`	`200 DatasetImportPage`

curl -sS "$API/api/datasets/imports/$IMPORT_ID" \
  -H "Authorization: Bearer $EVOLVE_API_KEY"

Statuses are QUEUED, RUNNING, COMPLETED, or FAILED. Completion means the version is READY and at least one task built. Some tasks can still have failed: inspect import warnings and the dataset version’s n_failed_tasks before assuming every task is runnable. A whole-dataset job selects READY tasks and reports build exclusions. Naming a failed task explicitly returns task_failed_to_build. Read a task’s build result using:

GET /api/datasets/{name}/versions/{version}/tasks/{task_name}/build

Success returns 200 TaskBuild with the task’s state and failure details when present. Failed tasks do not silently become zero-reward trials.

Read task packages

Use the prefix /api/datasets/{name}/versions/{version}/tasks/{task_name}. The version is required.

Method and suffix	Input	Success
`GET /filesystem`	None	`200 TaskPackageFilesystemStatus`
`GET /filesystem/files`	Optional `path`, `cursor`, `limit`	`200 FilesystemListing`
`GET /filesystem/files/{path}`	File path; optional `Range` header	`200` or `206` bytes

List path defaults to /. Lists default to 500 entries, maximum 1000, with next_cursor. These are the retained task files, with no live source, logs, processes, or watch operation. See task package methods for byte ranges and returned fields.

Resumable uploads

Use this protocol for large dataset archives. The SDK automatically selects it above 256 MiB. Job imports use the same protocol under /api/jobs/upload/uploads.

Method and path	Input	Success
`POST /api/datasets/publish/uploads`	Required JSON `size`, `sha256`; optional `name`, `version`, `org`	`201 UploadSessionCreated`
`GET /api/datasets/publish/uploads/{uploadId}`	Upload id	`200 UploadSession`
`HEAD /api/datasets/publish/uploads/{uploadId}`	Upload id	`200`, current offset headers
`PATCH /api/datasets/publish/uploads/{uploadId}`	Binary chunk with required headers	`204`, new `Upload-Offset`
`DELETE /api/datasets/publish/uploads/{uploadId}`	Upload id	`204`, no body
`POST /api/datasets/publish/uploads/{uploadId}/complete`	No body	`202 DatasetImport`

Creation declares the exact archive byte size and the entire archive’s sha256 as 64 lowercase hexadecimal characters. The response supplies chunk bounds. For dataset uploads, name/version can come from the archive manifest. Send chunks sequentially. Every PATCH requires:

Header	Value
`Content-Type`	`application/offset+octet-stream`
`Content-Length`	This chunk’s byte count
`Upload-Offset`	Current accepted byte offset
`Upload-Checksum`	`sha256` followed by the Base64-encoded SHA-256 digest of this chunk

Every chunk must be no larger than chunk_max_bytes. Non-final chunks must also be at least chunk_min_bytes. After a transfer error, use HEAD to recover the accepted offset; do not assume the last chunk failed. Complete only after every declared byte has been accepted. Completion verifies the entire archive and returns a pollable import. Repeating a successful completion returns the same import. A whole-archive digest mismatch fails the session; start a new upload. An incomplete transfer returns 409 upload_incomplete. For an HTTP client that does not implement chunk checksums and recovery, use the dataset SDK or job upload SDK.

Errors and ownership

Dataset writes require ownership. A name owned by another account returns 409 dataset_name_taken. Deletion refuses a dataset referenced by jobs with 409 dataset_in_use, including the blocking ids. Activation refuses unfinished versions with version_not_ready and failed or archived versions with version_not_activatable. Publish errors include unpinned_git_ref, invalid archives/manifests, 413 import_too_large, and temporary upload capacity refusals. Chunk errors distinguish offset mismatch, chunk digest mismatch, and full-archive digest mismatch. See dataset import and resumable-upload errors.

HTTP API

Python and TypeScript

Run agents

Shared behavior

SDK workflows

SDK resources

SDK methods