name@version. A bare name means the dataset’s active version.
Browse the catalog
dataset list prints every dataset you can run: the platform’s public datasets and your own. --search <text> filters by name and description. dataset show prints one dataset’s versions, its tasks with their timeouts, and for each task which sandbox providers can run it.
Publish your own
Any folder of task directories can be published: a single task directory, a directory with atasks/ folder, or a directory of task directories. What you publish is private to your organization; another account asking for its name reads dataset_not_found. A name belongs to its first publisher: re-publishing your own name adds a version, and a name owned by anyone else is refused with dataset_name_taken.
Check it first: the pre-flight is a dry run that uploads nothing and writes nothing.
- Local directory
- Git repository
- Fetchable source
dataset.toml manifest, --name and --version come from it and may be omitted.Everything in the directory is packed, dotfiles included, and executables stay executable. Only .git, .DS_Store and .venv are left out, and symlinks are never packed. A large upload resumes after a dropped connection, and evolve dataset watch can follow it from another machine while it uploads: it reads QUEUED (receiving) until the corpus has arrived.--watch follows the publish until the version is READY or FAILED. Each task builds on its own, so one broken task does not block the others; --watch ends with how many built. If the terminal is gone, evolve dataset watch <name> re-attaches to the same follow, from any machine. --skip-preflight uploads without the check; a task the check would have refused then fails at import instead.
The pre-flight
The pre-flight sends each task’stask.toml, and the dataset.toml if there is one, and answers with a verdict per task. It checks what a task.toml alone can decide, so an all-ok answer means no config refuses, not that every image will build. A NOTE is not a refusal; today it is tests_dockerfile_not_built. A refused task stops the publish before anything is uploaded.
The manifest
A corpus with adataset.toml imports what the manifest says: only the tasks listed under [[tasks]], each verified against its pinned digest. A listed task the checkout lacks fails the publish with manifest_task_missing, a digest mismatch with manifest_digest_mismatch. The [dataset] description reaches the catalog, and every version records the identity it imported under as manifest. A metric.py custom metric is refused with custom_metric_not_supported.
What happens when you publish
Every declaration intask.toml is honored or refused with the reason; a task never runs on weaker semantics than it declares.
Each task’s environment/ is built into an image once. Tasks build independently: one that fails any step is recorded FAILED with a typed reason, and the others keep building.
The version lands READY when at least one task built, and FAILED only when none did or on a corpus-level refusal; a FAILED version changes nothing, the dataset keeps serving what it served. On your own dataset, READY also makes the version active, so the bare name runs it. A job refuses any version that is not READY with version_not_ready.
The first trial of a task on a provider takes a few minutes longer than the rest; it is not a hang.
The import’s warnings name what a version will permanently lack: no_solutions_archived or partial_solutions_archived for the reference-solution record, tasks_failed_to_build when some tasks failed, tests_dockerfile_not_built for the tasks whose verifier never builds their tests/Dockerfile.
Partially built versions
On the version,task_count counts the READY tasks and n_failed_tasks the rest. dataset show name@version lists every failed task with its reason, and dataset publish --watch prints each task’s outcome. A whole-dataset or glob job on such a version runs the READY tasks and says so in its build_exclusions. Naming a failed task explicitly at job create is refused with task_failed_to_build; a fix is a new version.
Version states
A version movesDRAFT → RECEIVING (a large upload still streaming) → IMPORTING → BUILDING → READY, with FAILED and ARCHIVED as off-ramps. READY is the only state that accepts jobs.
An import is QUEUED (receiving: true while the corpus is still uploading), RUNNING, COMPLETED (the version is READY) or FAILED (read failure). A finished import stays readable as long as its dataset exists.
Versions
Each publish creates a version, and a version that lands READY is built and active. To point the bare name at a different READY version:version_not_ready; a FAILED or ARCHIVED one with version_not_activatable.
The owner of a dataset can download the original package back:
solution/ included, and the one call that returns task files. Only the owner may: a platform dataset has no owner and cannot be downloaded, and someone else’s dataset reads as not found.
Deleting a dataset removes its versions, tasks and archived solutions. You must own it (dataset_not_owned on a platform dataset), and a dataset any job ran against is never deleted: dataset_in_use names the blocking jobs.
A dataset published from git records what it was built from, and dataset list and dataset show print one line when the ref has moved upstream. A new version is always one you publish, unless you opt in with datasets().update(name, { upstream_auto_import: true }); that is refused with upstream_not_watchable on a dataset with no moving git ref.
dataset reference
Every flag of
evolve dataset.