> ## Documentation Index
> Fetch the complete documentation index at: https://docs.evolvingmachines.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# datasets
> The catalog: list, read, publish, download, activate, delete.
`datasets()` returns the catalog client. A `ref` is `name` or `name@version`.
## list, get, getActive
```ts theme={null}
list(options?: { search?: string; limit?: number; cursor?: string }): DatasetList
get(
ref: string,
options?: { limit?: number; cursor?: string },
): Promise
getActive(
name: string,
options?: { limit?: number; cursor?: string },
): Promise
```
The catalog, one dataset with one page of its tasks (default 200, at most 500), or the active version alone. `getActive` throws `NoActiveVersionError` when nothing is runnable. A task row carries public fields only; see [Task](/sdk-reference/types#task).
```ts theme={null}
const dataset = await datasets().get("terminal-bench-4@4.0");
```
```python theme={null}
def list(
*,
search: Optional[str] = None,
limit: Optional[int] = None,
cursor: Optional[str] = None,
)
async def get(
ref: str,
*,
limit: Optional[int] = None,
cursor: Optional[str] = None,
) -> Dataset
async def get_active(
name: str,
*,
limit: Optional[int] = None,
cursor: Optional[str] = None,
) -> ActiveDataset
```
The catalog, one dataset with one page of its tasks, or the active version alone. `get_active` raises `NoActiveVersionError` when nothing is runnable.
```python theme={null}
dataset = await datasets().get("terminal-bench-4@4.0")
```
## getTaskBuild
```ts theme={null}
getTaskBuild(
ref: string,
taskName: string,
): Promise
```
One task's build record: its outcome and, on failure, the failing step and log.
```ts theme={null}
const build = await datasets().getTaskBuild("my-swe@1.0", "broken-task");
```
```python theme={null}
async def get_task_build(
ref: str,
task_name: str,
) -> TaskBuild
```
One task's build record: its outcome and, on failure, the failing step and log.
```python theme={null}
build = await datasets().get_task_build("my-swe@1.0", "broken-task")
```
## preflight
```ts theme={null}
preflight(input: { source: { directory: string } }): Promise
```
Dry-run a local folder of tasks: only each task's metadata is sent, nothing is uploaded or written.
```ts theme={null}
const verdict = await datasets().preflight({ source: { directory: "./my-swe" } });
```
```python theme={null}
async def preflight(
*,
directory: str,
) -> DatasetPreflight
```
Dry-run a local folder of tasks: only each task's metadata is sent, nothing is uploaded or written.
```python theme={null}
verdict = await datasets().preflight(directory="./my-swe")
```
## publish
```ts theme={null}
publish(input: { source: DatasetSource; name?: string; version?: string }, options?: { onUploadProgress?: (sentBytes: number, totalBytes: number) => void; onRegistered?: (importId: string) => void }): Promise
```
Publish a version from one `source`: `{ git_url, git_ref, git_path? }` (an `https://` URL, and a subfolder that exists at the ref), `{ directory }`, `{ archive_url }` or `{ hub_package }`. Returns the import record; follow it with `watchImport`.
`onUploadProgress` reports sent bytes for a directory source, and `onRegistered` fires once with the import id when a large upload begins. What is packed is on [datasets](/core-concepts/datasets#publish-your-own).
```ts theme={null}
const imp = await datasets().publish({
source: { directory: "./my-swe" },
name: "my-swe",
version: "1.0",
});
```
```python theme={null}
async def publish(
*,
git_url=None,
git_ref=None,
git_path=None,
directory=None,
archive_url=None,
hub_package=None,
name=None,
version=None,
on_upload_progress=None,
on_registered=None,
) -> DatasetImport
```
Publish a version from one source: `git_url` with `git_ref` (and `git_path`), `directory`, `archive_url`, or `hub_package`. Returns the import record; follow it with `watch_import`.
```python theme={null}
imp = await datasets().publish(
directory="./my-swe",
name="my-swe",
version="1.0",
)
```
## getImport, watchImport, listImports
```ts theme={null}
getImport(id: string): Promise
watchImport(id: string, options?: { onStatus?: (datasetImport: DatasetImport) => void; onProgress?: (progress: DatasetImportProgress, datasetImport: DatasetImport) => void; onVersion?: (version: DatasetVersion, dataset: Dataset) => void; signal?: AbortSignal; pollIntervalMs?: number; settleTimeoutMs?: number }): Promise
listImports(options?: { status?: DatasetImportStatus; dataset?: string; limit?: number; cursor?: string }): DatasetImportList
```
Read one publish, follow it until the version is READY or FAILED, or list your publishes. `watchImport` polls every `pollIntervalMs` (default 2000): `onStatus` fires on every status change, `onProgress` on every change of the five-phase `progress` (`extracting`, `parsing`, `building`, `copying`, `verifying`), `onVersion` on the version's state. `settleTimeoutMs` (default 30 minutes) bounds the wait; past it the watch throws `ImportSettleError` with the last observed state. `listImports` filters by `status` and `dataset`.
```ts theme={null}
const done = await datasets().watchImport(imp.id);
```
```python theme={null}
async def get_import(id: str) -> DatasetImport
async def watch_import(
id: str,
*,
on_status=None,
on_progress=None,
on_version=None,
poll_interval_s: float = 2.0,
timeout_s: Optional[float] = None,
settle_timeout_s: float = ...,
) -> DatasetImport
def list_imports(
*,
status: Optional[str] = None,
dataset: Optional[str] = None,
limit: Optional[int] = None,
cursor: Optional[str] = None,
)
```
Read one publish, follow it until the version is READY or FAILED, or list your publishes. `watch_import` polls every `poll_interval_s`: `on_status` fires on every status change, `on_progress` on every change of the five-phase `progress` (`extracting`, `parsing`, `building`, `copying`, `verifying`), `on_version` on the version's state. `settle_timeout_s` (default 30 minutes) bounds the wait; past it the watch raises `ImportSettleError` with the last observed state. `list_imports` filters by `status` and `dataset`.
```python theme={null}
done = await datasets().watch_import(imp.id)
```
## download
```ts theme={null}
download(ref: string): Promise
download(
ref: string,
options: { to: string },
): Promise
download(
ref: string,
options: { stream: true },
): Promise>
```
The original package of a dataset you own: the bytes, the saved file's path, or a stream. The Buffer and `{ to }` shapes verify the download against the digest the server states and throw `EvolveDigestMismatchError` on a mismatch; `{ stream: true }` is unverified. A version published before packages were retained answers `package_not_retained`. Prefer `{ to }` for anything sizeable.
```ts theme={null}
const path = await datasets().download(
"my-swe@1.0",
{ to: "./corpora" },
);
```
```python theme={null}
async def download(
ref: str,
*,
to: Optional[str] = None,
)
```
The original package of a dataset you own: the bytes, or with `to` the saved file's path. Both shapes verify the download against the digest the server states, raising `EvolveDigestMismatchError` on a mismatch. A version published before packages were retained answers `package_not_retained`. Prefer `to` for anything sizeable.
```python theme={null}
path = await datasets().download(
"my-swe@1.0",
to="./corpora",
)
```
## activate, update, delete
```ts theme={null}
activate(
name: string,
version: string,
): Promise
update(
name: string,
patch: { upstream_auto_import: boolean },
): Promise
delete(name: string): Promise
```
Make a READY version the active one (`version_not_ready` while building, `version_not_activatable` for FAILED or ARCHIVED), switch automatic import of new upstream versions (`upstream_not_watchable` without a moving git ref), or delete a dataset you own (`dataset_not_owned` on a platform dataset, `dataset_in_use` naming the jobs that reference it).
```ts theme={null}
await datasets().activate("my-swe", "1.0");
```
```python theme={null}
async def activate(
name: str,
version: str,
) -> Dataset
async def update(
name: str,
*,
upstream_auto_import: bool,
) -> Dataset
async def delete(name: str) -> None
```
Make a READY version the active one (`version_not_ready` while building, `version_not_activatable` for FAILED or ARCHIVED), switch automatic import of new upstream versions (`upstream_not_watchable` without a moving git ref), or delete a dataset you own (`dataset_not_owned` on a platform dataset, `dataset_in_use` naming the jobs that reference it).
```python theme={null}
await datasets().activate("my-swe", "1.0")
```