Secrets

Store credentials in Evolve, then attach them to jobs by name. Secret values are write-only; reads return metadata.

1. Store the secret
   GITHUB_TOKEN
         ↓
2. Attach its reference
   --secret GITHUB_TOKEN
         ↓
3. Run the trial
   Value enters agent environment

Store a value

Set GITHUB_TOKEN in your shell to the credential you want to store, then run:

printf %s "$GITHUB_TOKEN" | evolve secrets set GITHUB_TOKEN \
  --label staging \
  --delivery direct

Piping keeps the value out of this command’s text. This example stores the staging variant. Omit --label to use default. Eval jobs support direct delivery: the value enters the agent sandbox. Brokered secrets are supported by other Evolve surfaces but are rejected for managed evals.

Attach it to a job

evolve run \
  -d my-dataset@1.0 \
  -a codex \
  -m gpt-6-luna \
  --secret GITHUB_TOKEN@staging \
  --max-trial-spend 1 \
  --max-retries 0 \
  --watch

Attachment	Meaning
`GITHUB_TOKEN`	Use the default label, or the only available label
`GITHUB_TOKEN@staging`	Select the `staging` value
`GITHUB_TOKEN@staging=GH_TOKEN`	Expose it as `GH_TOKEN` inside the sandbox

If several labels exist and none is default, select one explicitly. Every arm receives the attachment. Reserved routing names and the EVOLVE_ prefix cannot be overwritten.

Request it from a task

task.toml

[environment.env]
GITHUB_TOKEN = "${GITHUB_TOKEN}"

The template matches the attached environment name. A missing required attachment rejects the job. For a non-secret fallback, use "${NAME:-fallback}". Task literals belong in [environment.env]. Managed eval jobs do not accept arbitrary agent_env values through --ae.

List, replace, or remove

evolve secrets list
evolve secrets delete GITHUB_TOKEN --label staging

A different value under an existing name and label is rejected with secret_exists. Use a new label, or delete the old secret before storing its replacement. Removing or disabling a secret needed by an unfinished job can cause its later trials to fail.

Save and attach in one request

The SDK accepts both references and inline values in a job’s secrets list:

{
  "secrets": [
    {
      "name": "GITHUB_TOKEN",
      "label": "staging"
    },
    {
      "name": "SERVICE_TOKEN",
      "value": "VALUE",
      "delivery": "direct"
    }
  ]
}

An inline value is saved before the job stores its reference. Repeating the same value and delivery reuses the existing secret; a conflicting value is rejected. The CLI equivalent is --secret-inline NAME[@LABEL]:DELIVERY=VALUE.

Secret commands

Storage, labels, and delivery options.

Secret SDK

Manage secrets from Python or TypeScript.

Start here

Run evaluations

Task format

Validate your work

Read results

Work together