> ## Documentation Index > Fetch the complete documentation index at: https://docs.evolvingmachines.ai/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > Evolve has two products behind one API key. Edith answers questions about an agent trajectory over HTTP: `POST https://api.evolvingmachines.ai/v1/decisions`. Evaluations run coding agents on tasks in cloud sandboxes, through the `evolve` CLI, the Python and TypeScript SDKs, or HTTP. > A person creates the key at https://dashboard.evolvingmachines.ai/api-keys. Read it from `EVOLVE_API_KEY`; never ask for it in chat or write it to a file. > Install the CLI with `npm install -g @evolvingmachines/evolve`, then confirm the connection with `evolve auth status`. > The CLI carries the manual for its own version. `evolve skills get evals` prints the index and `evolve skills get evals ` prints one page, for example `evolve skills get evals cli-reference/run`. When the installed CLI is older than this site, follow the installed manual. > Commands take `--help` for their options and `--json` for machine-readable output. Evaluations spend credits: start with one task and set `--max-trial-spend`. # Environment lint > Flag reward hacks and grading holes in every task you publish, before anyone trains on them. Environment lint reads task files during publication and flags potential grading problems. It does not run the task or verifier. ```bash theme={"dark"} evolve dataset publish \ --dir ./tasks \ --name my-bench \ --version 1.0 \ --watch ``` Findings are advisory, including those marked **error**. They do not block publication or execution. Review the suggested fixes, update affected tasks, and publish a new version. ## Where you see it The report appears after task validation, before image builds: | Surface | What to read | | - | - | | `publish --watch` | Finding counts, affected rules, and suggested fixes | | `publish --watch --json` | The `import.lint` event | | SDK `watchImport` / `watch_import` | The `onLint` / `on_lint` callback | | `dataset show` | Summary plus task findings, with file and line | | Dataset detail | The `lint` summary and each task's `lint` findings | ```bash theme={"dark"} evolve dataset show my-bench@1.0 ``` The import includes an `environment_lint_findings` warning when it has errors or warnings. Rejected tasks are not linted; a publication with no accepted tasks has no report. | Report value | Meaning | | - | - | | **Template-wide** | A rule affects at least half the tasks and at least five tasks. Check for a shared template or verifier. The CLI prints it once with its task count. | | `lint: null` / **not linted** | No lint report is available, including for older publications. It does not mean the tasks passed lint. | | `truncated` / `?` in the LINT column | Error or warning findings were omitted to fit the report limit. A task with no listed findings may still have findings. | | `infos_truncated` | Informational findings were omitted. When `truncated` is false, all errors and warnings remain listed. | Summaries group findings by rule and severity, with the most severe first. Lint can detect only problems visible in the files. Use a [task check](https://docs.evolvingmachines.ai/core-concepts/check.md) to review behavior that requires execution, such as whether a numeric check rejects wrong answers. > [accordion] Rule origins > > The rules cover static checks from the defect classes reported by Tokenless for Terminal-Bench 4, DeepSWE 1.1, and BFCL v4. See [envcheck-findings](https://github.com/tokenless-ai/envcheck-findings). ## Rules | Rule | Level | What it catches | | - | - | - | | `answer-in-agent-image` | error | The agent's Dockerfile copies `solution/` or an oracle tree into the image and no later `RUN` deletes it. | | `reward-constant` | error | Every reward the verifier writes is a constant 1, with no `set -e` or `&&` that makes the write depend on the tests. | | `instruction-missing` | error, warning or info | `instruction.md` is missing or has fewer than 20 characters (error), is not UTF-8 text (warning), or is over 512 KiB, too large for the linter to read (info). | | `answer-material-in-tests` | warning or info | `tests/` holds copies of solution files the agent does not already have, or files named like answers: `oracle`, `golden`, `answer(s)`, `ground_truth`, `solution(s)`, `reference` (for code files only `oracle`, `golden`, `solution(s)`, `reference`). A warning when the verifier visibly runs the submission's code while `/tests` is readable, info otherwise. | | `verifier-runs-submission-test-config` | warning | Tests run inside the submission's tree, so its `conftest.py`, `pytest.ini`, `TestMain`, runner config and test helpers run with the graded tests. | | `verifier-installs-submission-deps` | warning | The verifier installs a requirements file, `package.json` or lock file from the agent's workspace with its own privileges (an install run as an unprivileged user is the fix, and is not flagged). | | `test-without-assertion` | warning | A test function, in `tests/` or added by `tests/test.patch`, asserts nothing. | | `test-trivial-assertion` | warning | A test asserts only a constant, such as `assert True`. | | `reward-not-written` | warning | Nothing under `tests/` writes `reward.txt` or `reward.json`. | | `verifier-network-install` | warning | The verifier fetches from the network (`pip`/`uv`/`npm` installs, `curl`, `wget`, `git clone`, `apt`) while its network mode is `no-network`. Requests to localhost and offline installs are not counted. | | `verifier-unpinned-install` | warning | The verifier installs packages without pinned versions at grading time. | | `verifier-remote-script` | warning | The verifier pipes a downloaded script into a shell. | | `solution-missing` | warning | There is no `solution/solve.sh`, so nobody can show a correct submission scores 1. | | `duplicate-instruction` | warning | Another task in the version has the same instruction and the same tests. | | `test-exit-code-only` | info | A test checks only an exit code or status code, never the state or output it should produce. | | `instruction-output-untested` | info | The instruction asks for an output file that no file under `tests/` mentions. | | `shared-verifier` | info | The verifier runs inside the agent's box, where anything the agent leaves behind runs during grading. | | `base-image-unpinned` | info | A base image has no fixed tag, so a rebuild can change the environment. | This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.