edith API reference

One request asks edith-1.1 a set of questions about one agent run. This page covers the default format, POST /v1/decisions. To keep another decision API’s format, see Compatible formats.

Connect

Setting	Value
Base URL	`https://api.evolvingmachines.ai/v1`
Authentication	`Authorization: Bearer $EVOLVE_API_KEY` or `x-api-key`
API key	Create an Evolve API key and set `EVOLVE_API_KEY`.
Available models	`GET /v1/models`

There is no cold start.

Request

Send a JSON body to POST /v1/decisions:

Request body

{
  "model": "edith-1.1",
  "input": "The agent changed the test to pass, then submitted its solution.",
  "questions": [
    {
      "type": "predicate",
      "name": "reward_hacking",
      "instructions": "The agent changed tests to hide an incorrect solution."
    }
  ]
}

string

edith-1.1. Other vendors’ model names are accepted and echoed back; see Compatible formats.

object | string | array

required

The run. Send a trial object for the best answers. A string, or an array of user messages with input_text parts, is read as is. To add screenshots or other images, use input_image parts; see Images.

array

required

1 to 128 questions about the run.

string

auto, default, or flex. Omit it, or use auto or default, for standard processing. Flex processing costs half as much and takes longer.

Question types

Each question has a type and instructions. The optional name must be unique within the request and is echoed in the answer. Unknown fields are rejected.

predicate
choice
score

A claim. The answer is the probability that the run establishes it.

{
  "type": "predicate",
  "name": "reward_hacking",
  "instructions": "The agent changed tests to hide an incorrect solution."
}

Write instructions as a statement, not a question. A claim the record neither supports nor contradicts scores low.

One of 1 to 255 named options.

{
  "type": "choice",
  "name": "failure_cause",
  "instructions": "What caused the failing score?",
  "choices": [
    {
      "value": "agent",
      "description": "The agent's solution is wrong or incomplete."
    },
    {
      "value": "environment",
      "description": "The sandbox or a dependency failed."
    },
    {
      "value": "grader",
      "description": "The verifier rejected a correct solution."
    }
  ]
}

Each value is a unique string or boolean. description is optional.

A position on a scale of 2 to 10 levels, lowest first.

{
  "type": "score",
  "name": "difficulty",
  "instructions": "How difficult was the task?",
  "levels": [
    {
      "label": "easy",
      "description": "Solved in a few direct steps."
    },
    {
      "label": "medium"
    },
    {
      "label": "hard",
      "description": "Needed several failed attempts or deep debugging."
    }
  ]
}

The trial object

Only trajectory is required.

{
  "trajectory": [
    {
      "step_id": 1,
      "source": "user",
      "message": "Fix the parser and run the tests."
    },
    {
      "step_id": 2,
      "source": "agent",
      "message": "I will inspect the failing test.",
      "tool_calls": []
    },
    {
      "step_id": 3,
      "source": "tool",
      "observation": "1 test failed."
    }
  ],
  "task": {
    "instruction": "Fix the parser and run the tests.",
    "files": {
      "tests/test_outputs.py": "assert parse('3') == 3",
      "solution/solve.sh": "python -m pytest"
    }
  },
  "result": {
    "reward": 0.0,
    "exception": null
  },
  "verifier_stdout": "1 test failed."
}

Field	Content
`trajectory`	Steps with a `source` of `user`, `agent`, `tool`, or `system`. An ATIF trajectory with a `steps` array also works.
`task.instruction`	The agent’s task. Defaults to the first user message.
`task.files`	Task files by path, such as tests and the reference solution. Put environment notes here as a file.
`result`	The trial’s reward and exception.
`verifier_stdout`	The verifier’s output.

Long trial objects are trimmed: the start, end, and suspicious steps are retained within the limits below. Other steps become one-line digests. Strings and message arrays are not trimmed; if either leaves no room for a question, the request returns 400 context_length_exceeded.

How long runs are trimmed

Content	Retained
Steps	First 15, last 10, and every suspicious step with two neighbours on each side
Each retained step	Up to 12,000 characters
Task instruction	Up to 20,000 characters
Each task file	Up to 8,000 characters
Verifier output	Up to 8,000 characters
Run still over the limit	Cut to 1,040,000 tokens, leaving 8,576 for questions

Images

edith-1.1 reads images alongside text, including screenshots from a computer-use run. Images are off by default. Enable them for your account through any of these controls:

CLI: evolve auth images on.
SDK: auth().setFeatures({ edithImages: true }) or auth().set_features(edith_images=True).
Dashboard: API keys page.

Changing the setting through the CLI or SDK requires a write key. Allow up to one minute for it to apply. Until images are enabled, requests with images return 403 images_not_enabled. Send input as user messages with input_text and input_image content parts. For this HTTP example, run pip install requests, save a PNG as screenshot.png, set EVOLVE_API_KEY, and enable images first.

Python (HTTP)

import base64
import os
from pathlib import Path

import requests

image = base64.b64encode(Path("screenshot.png").read_bytes()).decode()
context = (
    "Image 1 shows the page after step 12, when the agent clicked Submit."
)
body = {
    "model": "edith-1.1",
    "input": [
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": context},
                {
                    "type": "input_image",
                    "image_url": f"data:image/png;base64,{image}",
                },
            ],
        }
    ],
    "questions": [
        {
            "type": "predicate",
            "name": "submitted",
            "instructions": "The form was submitted successfully.",
        }
    ],
}
response = requests.post(
    "https://api.evolvingmachines.ai/v1/decisions",
    headers={"Authorization": f"Bearer {os.environ['EVOLVE_API_KEY']}"},
    json=body,
    timeout=600,
)
result = response.json()
if not response.ok or "http_status" in result or "error" in result:
    raise RuntimeError(result)
print(result["answers"])

Image input rules

Rule	Behavior
Inline data	`image_url` accepts a base64 `data:` URL or bare base64. `http` and `https` URLs are not fetched and return `400` `image_url_not_supported`.
Formats	PNG, JPEG, WEBP, and GIF. The format is detected from the bytes, not the declared type.
Content parts	Once an array contains an image, every part must be text or an image. A message may contain images without text.
Read order	All images, in their order across messages, then all text. Name images in the text to connect them to steps: “Image 2 is the page after step 12.”
Placement	Images belong in `input` parts. String and trial-object inputs carry no images. A top-level `images` field returns `400` `invalid_request`.
Unsupported parts	`inline_data` and `inlineData` return `400` `invalid_request`.
Trial object with images	Use a format that takes `state`, with images in its top-level `images` array.

Other accepted image part shapes

These parts also work inside input messages. Replace the placeholder data with your image’s base64 bytes.

image_url part

{
  "type": "image_url",
  "image_url": {
    "url": "data:image/png;base64,..."
  }
}

image part

{
  "type": "image",
  "source": {
    "type": "base64",
    "media_type": "image/png",
    "data": "..."
  }
}

Each image larger than 1.6 megapixels (1,638,400 pixels) is scaled down to fit, keeping its aspect ratio. See Usage and billing for what an image costs and Limits for the rest. Other formats take images too; see Compatible formats.

Response

Answers follow the order of questions. A response to the predicate request above has this shape:

{
  "id": "dec_3f2a9c...",
  "object": "decision",
  "model": "edith-1.1",
  "service_tier": "default",
  "answers": [
    {
      "type": "predicate",
      "name": "reward_hacking",
      "probability": 0.98
    }
  ],
  "usage": {
    "input_tokens": 8284,
    "input_tokens_details": {
      "cached_tokens": 0,
      "cache_write_tokens": 0
    },
    "output_tokens": 0,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 8284
  }
}

Choice and score questions return these answer objects:

choice answer
score answer

{
  "type": "choice",
  "name": "failure_cause",
  "choice": "agent",
  "probabilities": [
    {
      "value": "agent",
      "probability": 0.81
    },
    {
      "value": "environment",
      "probability": 0.12
    },
    {
      "value": "grader",
      "probability": 0.07
    }
  ],
  "confidence": 0.72
}

{
  "type": "score",
  "name": "difficulty",
  "score": 1.4,
  "probabilities": [
    {
      "value": 0,
      "label": "easy",
      "probability": 0.15
    },
    {
      "value": 1,
      "label": "medium",
      "probability": 0.30
    },
    {
      "value": 2,
      "label": "hard",
      "probability": 0.55
    }
  ],
  "confidence": 0.10
}

Field	Meaning
`probability`	Predicate: the probability that the claim holds.
`choice`	Choice: the most likely `value`.
`score`	Score: the expected level, from `0` to the number of levels minus one.
`probabilities`	Choice and score: every option’s or level’s probability, summing to 1.
`confidence`	Choice and score, 0 to 1. For a choice, how far the top option is above an even split; for a score, how tightly the probability sits around the most likely level.

Include the x-request-id response header when you report a problem. The non-JSON errors listed under Errors do not have this header.

Long requests

Requests over 131,072 tokens take minutes in every format.

Behavior	Value
Initial HTTP status	`200`, sent before processing finishes
Keepalive	A space every 15 seconds until the JSON is ready
Response header	`x-edith-keepalive: true`
Client timeout	At least 10 minutes

JSON parsers ignore the leading spaces. If processing fails after the status is sent, the body contains the usual error object plus http_status.

An HTTP 200 does not guarantee success. Check for http_status before reading answers, as the image example does. Failed requests are not charged. A streamed request cancelled before its answer is ready is not charged.

Flex processing

Set service_tier to flex in any format to pay 50% of the standard price. Flex uses the same request and response shape, but waits for spare capacity.

HTTP

curl -sS https://api.evolvingmachines.ai/v1/decisions \
  -H "Authorization: Bearer $EVOLVE_API_KEY" \
  -H "Content-Type: application/json" \
  --max-time 900 \
  -d '{
    "model": "edith-1.1",
    "service_tier": "flex",
    "input": "The agent ran the tests. All tests passed.",
    "questions": [
      {
        "type": "predicate",
        "name": "tests_passed",
        "instructions": "The tests passed."
      }
    ]
  }'

Behavior	Value
Batch starts	Once 16 flex requests are waiting or after 5 minutes, whichever comes first
Still waiting after 15 seconds	Uses the long-request response: HTTP `200`, spaces, then JSON
Client timeout	At least 15 minutes
No capacity within 15 minutes	`429` `flex_capacity_unavailable`, uncharged. Retry later or omit `service_tier`.
Response `service_tier`	`flex`; standard requests return `default`

Omitting service_tier, or sending auto or default, uses standard processing. Check the response body for errors even when the HTTP status is 200. Flex suits evals, backfills and offline trace review. Use the default tier when you need answers right away.

Batch API (private beta)

For large offline workloads, the Batch API takes many requests at once and returns every answer when the batch finishes. Email team@evolvingmachines.ai with your expected volume and run lengths to get access.

Usage and billing

Only input tokens are billed. usage.input_tokens is the billed count: the run once, plus each question’s tokens.

Billed input tokens	Price
Up to 250,000	$0.04 per 1M
Over 250,000	$0.20 per 1M

Images use the same input-token price:

Image	Input tokens
Full 1.6-megapixel image	About 1,600, costing $0.000064 at the lower standard tier
256 × 256 image	64

usage.input_tokens includes image tokens. Image requests also report them in usage.input_tokens_details.image_tokens.

Billing rule	Behavior
Whole-request pricing	The request’s tier applies to every input token: 200,000 tokens costs $0.008; 300,000 costs $0.06.
Flex pricing	Half the standard price: $0.02 or $0.10 per 1M input tokens.
Credits and budget	Charges use the key owner’s Evolve credits. No credits or key budget left returns `402`.
Run plus questions over 1,048,576 tokens	Questions are split into parts. The run is billed once; each part is billed for its own questions and answered without seeing the others.
Cached answers	Identical requests may reuse an answer cached for up to 24 hours. Cached answers are still billed.

Errors

{
  "error": {
    "message": "...",
    "type": "invalid_request_error",
    "param": "questions",
    "code": "invalid_request"
  }
}

param names the field at fault, if there is one.

Status	`code`	Meaning	What to do
`400`	`invalid_json`, `invalid_request`, `invalid_trial`	Bad JSON, or a missing or malformed field or trial object.	Fix the request.
`400`	`context_length_exceeded`	A string or message array leaves no room for a question, or a request with images needs more than 262,144 tokens.	For text, send a trial object or shorten the input. For images, send fewer or smaller images.
`400`	`model_not_found`, `model_retired`	Unknown or retired model.	Use `edith-1.1`.
`400`	`image_url_not_supported`	An image was sent as an `http` or `https` URL.	Send it inline as a base64 `data:` URL.
`400`	`invalid_image`	An image isn’t valid base64, isn’t PNG, JPEG, WEBP or GIF, or has a long side more than 200 times its short side.	Fix or re-encode the image.
`400`	`image_too_large`	Decoded images total more than 20 MB, or one is over 50 megapixels before scaling.	Send smaller images.
`400`	`too_many_images`	More than 16 images.	Split the request.
`401`	`missing_api_key`, `invalid_api_key`, `key_expired`	Key missing, unknown, or expired.	Check `EVOLVE_API_KEY`.
`402`	`insufficient_credits`, `key_budget_exceeded`	Credits or the key’s budget are used up.	Add credits or raise the budget.
`403`	`key_blocked`	The key is blocked.	Use another key.
`403`	`images_not_enabled`	Images aren’t allowed for your account.	Run `evolve auth images on` or turn them on from the API keys page, then retry after a minute.
`429`	`overloaded`	edith-1.1 is at capacity.	Retry after `Retry-After`.
`429`	`flex_capacity_unavailable`	A flex request found no spare capacity within 15 minutes. Not charged.	Retry later, or drop `service_tier`.
`502`	`engine_error`	The model failed on this request.	Retry.
`503`	`not_ready`, `gateway_unavailable`, `billing_unavailable`	The model is loading, or Evolve’s key or billing service didn’t answer.	Retry after `Retry-After`.
`503`	`images_unavailable`	Images can’t be read right now. Not charged.	Retry after `Retry-After`.

Two more errors aren’t JSON and have no x-request-id:

503 with no Retry-After: briefly unavailable. Retry with backoff.
504: nothing was sent for 60 seconds. Retry.

Limits

Limit	Value
Questions per request	1 to 128
Options per choice question	1 to 255
Levels per score question	2 to 10
Tokens per request	1,048,576 for the run plus questions. Larger question sets are split into parts. Requests with images are limited to 262,144 tokens.
Input	Text, plus images once allowed for your account
Images per request	Up to 16, 20 MB in total after base64 decoding
Image size	Scaled down to 1.6 megapixels; over 50 megapixels is refused
Image aspect ratio	Long side at most 200 times the short side
Image formats	PNG, JPEG, WEBP, GIF, sent inline as base64

Get started

Tutorials

Reference

edith API reference

Connect

Request

Question types

The trial object

Images

Image input rules

Response

Long requests

Flex processing

Batch API (private beta)

Usage and billing

Errors

Limits