Read an evaluation suite

Application-scoped assistant control plane. External pipelines, dataset ingestion, and training run in the builder's infrastructure. Eval suites run fixture tool-mode jobs or disposable offline Linux desktops through Temporal. Metrics/artifacts are externally reported; reference URLs are never fetched. Model credentials are write-only. Pull subscriptions replay unacknowledged events. Resource lists return up to 200 recent records; trigger lists return an ID next_cursor for after, and message and delivery feeds use sequence cursors.

GET/v1/applications/{id}/eval-suites/{suiteId}

Authorization

Authorizationstringrequired

Bearer tak_… — an application API key. Keep it on your backend; all resources are scoped to that application.

Create an application key →

Path parameters

idstringrequired

Min length: 1

suiteIdstringrequired

Min length: 1

Response

200Read an evaluation suite

X-Request-Idstring

Server-generated request identifier for logs and support.

application/json

idstringrequired

Min length: 1

created_atstringrequired

Format: date-time · Pattern: ^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d(?:\.\d+)?(?:Z|([+-](?:[01]\d|2[0-3]):[0-5]\d)))$

application_idstringrequired

Min length: 1

definition_jsonobjectrequired

Object fields

namestringrequired

Min length: 1 · Max length: 100

versionstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

environmentobject

Default: {"kind":"tools"}

Allowed shapes

object

kind"tools"required

object

kind"stateful"required

object

kind"desktop"required

imagestring

Pattern: ^(?:[a-zA-Z0-9._/:\-]+@)?sha256:[a-f0-9]{64}$

snapshot_idstring

Pattern: ^snap_[0-9a-f]{32}$

casesarray<object>

Min items: 1 · Max items: 20

Item fields

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

taskstringrequired

Min length: 1 · Max length: 12000

referencestring

Max length: 12000

proactiveboolean

Default: false

initial_stateobject

Additional keys: any

connectionsarray<object>

Default: [] · Max items: 12

Item fields

idstringrequired

Pattern: ^con_[a-f0-9]{32}$

providerstringrequired

Allowed: "gmail""calendar""github""drive""slack""figma"

accountstringrequired

Min length: 1 · Max length: 200

desktopobject

Object fields

filesarray<object>

Default: [] · Max items: 32

Item fields

pathstringrequired

Pattern: ^[a-zA-Z0-9_-][a-zA-Z0-9_./-]{0,199}$

contentstringrequired

Max length: 16000

start_pathstring | null

Default: null

historyarray<object>

Default: [] · Max items: 12

Item fields

rolestringrequired

Allowed: "user""assistant"

contentstringrequired

Max length: 4000

memoriesarray<object>

Default: [] · Max items: 20

Item fields

keystringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

contentstringrequired

Max length: 4000

sourcestringrequired

Max length: 500

toolsarray<object>

Default: [] · Max items: 20

Item fields

namestringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

descriptionstring

Max length: 2000

parametersobject

Additional keys: any

responseany

stateobject

Object fields

operationstringrequired

Allowed: "read""append""set"

pathstringrequired

Max length: 500 · Pattern: ^(?:\/(?:[^~]|~[01])*)?$

value_fromstring

Max length: 500 · Pattern: ^(?:\/(?:[^~]|~[01])*)?$

approvalstring

Allowed: "approve""reject"

checksarray<object>required

Min items: 1 · Max items: 20

Allowed shapes

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"state_equals"required

pathstringrequired

Max length: 500 · Pattern: ^(?:\/(?:[^~]|~[01])*)?$

valueanyrequired

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"state_count"required

pathstringrequired

Max length: 500 · Pattern: ^(?:\/(?:[^~]|~[01])*)?$

countintegerrequired

Minimum: 0 · Maximum: 1000

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"file_sha256"required

pathstringrequired

Min length: 1 · Max length: 1000 · Pattern: ^\/

sha256stringrequired

Pattern: ^[a-f0-9]{64}$

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"desktop_action"required

namestringrequired

Allowed: "click""double_click""type""press_key""hotkey""scroll""open_url"

argumentsobject

Additional keys: any

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"output_equals"required

textstringrequired

Max length: 12000

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"output_contains"required

textstringrequired

Min length: 1 · Max length: 12000

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"output_json"required

valueanyrequired

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"silent"required

expectedbooleanrequired

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"tool_call"required

namestringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

argumentsobject

Additional keys: any

scorersarray<object>

Max items: 8

Allowed shapes

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"model_judge"required

modelstringrequired

Min length: 1 · Max length: 200

rubricstringrequired

Min length: 1 · Max length: 8000

weightnumber

Default: 1 · Minimum: 0 · Maximum: 1000

pass_thresholdnumber

Default: 0.5 · Minimum: 0 · Maximum: 1

calibrationobject

Default: {"min_labels":20,"min_kappa":0.6}

Object fields

min_labelsinteger

Default: 20 · Minimum: 1 · Maximum: 10000

min_kappanumber

Default: 0.6 · Minimum: -1 · Maximum: 1

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"webhook"required

urlstringrequired

Max length: 2000

weightnumber

Default: 1 · Minimum: 0 · Maximum: 1000

taskset_idstring | null

numberinteger | null

task_countinteger | null

content_hashstring | null

4XX400 invalid input; 401 missing/revoked credentials; 403 disallowed operation/application; 404 missing or inaccessible resource; 409 conflicting submission/version/state; 413 body too large; 415 unsupported content type. Inspect error.code, not message text.

X-Request-Idstring

Server-generated request identifier for logs and support.

application/json

errorobjectrequired

Object fields

codestringrequired

messagestringrequired

request_idstringrequired

Additional keys: any

5XX500 unexpected failure or 503 unavailable dependency. A failed response does not prove that a submission was not committed. Retry job/flow admission with the same key and body.

X-Request-Idstring

Server-generated request identifier for logs and support.

application/json

errorobjectrequired

Object fields

codestringrequired

messagestringrequired

request_idstringrequired

Additional keys: any