trickroom / Docs
Sign in
API reference/assistants

Create a taskset: a draft of evaluation tasks to publish as versions

Application-scoped assistant control plane. External pipelines, dataset ingestion, and training run in the builder's infrastructure. Eval suites run fixture tool-mode jobs or disposable offline Linux desktops through Temporal. Metrics/artifacts are externally reported; reference URLs are never fetched. Model credentials are write-only. Pull subscriptions replay unacknowledged events. Resource lists return up to 200 recent records; trigger lists return an ID next_cursor for after, and message and delivery feeds use sequence cursors.

POST/v1/applications/{id}/tasksets

Authorization

Authorizationstringrequired

Bearer tak_… — an application API key. Keep it on your backend; all resources are scoped to that application.

Create an application key →

Path parameters

idstringrequired

Min length: 1

Body application/json

namestringrequired

Min length: 1 · Max length: 100

environmentobject

Default: {"kind":"tools"}

Allowed shapes

object

kind"tools"required

object

kind"stateful"required

object

kind"desktop"required
imagestring

Pattern: ^(?:[a-zA-Z0-9._/:\-]+@)?sha256:[a-f0-9]{64}$

snapshot_idstring

Pattern: ^snap_[0-9a-f]{32}$

scorersarray<object>

Max items: 8

Allowed shapes

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"model_judge"required
modelstringrequired

Min length: 1 · Max length: 200

rubricstringrequired

Min length: 1 · Max length: 8000

weightnumber

Default: 1 · Minimum: 0 · Maximum: 1000

pass_thresholdnumber

Default: 0.5 · Minimum: 0 · Maximum: 1

calibrationobject

Default: {"min_labels":20,"min_kappa":0.6}

Object fields
min_labelsinteger

Default: 20 · Minimum: 1 · Maximum: 10000

min_kappanumber

Default: 0.6 · Minimum: -1 · Maximum: 1

object

idstringrequired

Pattern: ^[a-zA-Z0-9_.-]{1,100}$

kind"webhook"required
urlstringrequired

Max length: 2000

weightnumber

Default: 1 · Minimum: 0 · Maximum: 1000

Response

201Create a taskset: a draft of evaluation tasks to publish as versions
X-Request-Idstring

Server-generated request identifier for logs and support.

application/json

idstringrequired

Min length: 1

created_atstringrequired

Format: date-time · Pattern: ^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d(?:\.\d+)?(?:Z|([+-](?:[01]\d|2[0-3]):[0-5]\d)))$

application_idstringrequired

Min length: 1

namestringrequired
environment_jsonobjectrequired

Additional keys: any

scorers_jsonarray<object>required

Additional keys: any

webhook_secret_setbooleanrequired
draft_tasksinteger

Minimum: -9007199254740991 · Maximum: 9007199254740991

latest_versioninteger | null
4XX400 invalid input; 401 missing/revoked credentials; 403 disallowed operation/application; 404 missing or inaccessible resource; 409 conflicting submission/version/state; 413 body too large; 415 unsupported content type. Inspect error.code, not message text.
X-Request-Idstring

Server-generated request identifier for logs and support.

application/json

errorobjectrequired
Object fields
codestringrequired
messagestringrequired
request_idstringrequired

Additional keys: any

5XX500 unexpected failure or 503 unavailable dependency. A failed response does not prove that a submission was not committed. Retry job/flow admission with the same key and body.
X-Request-Idstring

Server-generated request identifier for logs and support.

application/json

errorobjectrequired
Object fields
codestringrequired
messagestringrequired
request_idstringrequired

Additional keys: any