SDK reference · Assistants and evaluation
client.evals
client.evals.{ createSuite, suites, suite, run, runs, get, cancel }A suite is a set of cases; a run scores an assistant or model version against it.
createSuite
createSuite(input: EvalSuite, key: string): Promise<Resource>Creates a suite.
Parameters
keystring- An idempotency key. Retrying with the same key and input returns the first result instead of acting twice.
Returns Promise<Resource>
suites
suites(): Promise<Page>The application's suites.
Returns Promise<Page>
suite
suite(id: string): Promise<Resource>One suite.
Returns Promise<Resource>
run
run(input: EvalRun, key: string): Promise<Resource>Starts a run.
Parameters
keystring- An idempotency key. Retrying with the same key and input returns the first result instead of acting twice.
Returns Promise<Resource>
runs
runs(): Promise<Page>The application's runs.
Returns Promise<Page>
get
get(id: string): Promise<Resource>One run and its scores.
Returns Promise<Resource>
cancel
cancel(id: string, key: string): Promise<Resource>Stops a run.
Parameters
keystring- An idempotency key. Retrying with the same key and input returns the first result instead of acting twice.
Returns Promise<Resource>