Braintrust
App · AI Models · 2 actions
Run and record AI evaluations in Braintrust: create experiments and launch evals over your datasets, tasks, and scorers.
Create experiment
Register a new experiment in a project. Use this to record eval results you log yourself. To have Braintrust run an eval for you, use Launch evaluation instead.
Inputs
- Project
project_idtextRequired - The project the experiment belongs to.
- Experiment name
nametext - A name for the experiment. Leave blank to have Braintrust generate one.
- Description
descriptiontext - An optional description of the experiment.
- Dataset
dataset_idtext - An optional dataset to link to the experiment for reproducibility.
- Always create a new experiment
ensure_newtrue/false - By default, reusing an existing name returns that experiment. Turn this on to always create a fresh one with a unique name instead.Default:
false. - Public
publictrue/false - Turn on to make the experiment viewable by anyone with the link, inside or outside your organization.Default:
false.
Outputs
createdtext- When the experiment was created.
experiment_idtext- ID of the new experiment.
nametext- The experiment's name.
project_idtext- ID of the project the experiment is in.
publictrue/false- Whether the experiment is public.
Example output
{
"created": "2026-07-31T12:00:00Z",
"experiment_id": "9a8b7c6d-0000-1111-2222-333344445555",
"name": "Test Run",
"project_id": "b3f2c1a0-1111-2222-3333-444455556666",
"public": false
}Launch evaluation
Run an evaluation: Braintrust runs your task function over a dataset, grades each result with the scorers you pick, creates an experiment, and returns the summary.
Inputs
- Project
project_idtextRequired - The project to run the evaluation in.
- Dataset
dataset_idtextRequired - The dataset to evaluate against.
- Task
task_function_idtextRequired - The function to evaluate (the task being graded).
- Scorers
score_function_idslistRequired - One or more scorers used to grade each result. Choose built-in scorers or your project's own scoring functions.
- Experiment name
experiment_nametext - A name for the experiment this eval creates. Leave blank to have Braintrust generate one.
Outputs
experiment_nametext- Name of the experiment the eval created.
experiment_urltext- Link to the experiment in Braintrust.
metricsobject- Aggregate metrics (such as duration), keyed by name.
project_nametext- Name of the project.
project_urltext- Link to the project in Braintrust.
scoresobject- Average score for each scorer, keyed by scorer name.
Example output
{
"experiment_name": "eval-2026-07-31",
"experiment_url": "https://www.braintrust.dev/app/My%20Org/p/My%20Project/experiments/eval-2026-07-31",
"metrics": {
"duration": {
"improvements": 20,
"metric": 1.42,
"name": "duration",
"regressions": 5,
"unit": "seconds"
}
},
"project_name": "My Project",
"project_url": "https://www.braintrust.dev/app/My%20Org/p/My%20Project",
"scores": {
"Factuality": {
"improvements": 12,
"name": "Factuality",
"regressions": 3,
"score": 0.87
}
}
}