# Braintrust

> Run and record AI evaluations in Braintrust: create experiments and launch evals over your datasets, tasks, and scorers.

App · AI Models · 2 actions. Web page: https://docs.lodol.com/docs/integrations/braintrust

## Actions

### Create experiment

Register a new experiment in a project. Use this to record eval results you log yourself. To have Braintrust run an eval for you, use Launch evaluation instead.

**Inputs**

- Project (`project_id`, text, required): The project the experiment belongs to.
- Experiment name (`name`, text): A name for the experiment. Leave blank to have Braintrust generate one.
- Description (`description`, text): An optional description of the experiment.
- Dataset (`dataset_id`, text): An optional dataset to link to the experiment for reproducibility.
- Always create a new experiment (`ensure_new`, true/false): By default, reusing an existing name returns that experiment. Turn this on to always create a fresh one with a unique name instead. Default: `false`.
- Public (`public`, true/false): Turn on to make the experiment viewable by anyone with the link, inside or outside your organization. Default: `false`.

**Outputs**

- `created` (text): When the experiment was created.
- `experiment_id` (text): ID of the new experiment.
- `name` (text): The experiment's name.
- `project_id` (text): ID of the project the experiment is in.
- `public` (true/false): Whether the experiment is public.

**Example output**

```json
{
  "created": "2026-07-31T12:00:00Z",
  "experiment_id": "9a8b7c6d-0000-1111-2222-333344445555",
  "name": "Test Run",
  "project_id": "b3f2c1a0-1111-2222-3333-444455556666",
  "public": false
}
```

### Launch evaluation

Run an evaluation: Braintrust runs your task function over a dataset, grades each result with the scorers you pick, creates an experiment, and returns the summary.

**Inputs**

- Project (`project_id`, text, required): The project to run the evaluation in.
- Dataset (`dataset_id`, text, required): The dataset to evaluate against.
- Task (`task_function_id`, text, required): The function to evaluate (the task being graded).
- Scorers (`score_function_ids`, list, required): One or more scorers used to grade each result. Choose built-in scorers or your project's own scoring functions.
- Experiment name (`experiment_name`, text): A name for the experiment this eval creates. Leave blank to have Braintrust generate one.

**Outputs**

- `experiment_name` (text): Name of the experiment the eval created.
- `experiment_url` (text): Link to the experiment in Braintrust.
- `metrics` (object): Aggregate metrics (such as duration), keyed by name.
- `project_name` (text): Name of the project.
- `project_url` (text): Link to the project in Braintrust.
- `scores` (object): Average score for each scorer, keyed by scorer name.

**Example output**

```json
{
  "experiment_name": "eval-2026-07-31",
  "experiment_url": "https://www.braintrust.dev/app/My%20Org/p/My%20Project/experiments/eval-2026-07-31",
  "metrics": {
    "duration": {
      "improvements": 20,
      "metric": 1.42,
      "name": "duration",
      "regressions": 5,
      "unit": "seconds"
    }
  },
  "project_name": "My Project",
  "project_url": "https://www.braintrust.dev/app/My%20Org/p/My%20Project",
  "scores": {
    "Factuality": {
      "improvements": 12,
      "name": "Factuality",
      "regressions": 3,
      "score": 0.87
    }
  }
}
```