Skip to main content
Loop can find a failure in your logs, write a scorer for it, and open that scorer for review in a single turn. It combines tools for playgrounds, custom views, dataset snapshots, and the workspace with the Braintrust MCP server’s read and write tools. Explore its capabilities and example prompts by task: investigate data, build objects, run evaluations and automations, or get help.

​
Investigate

Ask questions about your data, from a single trace up to every project in your organization.

​
Explore your data

Loop queries logs, experiments, and datasets with SQL and inspects their schemas. For large or complex traces, it can read, search, and list sandbox files or run commands. Open any trace it resolves or links to in the workspace beside the thread. Loop can also:
  • Hand off to the SQL sandbox. Loop can write and run a query in the SQL sandbox, then refine or fix it on request. The query stays there for you to edit.
  • Start from the logs table. Loop can turn a question into a SQL filter and find traces similar to your selection.
  • Resolve identities. Loop resolves span_attributes.created_by_user_id to organization member names and emails, and span_attributes.created_by_api_key_id to API key and service token names, so you can ask about people and keys by name instead of by ID.
Fields you define in metadata, including ones named like metadata.user_id, describe your own application’s users. Loop doesn’t read them as Braintrust members.
Example prompts:

​
Analyze traces

From the trace view, Loop can diagnose a failure or summarize work across LLM and tool calls. See Debug a trace. To ask about a trace in a Loop thread, paste its permalink or ID. Loop resolves it and loads its spans. Example prompts:

​
Find patterns

Loop searches logs and raw traces for recurring problems and trends, even without a scorer measuring them, and records them as patterns. Each pattern includes supporting evidence. Loop updates an existing pattern when it finds related evidence, rather than creating a duplicate. Example prompts:
Loop can also find patterns on a schedule, without being asked. See Enable Patterns.

​
Analyze projects

Loop can investigate a whole project or compare projects across your organization. For a single project, open Loop agent at the bottom right of its Overview page and enter a prompt. Loop can inspect the project’s logs, recent experiments, datasets, prompts, and score progress chart. Example prompts:
To compare projects, select Loop agent from the organization-level projects list. Ask about project metrics, usage, or performance trends. Example prompts:

​
Build

Turn findings into saved Braintrust objects. Loop asks for approval before creating or changing them.

​
Author prompts and evaluators

Loop can suggest, create, and save evaluators based on your application’s behavior. A scorer returns a number, and a classifier returns a label. New and updated evaluators open in the workspace for review. Loop can also create and update prompts, including revising playground prompts from your annotations on individual outputs. Example prompts:

​
Curate datasets

Loop can generate datasets from log patterns and edit existing rows. Edits open in a read-only workspace table that refreshes as Loop works. Select a row to inspect its trace inline. Loop can also manage dataset versions and the environments that point at them:
  • Snapshots. Save a snapshot before editing, or restore one to undo edits. Loop can list snapshots, their environment tags, and the current head version. Restoring uses the dataset UI’s confirmation flow and can save the current state first.
  • Environments. Point an environment, such as production or staging, at a snapshot or clear its mapping. Loop can list your organization’s environment slugs and create missing environments.
Restoring a dataset to a snapshot replaces its current rows, and retagging an environment changes which dataset state your applications resolve. Review what Loop proposes before you approve either action.
Example prompts:

​
Build views and dashboards

Loop can create and edit custom views for traces and dataset rows, and build dashboard charts. Open new or updated dashboards in the workspace to review them beside the thread. Example prompts:

​
Configure Topics

Give Loop a goal to configure the Topics pipeline:
  • Check the preprocessor. Loop tests your project’s effective default preprocessor on real traces for usable conversation text. If it returns nothing, Loop writes and tests a custom preprocessor for your trace shape and can make it the project default.
  • Confirm the scope. Loop looks for a grouping key, such as metadata.conversation_id, and recommends analyzing a span, trace, or group of related traces.
  • Validate the facets. Loop tests a facet on up to ten representative traces and shows results inline for your feedback. Reuse that trace set as you refine the prompt. For larger sets, Loop runs an eval.
  • Evaluate candidates. Loop compares candidate facet prompts in evals before saving them.
  • Enable the automation. Loop creates the facets, enables the Topics automation, and offers to process a historical window.
Changing the project default preprocessor affects every function that uses it, including built-in Topics facets. Loop asks for approval before changing it.
Rewinding a Topics automation processes historical traces and draws from your monthly model credits. Loop estimates the volume and asks for approval before it rewinds.
Example prompts:

​
Run and automate

Test changes with evaluations, then schedule recurring checks or alerts.

​
Run evaluations

Loop runs evaluations two ways:
  • In a playground or experiment. Loop can edit the task prompt or code, add or modify scorers, and re-run the evaluation to show the effect of a change.
  • As a new experiment. Loop can run an experiment from a dataset, from rows it supplies inline, or from a prior experiment, using a saved or inline task and any scorers you choose. When a prior experiment supplies the data, its outputs become the expected values.
Evaluation runs open in the workspace’s Experiments panel, where you can compare progress and scores without leaving the chat. Example prompts:
Loop runs evaluations in Braintrust rather than by writing an Eval() script. To keep evaluations in your own repository and run them in CI, see Run evaluations in code.

​
Manage automations and alerts

Turn an investigation into an alert or a scheduled check:
  • Alerts. Create an alert for matching logs, aggregates crossing a threshold over a recent window, or environment updates. Loop generates the filter from your description.
  • Automations. Create a scheduled Loop job with its own instructions and schedule. Loop can also list existing online scoring rules, alerts, exports, retention policies, and Topics automations with their full configurations.
  • Status. Pause an automation or make it active again.
  • Updates. Rewrite an alert or Loop automation under its existing name. Scheduled exports and retention policies are inspect-only.
Rewriting an automation replaces its entire configuration. Ask Loop to read the current configuration first, then review the proposed replacement before approving it.
You can also edit any automation directly in Settings > Automations. Example prompts:

​
Get help

Find documentation and objects, or contact support through Loop.

​
Search docs and find objects

Loop can search the Braintrust documentation, resolve object names and URLs, list recent objects, summarize an experiment’s metrics against a baseline, and generate shareable links to objects it creates. Example prompts:

​
Request support

Ask Loop for help, or type / and choose the support command, to file a ticket with Braintrust. Loop guides you through the request and includes relevant context from your page and conversation.

​
Next steps