Engineers

Setting up evals

1. 01 Bring your files Import eval results from promptfoo, Inspect, lm-eval or OpenAI

   Also opens Convert the file to another format

2. 02 Check the results file for errors Eval results auditor: find errored, empty and unreadable rows
3. 03 What went wrong, case by case Error analysis workbench for LLM traces and eval results

   Also opens Your first error-analysis session

4. 04 Pick a grader and try it Find your evaluator and metric

   Also opens Code grader playground: score one expected and actual pair

5. 05 Can I trust my judge? Write the validation report

In production

1. 06 Is this difference real? Same items, joined on id: see what changed
2. 07 Wire evals into CI Write a CI eval gate with three verdicts and a false-block forecast
3. 08 Your system uses an AI agent Evals when the system is an agent