prompt_evaluations

This skill provides a comprehensive course on prompt evaluations using the Anthropic API. It covers various evaluation techniques, including human-graded evals, code-graded evals, and model-graded evals. The course also introduces Promptfoo, a tool for streamlining and managing prompt evaluations.

The skill guides users through writing different types of evaluations, such as classification evals and custom graders. It also demonstrates how to use Promptfoo for model-graded evals and custom model-graded evals. By completing this course, users will gain the knowledge and skills necessary to effectively evaluate and improve their prompts.

The lessons are designed to build upon each other, starting with an introduction to evaluations and progressing to more advanced topics. Each lesson includes practical examples and exercises to reinforce learning.

What It Does

Provides a comprehensive course on prompt evaluations, teaching users how to implement various evaluation techniques with the Anthropic API and Promptfoo.

When To Use

When you need to evaluate and improve the performance of prompts used with the Anthropic API, ensuring accuracy, reliability, and desired outcomes.

Installation

Copy SKILL.md to your skills directory