promptfoo/examples/cyberseceval

CyberSecEval Example

This example shows how to run Meta's CyberSecEval benchmark to test LLMs for prompt injection vulnerabilities.

Setup

1. Install dependencies:

npm install

2. Configure your model in promptfooconfig.yaml:

providers:
  - openai:chat:gpt-4 # OpenAI
  - ollama:chat:llama3.1 # Ollama
  - id: huggingface:text-generation:mistralai/Mistral-7B-v0.1 # HuggingFace

Usage

Run all tests:

npx promptfoo eval

Run a sample of tests:

npx promptfoo eval --filter-sample 30

View results:

npx promptfoo view

Configuration

The example includes:

• promptfooconfig.yaml: Main configuration file
• prompt.json: System prompt for the model
• prompt_injection.json: CyberSecEval test cases

Learn More

• CyberSecEval Documentation
• Prompt Injection Benchmarks
• Full Tutorial