nirbarazida/promptfoo

Michael D'Angelo 48551492c9

style: format redteam provider override example config file

1 year ago

Commits

amazon-bedrock

d967700f42

feat: add zod schema validation to promptfooconfig.yaml (#1185)

1 year ago

assistant-cli

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

azure-openai

96c6af17c9

feat: add logprobs support for azure openai (#376)

1 year ago

azure-openai-assistant

3014f7ebf0

chore: formatting

1 year ago

claude-vision

71dfc6a8f9

fix: support additional models via AWS Bedrock (#1034)

1 year ago

claude-vs-gpt

7ad60fde34

docs: update claude vs gpt example to latest models

1 year ago

claude-vs-gpt-image

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

cloudflare-ai

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

cohere

017210ecc3

feat(cohere): add support for embeddings api (#1502)

1 year ago

cohere-benchmark

d967700f42

feat: add zod schema validation to promptfooconfig.yaml (#1185)

1 year ago

colab-notebook

332c20943e

fix: reorder imports in promptfoo_example.ipynb

1 year ago

custom-grading-prompt

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

custom-prompt-function

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

custom-provider

cefe5a7d0b

chore(eslint): add and configure eslint-plugin-unicorn (#1489)

1 year ago

custom-provider-embeddings

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

dbrx-benchmark

3b791dc300

feat: first-class support for openrouter (#702)

1 year ago

docker-code-generation-sandbox

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

dynamic-var

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

extension-api

381d3eb98a

feat: extension hook api (#1249)

1 year ago

external-provider-config

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

gemma-vs-llama

3014f7ebf0

chore: formatting

1 year ago

gemma-vs-mistral

3014f7ebf0

chore: formatting

1 year ago

github-action

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

google-aistudio-gemini

3014f7ebf0

chore: formatting

1 year ago

google-sheets

c826bdcb02

fix: google sheets output (#1367)

1 year ago

google-vertex

7f3137d6f4

feat: add support for gemini embeddings via vertex (#1004)

1 year ago

gpt-4o-temperature-comparison

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

gpt-4o-vs-4o-mini

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

groq

3fdaa5af1b

chore(envars): add GROQ_API_KEY and alphabetize 3rd party env vars

1 year ago

http-provider

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

huggingface-hate-speech-detection

b0bc0984ee

feat(huggingface): Support sentence similarity inference API (#425)

1 year ago

huggingface-inference-endpoint

3014f7ebf0

chore: formatting

1 year ago

huggingface-pii

3014f7ebf0

chore: formatting

1 year ago

huggingface-similarity

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

ibm-granite-vs-llama

92830ecb92

fix: resolve issues with relative prompt paths (#1066)

1 year ago

image-classification

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

javascript-assert-external

ed319b695d

feat: add support for nested grading results (#1101)

1 year ago

jest-integration

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

js-config

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

json-output

3014f7ebf0

chore: formatting

1 year ago

langchain-python

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

llama-cpp

bbec5e9ed6

docs: add example and configuration guide for using llama.cpp (#1104)

1 year ago

llama-gpt-comparison

23e57def26

chore: update llama examples and add support for chat formatted prompts in Replicate (#1261)

1 year ago

lm-studio

6802ce0ad1

fix(http provider): fix JSON prompt escaping in HTTP provider and add LM Studio example (#1156)

1 year ago

mistral-llama-comparison

23e57def26

chore: update llama examples and add support for chat formatted prompts in Replicate (#1261)

1 year ago

moderation

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

multiple-configs

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

multiple-translations

5f25ab91fe

feat: ability to import vars using glob (#641)

1 year ago

multiple-translations-scenarios

95072d7abf

chore: prefer yaml array syntax in examples

1 year ago

multiple-turn-conversation

5480efa410

Add _conversation variable for testing multiple-turn chat conversations (#224)

1 year ago

multishot

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

named-metrics

270766004e

fix: derived metrics (#985)

1 year ago

node-package

6e4fdcd886

chore: sort imports (#1006)

1 year ago

node-package-typescript

ad33256d02

chore: enforce consistent type imports (#1341)

1 year ago

nunjucks-custom-filters

788a6b82e6

Add support for nunjucks filters (#243)

1 year ago

ollama-comparison

92830ecb92

fix: resolve issues with relative prompt paths (#1066)

1 year ago

openai-chat-history

9c103978dc

docs: example of escaping multiline var in nunjucks template

1 year ago

openai-dalle-images

a3679d8d27

docs: move dalle example

1 year ago

openai-eval-factuality

bb342e5d99

fix(factuality): make factuality output case-insensitive (#468)

1 year ago

openai-function-call

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

openai-multiline-yaml

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

openai-tools-call

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

openai-vision

580a00d56f

docs: expand on openai image examples

1 year ago

perplexity.ai-example

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

phi-vs-llama

806e52216f

chore(docs): add shell syntax highlighting and fix typos (#953)

1 year ago

portkey-test

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

prompt-labels

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

prompts-per-model

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

python-assert-external

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

python-assert-inline

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

python-provider

d2ed96dd71

chore: format python and add check job to GitHub Actions workflow (#1105)

1 year ago

rag-eval

09fbd07470

docs: Full rag example (#1228)

1 year ago

rag-full

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

redteam-dalle

c6bfcba9c3

docs: dalle jailbreak blog post (#1052)

1 year ago

redteam-provider-override

48551492c9

style: format redteam provider override example config file

1 year ago

replicate-lifeboat

e39d761cb5

chore: format

1 year ago

replicate-llama2

770e36370d

fix(replicate): support non-array outputs (#547)

1 year ago

select-best-example

3014f7ebf0

chore: formatting

1 year ago

self-grading

95072d7abf

chore: prefer yaml array syntax in examples

1 year ago

separate-test-configs

95072d7abf

chore: prefer yaml array syntax in examples

1 year ago

simple-cli

675b94f17a

feat: add support for `tags` property in config (#1526)

1 year ago

simple-csv

a36cd41b82

chore: update to prettier v3 (#901)

1 year ago

simple-test

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

sql-validation

09fbd07470

docs: Full rag example (#1228)

1 year ago

standalone-assertions

3014f7ebf0

chore: formatting

1 year ago

store-and-reuse-outputs

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

structured-outputs-config

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

summarization

09fbd07470

docs: Full rag example (#1228)

1 year ago

test-markdown

5c88b43ac0

docs: add markdown test. Related to #873

1 year ago

tool-use

cd8b724dbf

feat(groq): integrate native Groq SDK and update documentation (#1479)

1 year ago

transform-file

d9e79d506e

feat(transform): support custom function names in file transforms (#1435)

1 year ago

ts-config

adec5ba3a8

docs: add TypeScript promptfooconfig example (#1427)

1 year ago

vars-referencing-vars

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

voyage-embeddings

a22257f01e

feat: add Voyage AI embeddings provider (#931)

1 year ago

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida / promptfoo mirror of https://github.com/promptfoo/promptfoo

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida
/
promptfoo
mirror of https://github.com/promptfoo/promptfoo