nirbarazida/promptfoo

Michael 673e1431a8

chore(prompts): support j2 files (#3338)

5 months ago

Commits

adaline-gateway

95727630df

chore(providers): make adaline a peer dependency (#2833)

7 months ago

amazon-bedrock

be8378f864

feat: base64 loader for images (#3262)

5 months ago

assertion-scoring-override

42ab66b096

feat(assertions): add custom assertion scoring functions (#3142)

6 months ago

assistant-cli

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

azure-openai

c6024f512f

chore(providers): improve support for azure reasoning models and update documentation (#3332)

5 months ago

azure-openai-assistant

9462f7d56c

chore: azureopenai -> azure (#2113)

9 months ago

claude-thinking

ccbbe46727

chore: add cursor AI rules for development workflow (#3326)

5 months ago

claude-vision

f14f76e927

fix(providers): add regional model support to bedrock (#2354)

8 months ago

claude-vs-gpt

264844483c

chore(providers): add support for Anthropic Claude 3.7 Sonnet model (#3202)

6 months ago

claude-vs-gpt-image

6caea9c090

fix(providers): handle system messages correctly for bedrock Claude models (#2141)

9 months ago

cloudflare-ai

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

cohere

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

cohere-benchmark

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

colab-notebook

6bbaf876e7

docs(press): add dedicated press page (#2990)

6 months ago

custom-grader-csv

eb47d2d8a8

fix: support javascript files in csv assertions file:// protocol (#2723)

7 months ago

custom-grading-prompt

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

custom-prompt-function

673e1431a8

chore(prompts): support j2 files (#3338)

5 months ago

custom-provider

e41f885740

feat: support typescript custom providers (#2285)

8 months ago

custom-provider-embeddings

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

custom-provider-mjs

e41f885740

feat: support typescript custom providers (#2285)

8 months ago

custom-provider-typescript

e41f885740

feat: support typescript custom providers (#2285)

8 months ago

cyberseceval

866a6876d2

chore: add english language cyberseceval (#2561)

7 months ago

dbrx-benchmark

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

deepseek-r1-vs-openai-o1

7ef3b75451

chore(testCases): add HuggingFace Hub token support for dataset access (#3063)

6 months ago

docker-code-generation-sandbox

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

dynamic-var

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

errors-vs-failures

8afa0ffe8b

chore: separate errors from assert failures (#2214)

8 months ago

extension-api

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

external-provider-config

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

f-score

4426dc8290

docs: add F-score example (#2198)

9 months ago

function-tools-callback

f2d23ac32f

fix(providers): openai function tool callbacks handling (#1648)

11 months ago

g-eval

e822ec6a6e

feat: add G-Eval assertion (#2436)

8 months ago

gemma-vs-llama

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

gemma-vs-mistral

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

getting-started

0e6f688c63

docs: improve getting started guide (#3065)

6 months ago

github-action

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

golang-provider

b69af66a3a

fix(go-mod): update go toolchain version to valid syntax (#3170)

6 months ago

google-aistudio-gemini

599111ce42

chore(providers): update google provider with better support for latest gemini models (#2838)

6 months ago

google-multimodal-live

055cc08d03

feat: Google Multimodal Live provider with 2 examples (#3270)

5 months ago

google-multimodal-live-tools

055cc08d03

feat: Google Multimodal Live provider with 2 examples (#3270)

5 months ago

google-sheets

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

google-vertex

8fe1cc593c

chore(providers): add support for Vertex llama + update documentation (#3107)

5 months ago

google-vertex-tools

8fe1cc593c

chore(providers): add support for Vertex llama + update documentation (#3107)

5 months ago

google-vertex-tools-callback

e4b232af24

feat(providers): add support for execution of function/tool callbacks in Vertex provider (#3215)

5 months ago

gpt-4o-temperature-comparison

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

gpt-4o-vs-4o-mini

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

groq

f9b389314c

chore(providers): update Groq documentation with latest models (#2733)

7 months ago

harmbench

6bbaf876e7

docs(press): add dedicated press page (#2990)

6 months ago

headless-browser

841a40dcda

feat(providers): add transformRequest to HTTP provider. Rename responseParser to transformResponse (#2228)

8 months ago

http-provider

ded1f3ef7c

feat: add HTTP provider configuration generator (#2409)

8 months ago

http-provider-auth-signature

482cae38b7

chore: improve http signature setup (#2779)

7 months ago

huggingface-dataset

f77a390ef9

feat: add support for huggingface datasets (#2497)

8 months ago

huggingface-hate-speech-detection

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

huggingface-inference-endpoint

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

huggingface-pii

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

huggingface-similarity

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

ibm-granite-vs-llama

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

image-classification

e1b44fbd73

chore(examples): modernize image classification example (#2197)

9 months ago

javascript-assert-external

1ca234eb25

feat(assertions): add support for specifying function names in external assertions (#2548)

7 months ago

javascript-test-cases

ab2de8cce4

feat(testcases): add support for loading dynamic test cases from Python and Javascript/Typescript files (#2993)

6 months ago

jest-integration

5fb89d63f2

chore(deps): update vitest to resolve CVE issues (#3016)

6 months ago

js-config

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

json-output

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

langchain-python

305a5e62be

chore(deps): bump langchain from 0.1.14 to 0.3.0 in /examples/langchain-python (#2035)

9 months ago

llama-cpp

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

llama-gpt-comparison

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

lm-studio

841a40dcda

feat(providers): add transformRequest to HTTP provider. Rename responseParser to transformResponse (#2228)

8 months ago

mistral

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

mistral-llama-comparison

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

moderation

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

multiple-configs

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

multiple-translations

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

multiple-translations-scenarios

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

multiple-turn-conversation

a397e11e98

feat(eval): support more than one multi-turn conversations in the same eval with conversationId meta field (#2360)

8 months ago

multishot

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

named-metrics

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

node-module-package

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

node-package

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

node-package-typescript

ad33256d02

chore: enforce consistent type imports (#1341)

1 year ago

nunjucks-custom-filters

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

ollama-comparison

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-audio

90f99d8a7b

feat(providers): add support for gpt-4o-audio-preview (#3302)

5 months ago

openai-chat-history

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-dalle-images

eecb2d4327

chore(providers): refactor OpenAI image provider to remove OpenAI Node SDK dependency (#3245)

5 months ago

openai-eval-factuality

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-function-call

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-multiline-yaml

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-structured-output

cbebe2026f

chore(providers): add openai gpt-4o-2024-11-20 to supported models (#2203)

9 months ago

openai-tools-call

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-vision

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

pdf-variables

dc5f3ad25f

chore(ci): add shell format check to CI workflow (#2669)

7 months ago

perplexity.ai-example

8fbc014752

chore(providers): enhance perplexity documentation and add alias (#2836)

6 months ago

phi-vs-llama

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

portkey-test

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

prompt-labels

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

prompts-per-model

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

python-assert-external

875b4813d9

chore(assertions): improve python assertion configuration passing (#2583)

7 months ago

python-assert-inline

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

python-provider

1491fc5d45

docs: ruff formatting

7 months ago

python-test-cases

ab2de8cce4

feat(testcases): add support for loading dynamic test cases from Python and Javascript/Typescript files (#2993)

6 months ago

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida / promptfoo mirror of https://github.com/promptfoo/promptfoo

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida
/
promptfoo
mirror of https://github.com/promptfoo/promptfoo