nirbarazida/promptfoo

dependabot[bot] 80c0431731

chore: bump pypdf from 5.7.0 to 6.0.0 in /examples/rag-full in the pip group across 1 directory (#5252)

1 week ago

Commits

adaline-gateway

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

amazon-bedrock

265ffe6dae

docs: Improves `contextTransform` documentation (#4854)

1 month ago

amazon-sagemaker

61b7c246ba

docs: Guide for Evaluating LangGraph Agents with Promptfoo (#4926)

2 weeks ago

assertion-scoring-override

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

assertions-generate

a796527295

feat: assertion generation (#4559)

2 months ago

assistant-cli

ff7b5cc61a

chore(eval): remove redundant 'Done.' message from evaluation output (#4810)

1 month ago

azure-openai

7537bea3cc

docs(examples): add Azure OpenAI vision example (#4806)

1 month ago

azure-openai-assistant

0d3f7b0d33

chore(providers): add thread id to context arg for openai and azure assistant function call (#2263)

1 month ago

bert-score

ee85b4a415

docs: add bert-score example (#5091)

3 weeks ago

browser-existing-session

d02bbfffea

feat: add support for connecting to existing Chrome browser sessions (#5069)

3 weeks ago

cerebras

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

claude-thinking

7857266c65

chore(providers): add Claude 4 support to anthropic, bedrock, and vertex providers (#4129)

2 months ago

claude-vision

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

claude-vs-gpt

7857266c65

chore(providers): add Claude 4 support to anthropic, bedrock, and vertex providers (#4129)

2 months ago

claude-vs-gpt-image

abf1612226

docs(providers): remove deprecated claude-3-sonnet-20240229 model references (#5018)

1 month ago

cloudflare-ai

cbe7d1f9ac

chore(providers): convert Cloudflare AI to use OpenAI-compatible endpoints (#4683)

1 month ago

cohere

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

cohere-benchmark

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

colab-notebook

5b3fcfb728

chore: Add README for missing examples (#4404)

2 months ago

conversation-relevance

6b7b0d228e

feat(assertions): add conversational relevancy metric (#2130)

3 weeks ago

crewai

61b7c246ba

docs: Guide for Evaluating LangGraph Agents with Promptfoo (#4926)

2 weeks ago

csv-metadata

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

custom-grader-csv

f165990574

docs(examples): fix custom-grader-csv README inconsistencies (#4474)

2 months ago

custom-grading-prompt

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

custom-prompt-function

becfa34b34

docs: standardize file references to use file:// scheme (#4291)

2 months ago

custom-provider

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

custom-provider-embeddings

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

custom-provider-mjs

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

custom-provider-typescript

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

cyberseceval

6ace43376b

docs: update Ollama documentation with latest models and defaultTest guidance (#5084)

3 weeks ago

databricks

3cdff2b898

docs(providers): update available databricks models (#4887)

1 month ago

dbrx-benchmark

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

deepseek-r1-vs-openai-o1

1f15e05d88

docs: add schema header to example configs (#4277)

2 months ago

docker

0ed7a5299b

feat(providers): Add support for Docker Model Runner provider (#5081)

3 weeks ago

docker-code-generation-sandbox

98488d06c3

chore: bump urllib3 from 1.26.19 to 2.5.0 in /examples/docker-code-generation-sandbox in the pip group across 1 directory (#4556)

2 months ago

donotanswer

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

dynamic-var

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

errors-vs-failures

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

extension-api

70385eb0be

fix: sessionid(s) in extension hooks (#5053)

4 weeks ago

external-defaulttest

3e5d31a14f

feat(config): add support for loading defaultTest from external files (#4720)

1 month ago

external-provider-config

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

f-score

c514791528

chore(deps): update all example dependencies to latest versions (#4900)

1 month ago

fal-image-generation

5e3223a2cb

chore(providers): update fal provider (#4182)

2 months ago

function-tools-callback

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

g-eval

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

gemma-vs-llama

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

gemma-vs-mistral

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

getting-started

dfaf4d1575

feat: GPT-5 support (#5205)

2 weeks ago

github-action

93a93af382

docs: update GitHub Actions to v4 across documentation and examples (#5008)

4 weeks ago

golang-provider

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

google-aistudio-gemini

0247f4672f

docs: Add examples for gemini URL context and code execution tools (#4923)

1 month ago

google-aistudio-tools

0247f4672f

docs: Add examples for gemini URL context and code execution tools (#4923)

1 month ago

google-imagen

cd6973b051

feat: add Google Imagen image generation support (#5104)

3 weeks ago

google-live

becfa34b34

docs: standardize file references to use file:// scheme (#4291)

2 months ago

google-live-audio

8dd32a6feb

feat: Google live audio output (#4280)

2 months ago

google-sheets

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

google-vertex

0247f4672f

docs: Add examples for gemini URL context and code execution tools (#4923)

1 month ago

google-vertex-tools

b5f0766391

chore(deps): update minor and patch dependencies (#4686)

1 month ago

gpt-4o-temperature-comparison

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

gpt-4o-vs-4o-mini

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

grok-4-political-bias

1a29cacf42

docs(blog): grok-4 political bias post (#4953)

1 month ago

groq

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

harmbench

6ace43376b

docs: update Ollama documentation with latest models and defaultTest guidance (#5084)

3 weeks ago

headless-browser

8ed866c6ce

docs(browser): improve browser provider documentation and examples (#5030)

1 month ago

helicone

43f6a9302e

feat(providers): add Helicone AI Gateway provider (#4662)

1 month ago

http-provider

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

http-provider-auth-signature

3e3b7539b6

chore: update dependencies to latest minor and patch versions (#5109)

3 weeks ago

http-provider-auth-signature-jks

3e3b7539b6

chore: update dependencies to latest minor and patch versions (#5109)

3 weeks ago

http-provider-auth-signature-pfx

3e3b7539b6

chore: update dependencies to latest minor and patch versions (#5109)

3 weeks ago

huggingface-dataset

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

huggingface-dataset-factuality

c592fdb629

feat(tests): add configuration support for test generators (#4301)

2 months ago

huggingface-hate-speech-detection

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

huggingface-hle

a072e4613d

docs(site): add guide on humanity's last exam (#4694)

1 month ago

huggingface-inference-endpoint

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

huggingface-pii

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

huggingface-similarity

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

hyperbolic

1b0e8b4c87

feat(providers): add support for hyperbolic image and audio providers (#4260)

2 months ago

ibm-granite-vs-llama

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

image-classification

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

javascript-assert-external

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

javascript-test-cases

c592fdb629

feat(tests): add configuration support for test generators (#4301)

2 months ago

jest-integration

c514791528

chore(deps): update all example dependencies to latest versions (#4900)

1 month ago

js-config

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

json-output

00089bdcf6

fix: Fix json-output example (#5213)

2 weeks ago

jsonl-test-cases

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

lambdalabs

598e71c089

feat(providers): add Lambda Labs integration (#3601)

4 months ago

langchain-python

c514791528

chore(deps): update all example dependencies to latest versions (#4900)

1 month ago

langfuse-labels

02d4023164

feat: add Langfuse prompt label support with improved parsing (#4847)

1 month ago

langgraph

61b7c246ba

docs: Guide for Evaluating LangGraph Agents with Promptfoo (#4926)

2 weeks ago

litellm

2627f13476

docs(litellm): fix example with modern model IDs and proper embedding config (#4885)

1 month ago

llama-cpp

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

llama-gpt-comparison

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

lm-studio

b5f0766391

chore(deps): update minor and patch dependencies (#4686)

1 month ago

max-score-selection

3abc6b0ad2

feat: add max-score assertion for objective output selection (#5067)

3 weeks ago

mistral

acd51ac2bc

feat(providers): add Mistral Magistral reasoning models (#4435)

2 months ago

mistral-llama-comparison

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

moderation

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

multiple-configs

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

multiple-translations

193cf868e6

docs(examples): consolidate translation examples (#4590)

2 months ago

multiple-turn-conversation

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida / promptfoo mirror of https://github.com/promptfoo/promptfoo

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida
/
promptfoo
mirror of https://github.com/promptfoo/promptfoo