nirbarazida/promptfoo

AI Simply Explained c97a6d8812

feat(providers): add grok-4 support (#4855)

1 month ago

Commits

adaline-gateway

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

amazon-bedrock

265ffe6dae

docs: Improves `contextTransform` documentation (#4854)

1 month ago

amazon-sagemaker

baa49348f6

chore(providers): remove unused provider from AWS SageMaker AI (#4667)

1 month ago

assertion-scoring-override

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

assertions-generate

a796527295

feat: assertion generation (#4559)

2 months ago

assistant-cli

ff7b5cc61a

chore(eval): remove redundant 'Done.' message from evaluation output (#4810)

1 month ago

azure-openai

7537bea3cc

docs(examples): add Azure OpenAI vision example (#4806)

1 month ago

azure-openai-assistant

0d3f7b0d33

chore(providers): add thread id to context arg for openai and azure assistant function call (#2263)

1 month ago

cerebras

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

claude-thinking

7857266c65

chore(providers): add Claude 4 support to anthropic, bedrock, and vertex providers (#4129)

3 months ago

claude-vision

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

claude-vs-gpt

7857266c65

chore(providers): add Claude 4 support to anthropic, bedrock, and vertex providers (#4129)

3 months ago

claude-vs-gpt-image

becfa34b34

docs: standardize file references to use file:// scheme (#4291)

2 months ago

cloudflare-ai

cbe7d1f9ac

chore(providers): convert Cloudflare AI to use OpenAI-compatible endpoints (#4683)

1 month ago

cohere

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

cohere-benchmark

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

colab-notebook

5b3fcfb728

chore: Add README for missing examples (#4404)

2 months ago

csv-metadata

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

custom-grader-csv

f165990574

docs(examples): fix custom-grader-csv README inconsistencies (#4474)

2 months ago

custom-grading-prompt

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

custom-prompt-function

becfa34b34

docs: standardize file references to use file:// scheme (#4291)

2 months ago

custom-provider

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

custom-provider-embeddings

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

custom-provider-mjs

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

custom-provider-typescript

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

cyberseceval

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

databricks

3cdff2b898

docs(providers): update available databricks models (#4887)

1 month ago

dbrx-benchmark

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

deepseek-r1-vs-openai-o1

1f15e05d88

docs: add schema header to example configs (#4277)

2 months ago

docker-code-generation-sandbox

98488d06c3

chore: bump urllib3 from 1.26.19 to 2.5.0 in /examples/docker-code-generation-sandbox in the pip group across 1 directory (#4556)

2 months ago

donotanswer

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

dynamic-var

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

errors-vs-failures

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

extension-api

4cbf0cbf30

chore(eval): Adds first-class support for `beforeAll` and `beforeEach` extension hooks mutation of context (#4197)

2 months ago

external-defaulttest

3e5d31a14f

feat(config): add support for loading defaultTest from external files (#4720)

1 month ago

external-provider-config

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

f-score

c514791528

chore(deps): update all example dependencies to latest versions (#4900)

1 month ago

fal-image-generation

5e3223a2cb

chore(providers): update fal provider (#4182)

3 months ago

function-tools-callback

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

g-eval

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

gemma-vs-llama

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

gemma-vs-mistral

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

getting-started

a0d2d07630

chore: update experimental Gemini model IDs to stable versions (#4894)

1 month ago

github-action

5b3fcfb728

chore: Add README for missing examples (#4404)

2 months ago

golang-provider

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

google-aistudio-gemini

3ed9e83191

feat(providers): Image understanding for Google providers (#4767)

1 month ago

google-aistudio-tools

3ed9e83191

feat(providers): Image understanding for Google providers (#4767)

1 month ago

google-live

becfa34b34

docs: standardize file references to use file:// scheme (#4291)

2 months ago

google-live-audio

8dd32a6feb

feat: Google live audio output (#4280)

2 months ago

google-sheets

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

google-vertex

a0d2d07630

chore: update experimental Gemini model IDs to stable versions (#4894)

1 month ago

google-vertex-tools

b5f0766391

chore(deps): update minor and patch dependencies (#4686)

1 month ago

gpt-4o-temperature-comparison

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

gpt-4o-vs-4o-mini

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

groq

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

harmbench

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

headless-browser

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

helicone

43f6a9302e

feat(providers): add Helicone AI Gateway provider (#4662)

1 month ago

http-provider

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

http-provider-auth-signature

c514791528

chore(deps): update all example dependencies to latest versions (#4900)

1 month ago

huggingface-dataset

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

huggingface-dataset-factuality

c592fdb629

feat(tests): add configuration support for test generators (#4301)

2 months ago

huggingface-hate-speech-detection

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

huggingface-hle

a072e4613d

docs(site): add guide on humanity's last exam (#4694)

1 month ago

huggingface-inference-endpoint

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

huggingface-pii

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

huggingface-similarity

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

hyperbolic

1b0e8b4c87

feat(providers): add support for hyperbolic image and audio providers (#4260)

2 months ago

ibm-granite-vs-llama

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

image-classification

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

javascript-assert-external

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

javascript-test-cases

c592fdb629

feat(tests): add configuration support for test generators (#4301)

2 months ago

jest-integration

c514791528

chore(deps): update all example dependencies to latest versions (#4900)

1 month ago

js-config

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

json-output

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

jsonl-test-cases

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

lambdalabs

598e71c089

feat(providers): add Lambda Labs integration (#3601)

4 months ago

langchain-python

c514791528

chore(deps): update all example dependencies to latest versions (#4900)

1 month ago

langfuse-labels

02d4023164

feat: add Langfuse prompt label support with improved parsing (#4847)

1 month ago

litellm

2627f13476

docs(litellm): fix example with modern model IDs and proper embedding config (#4885)

1 month ago

llama-cpp

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

llama-gpt-comparison

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

lm-studio

b5f0766391

chore(deps): update minor and patch dependencies (#4686)

1 month ago

mistral

acd51ac2bc

feat(providers): add Mistral Magistral reasoning models (#4435)

2 months ago

mistral-llama-comparison

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

moderation

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

multiple-configs

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

multiple-translations

193cf868e6

docs(examples): consolidate translation examples (#4590)

2 months ago

multiple-turn-conversation

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

multishot

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

named-metrics

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

node-module-package

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

node-package

5b3fcfb728

chore: Add README for missing examples (#4404)

2 months ago

node-package-typescript

5b3fcfb728

chore: Add README for missing examples (#4404)

2 months ago

nunjucks-custom-filters

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

ollama-comparison

becfa34b34

docs: standardize file references to use file:// scheme (#4291)

2 months ago

openai-agents

c514791528

chore(deps): update all example dependencies to latest versions (#4900)

1 month ago

openai-audio

360d7b519d

docs(examples): add uniform init commands to all example READMEs (#4068)

3 months ago

openai-chat-history

fc4a6bbc7d

docs(examples): update model references from gpt-4o-mini to gpt-4.1-mini (#4289)

2 months ago

openai-dalle-images

874d93b56e

docs: Update example configs with descriptions (#4283)

2 months ago

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida / promptfoo mirror of https://github.com/promptfoo/promptfoo

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida
/
promptfoo
mirror of https://github.com/promptfoo/promptfoo