nirbarazida/promptfoo

vedantr 1de25ad380

chore: minor change to python assert example and revert provider to gpt4 mini (#2564)

7 months ago

Commits

adaline-chat-history

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

adaline-embedding-similarity

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

adaline-eval-factuality

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

adaline-multi-provider

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

adaline-openai-format

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

adaline-structured-output

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

adaline-tool-call

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

adaline-vision

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

adaline-vision-base64

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

amazon-bedrock

f14f76e927

fix(providers): add regional model support to bedrock (#2354)

8 months ago

assistant-cli

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

azure-openai

9462f7d56c

chore: azureopenai -> azure (#2113)

9 months ago

azure-openai-assistant

9462f7d56c

chore: azureopenai -> azure (#2113)

9 months ago

claude-vision

f14f76e927

fix(providers): add regional model support to bedrock (#2354)

8 months ago

claude-vs-gpt

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

claude-vs-gpt-image

6caea9c090

fix(providers): handle system messages correctly for bedrock Claude models (#2141)

9 months ago

cloudflare-ai

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

cohere

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

cohere-benchmark

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

colab-notebook

332c20943e

fix: reorder imports in promptfoo_example.ipynb

1 year ago

custom-grading-prompt

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

custom-prompt-function

44415863af

fix: typescript for prompt functions (#2287)

8 months ago

custom-provider

e41f885740

feat: support typescript custom providers (#2285)

8 months ago

custom-provider-embeddings

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

custom-provider-mjs

e41f885740

feat: support typescript custom providers (#2285)

8 months ago

custom-provider-typescript

e41f885740

feat: support typescript custom providers (#2285)

8 months ago

cyberseceval

866a6876d2

chore: add english language cyberseceval (#2561)

7 months ago

dbrx-benchmark

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

docker-code-generation-sandbox

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

dynamic-var

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

errors-vs-failures

8afa0ffe8b

chore: separate errors from assert failures (#2214)

8 months ago

extension-api

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

external-provider-config

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

f-score

4426dc8290

docs: add F-score example (#2198)

9 months ago

function-tools-callback

f2d23ac32f

fix(providers): openai function tool callbacks handling (#1648)

11 months ago

g-eval

e822ec6a6e

feat: add G-Eval assertion (#2436)

8 months ago

gemma-vs-llama

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

gemma-vs-mistral

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

github-action

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

golang-provider

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

google-aistudio-gemini

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

google-sheets

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

google-vertex

ab0c382085

fix(gemini): support gemini thinking model (#2526)

7 months ago

gpt-4o-temperature-comparison

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

gpt-4o-vs-4o-mini

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

groq

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

headless-browser

841a40dcda

feat(providers): add transformRequest to HTTP provider. Rename responseParser to transformResponse (#2228)

8 months ago

http-provider

ded1f3ef7c

feat: add HTTP provider configuration generator (#2409)

8 months ago

huggingface-dataset

f77a390ef9

feat: add support for huggingface datasets (#2497)

8 months ago

huggingface-hate-speech-detection

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

huggingface-inference-endpoint

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

huggingface-pii

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

huggingface-similarity

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

ibm-granite-vs-llama

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

image-classification

e1b44fbd73

chore(examples): modernize image classification example (#2197)

9 months ago

javascript-assert-external

1ca234eb25

feat(assertions): add support for specifying function names in external assertions (#2548)

7 months ago

jest-integration

e88e2aa5e5

chore(deps-dev): bump vite from 5.3.3 to 5.4.9 in /examples/jest-integration (#1960)

10 months ago

js-config

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

json-output

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

langchain-python

305a5e62be

chore(deps): bump langchain from 0.1.14 to 0.3.0 in /examples/langchain-python (#2035)

9 months ago

llama-cpp

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

llama-gpt-comparison

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

lm-studio

841a40dcda

feat(providers): add transformRequest to HTTP provider. Rename responseParser to transformResponse (#2228)

8 months ago

mistral

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

mistral-llama-comparison

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

moderation

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

multiple-configs

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

multiple-translations

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

multiple-translations-scenarios

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

multiple-turn-conversation

a397e11e98

feat(eval): support more than one multi-turn conversations in the same eval with conversationId meta field (#2360)

8 months ago

multishot

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

named-metrics

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

node-module-package

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

node-package

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

node-package-typescript

ad33256d02

chore: enforce consistent type imports (#1341)

1 year ago

nunjucks-custom-filters

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

ollama-comparison

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-chat-history

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-dalle-images

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-eval-factuality

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-function-call

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-multiline-yaml

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-structured-output

cbebe2026f

chore(providers): add openai gpt-4o-2024-11-20 to supported models (#2203)

9 months ago

openai-tools-call

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

openai-vision

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

pdf-variables

778cbf4943

feat: read variables from PDF (#2218)

9 months ago

perplexity.ai-example

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

phi-vs-llama

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

portkey-test

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

prompt-labels

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

prompts-per-model

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

python-assert-external

1de25ad380

chore: minor change to python assert example and revert provider to gpt4 mini (#2564)

7 months ago

python-assert-inline

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

python-provider

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

rag-eval

76797692e5

feat(schema): add yaml schema validation to config files (#1990)

10 months ago

rag-full

665242dda8

chore(examples): add yaml schema and descriptions to config files (#2358)

8 months ago

redteam-beavertails

f77a390ef9

feat: add support for huggingface datasets (#2497)

8 months ago

redteam-bestOfN-strategy

91a8a129a8

feat: best-of-n jailbreak (#2495)

8 months ago

redteam-chatbot

3983f18be6

chore(redteam): respect --delay with redteam providers (#2369)

8 months ago

redteam-custom-strategy

532da105ee

feat(redteam): custom strategy (#2166)

9 months ago

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida / promptfoo mirror of https://github.com/promptfoo/promptfoo

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida
/
promptfoo
mirror of https://github.com/promptfoo/promptfoo