nirbarazida/promptfoo

Michael fdcaa99fdb

chore(providers): add support for file-based response parser for http provider (#1808)

10 months ago

Commits

amazon-bedrock

6af803b236

chore(bedrock): improve support for LLAMA3.1 and LLAMA3.2 model configuration (#1777)

11 months ago

assistant-cli

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

azure-openai

96c6af17c9

feat: add logprobs support for azure openai (#376)

1 year ago

azure-openai-assistant

3014f7ebf0

chore: formatting

1 year ago

claude-vision

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

claude-vs-gpt

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

claude-vs-gpt-image

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

cloudflare-ai

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

cohere

017210ecc3

feat(cohere): add support for embeddings api (#1502)

1 year ago

cohere-benchmark

cee712c92e

docs: update default LLM models

11 months ago

colab-notebook

332c20943e

fix: reorder imports in promptfoo_example.ipynb

1 year ago

custom-grading-prompt

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

custom-prompt-function

d2efa190d5

feat: prompts as python classmethods (#1799)

10 months ago

custom-provider

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

custom-provider-embeddings

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

dbrx-benchmark

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

docker-code-generation-sandbox

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

dynamic-var

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

extension-api

a6c2affdb5

docs: fix typo in hooks documentation

11 months ago

external-provider-config

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

function-tools-callback

f2d23ac32f

fix(providers): openai function tool callbacks handling (#1648)

11 months ago

gemma-vs-llama

3014f7ebf0

chore: formatting

1 year ago

gemma-vs-mistral

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

github-action

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

golang-provider

1409a80a5e

docs: update golang example

11 months ago

google-aistudio-gemini

3014f7ebf0

chore: formatting

1 year ago

google-sheets

c826bdcb02

fix: google sheets output (#1367)

1 year ago

google-vertex

7f3137d6f4

feat: add support for gemini embeddings via vertex (#1004)

1 year ago

gpt-4o-temperature-comparison

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

gpt-4o-vs-4o-mini

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

groq

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

headless-browser

9fcc9f5974

fix: browser provider ignores cert errors

11 months ago

http-provider

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

huggingface-hate-speech-detection

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

huggingface-inference-endpoint

3014f7ebf0

chore: formatting

1 year ago

huggingface-pii

cee712c92e

docs: update default LLM models

11 months ago

huggingface-similarity

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

ibm-granite-vs-llama

92830ecb92

fix: resolve issues with relative prompt paths (#1066)

1 year ago

image-classification

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

javascript-assert-external

1b55a15e80

chore: support config object in javascript and python assertions (#1729)

11 months ago

jest-integration

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

js-config

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

json-output

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

langchain-python

cee712c92e

docs: update default LLM models

11 months ago

llama-cpp

bbec5e9ed6

docs: add example and configuration guide for using llama.cpp (#1104)

1 year ago

llama-gpt-comparison

23e57def26

chore: update llama examples and add support for chat formatted prompts in Replicate (#1261)

1 year ago

lm-studio

fdcaa99fdb

chore(providers): add support for file-based response parser for http provider (#1808)

10 months ago

mistral

091cfd5847

feat(mistral): update chat models and add embedding provider (#1614)

11 months ago

mistral-llama-comparison

23e57def26

chore: update llama examples and add support for chat formatted prompts in Replicate (#1261)

1 year ago

moderation

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

multiple-configs

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

multiple-translations

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

multiple-translations-scenarios

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

multiple-turn-conversation

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

multishot

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

named-metrics

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

node-package

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

node-package-typescript

ad33256d02

chore: enforce consistent type imports (#1341)

1 year ago

nunjucks-custom-filters

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

ollama-comparison

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

openai-chat-history

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

openai-dalle-images

a3679d8d27

docs: move dalle example

1 year ago

openai-eval-factuality

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

openai-function-call

333e61958f

fix(providers): fix error in loading openai externally defined function calls

11 months ago

openai-multiline-yaml

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

openai-structured-output

cd48ec371b

chore(openai): improve support for structured outputs (#1656)

11 months ago

openai-tools-call

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

openai-vision

6acb66d710

feat: support input transforms (#1704)

11 months ago

perplexity.ai-example

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

phi-vs-llama

806e52216f

chore(docs): add shell syntax highlighting and fix typos (#953)

1 year ago

portkey-test

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

prompt-labels

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

prompts-per-model

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

python-assert-external

cee712c92e

docs: update default LLM models

11 months ago

python-assert-inline

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

python-provider

8e6f4eed10

chore(providers): support file:// syntax for Python providers (#1748)

11 months ago

rag-eval

09fbd07470

docs: Full rag example (#1228)

1 year ago

rag-full

8e6f4eed10

chore(providers): support file:// syntax for Python providers (#1748)

11 months ago

redteam-dalle

e96985c9d6

chore: alias eval in redteam namespace and prefer redteam.yaml over promptfooconfig.yaml (#1664)

11 months ago

redteam-provider-override

e96985c9d6

chore: alias eval in redteam namespace and prefer redteam.yaml over promptfooconfig.yaml (#1664)

11 months ago

replicate-lifeboat

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

replicate-llama2

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

select-best-example

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

self-grading

cee712c92e

docs: update default LLM models

11 months ago

separate-test-configs

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

simple-cli

c0caa2f4ba

docs: fix simple test

11 months ago

simple-csv

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

simple-test

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

sql-validation

09fbd07470

docs: Full rag example (#1228)

1 year ago

standalone-assertions

3014f7ebf0

chore: formatting

1 year ago

store-and-reuse-outputs

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

structured-outputs-config

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

summarization

09fbd07470

docs: Full rag example (#1228)

1 year ago

test-markdown

5c88b43ac0

docs: add markdown test. Related to #873

1 year ago

tool-use

cee712c92e

docs: update default LLM models

11 months ago

transform-file

d9e79d506e

feat(transform): support custom function names in file transforms (#1435)

1 year ago

ts-config

be1cb847b5

chore: update ts-config example README with tsx loader options

10 months ago

vars-referencing-vars

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

voyage-embeddings

a22257f01e

feat: add Voyage AI embeddings provider (#931)

1 year ago

websockets

9b2b268013

feat: add websocket provider (#1712)

11 months ago

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida / promptfoo mirror of https://github.com/promptfoo/promptfoo

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida
/
promptfoo
mirror of https://github.com/promptfoo/promptfoo