nirbarazida/promptfoo

Michael D'Angelo 333e61958f

fix(providers): fix error in loading openai externally defined function calls

11 months ago

Commits

amazon-bedrock

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

assistant-cli

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

azure-openai

96c6af17c9

feat: add logprobs support for azure openai (#376)

1 year ago

azure-openai-assistant

3014f7ebf0

chore: formatting

1 year ago

claude-vision

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

claude-vs-gpt

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

claude-vs-gpt-image

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

cloudflare-ai

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

cohere

017210ecc3

feat(cohere): add support for embeddings api (#1502)

1 year ago

cohere-benchmark

cee712c92e

docs: update default LLM models

11 months ago

colab-notebook

332c20943e

fix: reorder imports in promptfoo_example.ipynb

1 year ago

custom-grading-prompt

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

custom-prompt-function

96306c3b1a

feat: add support for markdown prompts (#1616)

11 months ago

custom-provider

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

custom-provider-embeddings

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

dbrx-benchmark

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

docker-code-generation-sandbox

0e17c18086

chore(python): enhance documentation, tests, formatting, and CI (#1282)

1 year ago

dynamic-var

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

extension-api

a6c2affdb5

docs: fix typo in hooks documentation

11 months ago

external-provider-config

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

function-tools-callback

f2d23ac32f

fix(providers): openai function tool callbacks handling (#1648)

11 months ago

gemma-vs-llama

3014f7ebf0

chore: formatting

1 year ago

gemma-vs-mistral

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

github-action

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

google-aistudio-gemini

3014f7ebf0

chore: formatting

1 year ago

google-sheets

c826bdcb02

fix: google sheets output (#1367)

1 year ago

google-vertex

7f3137d6f4

feat: add support for gemini embeddings via vertex (#1004)

1 year ago

gpt-4o-temperature-comparison

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

gpt-4o-vs-4o-mini

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

groq

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

http-provider

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

huggingface-hate-speech-detection

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

huggingface-inference-endpoint

3014f7ebf0

chore: formatting

1 year ago

huggingface-pii

cee712c92e

docs: update default LLM models

11 months ago

huggingface-similarity

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

ibm-granite-vs-llama

92830ecb92

fix: resolve issues with relative prompt paths (#1066)

1 year ago

image-classification

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

javascript-assert-external

cee712c92e

docs: update default LLM models

11 months ago

jest-integration

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

js-config

2a5d88c4b7

chore(openai): update model list (#1219)

1 year ago

json-output

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

langchain-python

cee712c92e

docs: update default LLM models

11 months ago

llama-cpp

bbec5e9ed6

docs: add example and configuration guide for using llama.cpp (#1104)

1 year ago

llama-gpt-comparison

23e57def26

chore: update llama examples and add support for chat formatted prompts in Replicate (#1261)

1 year ago

lm-studio

6802ce0ad1

fix(http provider): fix JSON prompt escaping in HTTP provider and add LM Studio example (#1156)

1 year ago

mistral

091cfd5847

feat(mistral): update chat models and add embedding provider (#1614)

11 months ago

mistral-llama-comparison

23e57def26

chore: update llama examples and add support for chat formatted prompts in Replicate (#1261)

1 year ago

moderation

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

multiple-configs

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

multiple-translations

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

multiple-translations-scenarios

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

multiple-turn-conversation

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

multishot

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

named-metrics

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

node-package

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

node-package-typescript

ad33256d02

chore: enforce consistent type imports (#1341)

1 year ago

nunjucks-custom-filters

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

ollama-comparison

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

openai-chat-history

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

openai-dalle-images

a3679d8d27

docs: move dalle example

1 year ago

openai-eval-factuality

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

openai-function-call

333e61958f

fix(providers): fix error in loading openai externally defined function calls

11 months ago

openai-multiline-yaml

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

openai-structured-output

cd48ec371b

chore(openai): improve support for structured outputs (#1656)

11 months ago

openai-tools-call

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

openai-vision

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

perplexity.ai-example

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

phi-vs-llama

806e52216f

chore(docs): add shell syntax highlighting and fix typos (#953)

1 year ago

portkey-test

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

prompt-labels

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

prompts-per-model

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

python-assert-external

cee712c92e

docs: update default LLM models

11 months ago

python-assert-inline

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

python-provider

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

rag-eval

09fbd07470

docs: Full rag example (#1228)

1 year ago

rag-full

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

redteam-dalle

c6bfcba9c3

docs: dalle jailbreak blog post (#1052)

1 year ago

redteam-provider-override

48551492c9

style: format redteam provider override example config file

1 year ago

replicate-lifeboat

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

replicate-llama2

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

select-best-example

54c01d5ecc

docs: update YAML syntax for prompts and providers arrays (#1574)

11 months ago

self-grading

cee712c92e

docs: update default LLM models

11 months ago

separate-test-configs

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

simple-cli

c0caa2f4ba

docs: fix simple test

11 months ago

simple-csv

21f4d639b2

docs: update model references from gpt-3.5-turbo to gpt-4o-mini (#1655)

11 months ago

simple-test

0c99bc73a7

chore: support file:// prefix for local file paths in `tests:` field in config (#1651)

11 months ago

sql-validation

09fbd07470

docs: Full rag example (#1228)

1 year ago

standalone-assertions

3014f7ebf0

chore: formatting

1 year ago

store-and-reuse-outputs

2ac593c9cb

chore: Fix formatting and add style check to CI (#872)

1 year ago

structured-outputs-config

6bb0342037

feat: ability to attach configs to prompts (#1391)

1 year ago

summarization

09fbd07470

docs: Full rag example (#1228)

1 year ago

test-markdown

5c88b43ac0

docs: add markdown test. Related to #873

1 year ago

tool-use

cee712c92e

docs: update default LLM models

11 months ago

transform-file

d9e79d506e

feat(transform): support custom function names in file transforms (#1435)

1 year ago

ts-config

adec5ba3a8

docs: add TypeScript promptfooconfig example (#1427)

1 year ago

vars-referencing-vars

98b8159de3

docs: update examples and docs to use gpt-4o and gpt-4o-mini models (#1323)

1 year ago

voyage-embeddings

a22257f01e

feat: add Voyage AI embeddings provider (#931)

1 year ago

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida / promptfoo mirror of https://github.com/promptfoo/promptfoo

Comments

Use AWS S3 as storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Use Google Cloud Storage!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Use Azure Cloud Storage!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Use any S3 compatible storage!

Specify your S3 bucket

Access key (If needed)

Congratulations!

nirbarazida
/
promptfoo
mirror of https://github.com/promptfoo/promptfoo