Connect to our Discord channel
Models
DVC
To get started, have a look at asserts.yaml
If you use a model-graded eval, you must set your OPENAI_API_KEY environment variable or override the provider (see https://promptfoo.dev/docs/configuration/expected-outputs/model-graded/#overriding-the-llm-grader).
Then run:
Annotations
Ian Webster
promptfoo eval --assertions asserts.yaml --model-outputs outputs.json
outputs-with-tags.json shows a different JSON format that allows you to show basic string metadata in the web UI view.
Afterwards, you can view the results by running promptfoo view
Press p or to see the previous file or, n or to see the next file
Browsing data directories saved to S3 is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!
promptfoo is now integrated with AWS S3!
Are you sure you want to delete this access key?
Browsing data directories saved to Google Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!
promptfoo is now integrated with Google Cloud Storage!
Are you sure you want to delete this access key?
Browsing data directories saved to Azure Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!
promptfoo is now integrated with Azure Cloud Storage!
Are you sure you want to delete this access key?
Browsing data directories saved to S3 compatible storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!
promptfoo is now integrated with your S3 compatible storage!
Are you sure you want to delete this access key?