Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel
Integration:  dvc git github
da0a0edfaf
remove dvc pipeline
1 year ago
eef417eac8
add .gitkeep to data/raw
1 year ago
src
738f816178
edit proces data
1 year ago
f6390cde33
finish segmentation
2 years ago
ce34c3f07f
add dvc yaml
2 years ago
de52079bda
add processed functions
2 years ago
da0a0edfaf
remove dvc pipeline
1 year ago
484f2da37c
add prefect
1 year ago
ff08115759
edit code
1 year ago
eef417eac8
add .gitkeep to data/raw
1 year ago
a28ead1b57
track metrics.csv with git
2 years ago
10ef650412
Update README.md
1 year ago
b79a5a3969
process_1
2 years ago
2d289e3c1f
remove notebook
1 year ago
Storage Buckets
Data Pipeline
Legend
DVC Managed File
Git Managed File
Metric
Stage File
External File

README.md

You have to be logged in to leave a comment. Sign In

Prefect + DVC Demo

A demo on how to use Prefect with DVC

Tools Used in This Project

  • DVC: Data version control
  • Prefect: Orchestrate and observe dataflow

Project Structure

  • src: consists of Python scripts
  • config: consists of configuration files
  • data: consists of data
  • notebook: consists of Jupyter Notebooks
  • tests: consists of test files

Set Up the Project

  • Install requirements:
pip install -r requirements.txt

Run the Project

To run all flows, type:

python src/main.py
Tip!

Press p or to see the previous file or, n or to see the next file

About

No description

Collaborators 1

Comments

Loading...