Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel

data_validation.yml 1.5 KB

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
  1. name: Data validation with deepchecks
  2. on:
  3. push:
  4. branches: [ main ]
  5. pull_request:
  6. branches: [ main ]
  7. permissions: write-all
  8. jobs:
  9. run_data_checks:
  10. runs-on: ubuntu-latest
  11. env:
  12. DVC_USERNAME: ${{ secrets.DVC_USERNAME }}
  13. DVC_PASSWORD: ${{ secrets.DVC_PASSWORD }}
  14. steps:
  15. - uses: actions/checkout@v3
  16. - name: Set up Python 3.10
  17. uses: actions/setup-python@v3
  18. with:
  19. python-version: "3.10"
  20. - name: Install dependencies
  21. run: |
  22. python -m pip install --upgrade pip
  23. pip install -r requirements.txt
  24. - name: Get data
  25. run: |
  26. # dvc remote add origin https://dagshub.com/eryk.lewinson/mario_vs_wario_v2.dvc
  27. dvc remote modify origin --local auth basic
  28. dvc remote modify origin --local user $DVC_USERNAME
  29. dvc remote modify origin --local password $DVC_PASSWORD
  30. dvc pull create_train_test_split -r origin
  31. - name: Remove the old validation report
  32. run:
  33. rm -f data_validation.html
  34. - name: Validate data with deepchecks
  35. run: |
  36. python src/validate_data.py
  37. - name: Commit the validation report file
  38. uses: stefanzweifel/git-auto-commit-action@v4
  39. with:
  40. commit_message: Added the data validation file
  41. - name: Create a comment
  42. uses: peter-evans/commit-comment@v1
  43. with:
  44. body: |
  45. Please check the data_validation.html file for a full validation report.
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Loading...