Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel

#2 Reorganize DVC pipeline and parameters

Merged
Ghost merged 1 commits into produdez:main from produdez:reorganize
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
  1. # Ref: Yaml variable: https://stackoverflow.com/questions/4150782/use-yaml-with-variables
  2. # ---- Variables ----
  3. data:
  4. main-data: &main-data
  5. path: 'data/kangaroo'
  6. size: 164
  7. train_size: 150
  8. sample-data: &sample-data
  9. path: 'data/sample'
  10. size: 10
  11. train_size: 5
  12. test-data: &test-data
  13. path: 'data/kangaroo-kaggle'
  14. size: 313
  15. train_size: NULL
  16. # ---- Misc ----
  17. src:
  18. current-model: 'src/submodules/akTwelve_MaskRCNN'
  19. pipeline:
  20. environment-check: 'src/pipeline/environment_check.py'
  21. train-model: 'src/pipeline/train_model.py'
  22. evaluate: 'src/pipeline/evaluate.py'
  23. # ---- Stages/Pipelines ----
  24. gen-dag:
  25. output: out/dependency_diagram.md
  26. environment-check:
  27. environment-name: test-kangaroo
  28. output: out/environment-check.out.json
  29. train-pipeline:
  30. data: &training-data
  31. <<: *sample-data
  32. # <<: *main-data
  33. train_dir: 'model/training'
  34. output:
  35. model: &training-weight
  36. 'out/current-model.h5'
  37. log: 'out/model-train.output.log'
  38. metric:
  39. folder: out/train.metrics/
  40. general: 'general.json'
  41. others:
  42. - epoch_mrcnn_bbox_loss
  43. - epoch_val_mrcnn_bbox_loss
  44. summary: 'out/model-train.summary.txt'
  45. evaluate:
  46. data: *training-data
  47. model-weight: *training-weight
  48. evaluate_dir: 'model/evaluate'
  49. output:
  50. metric: 'out/evaluate.json'
  51. log: 'out/evaluate.output.log'
Discard
Tip!

Press p or to see the previous file or, n or to see the next file