Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel
Resources  
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Explore Pricing
Product  
Data Engine LLMs Platform Enterprise
Connect to Dagshub Discord channel
Register
Login

avatar Dean
/
Bookdata-tools
mirror of https://github.com/BoiseState/bookdata-tools

Watch 2
Issues   Discussions 0 Invite collaborators

dvc.lock 2.7 KB
History Raw

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91 
  1. schema: '2.0'
    2. 

  2. stages:
    3. 

  3.   scan-authors:
    4. 

  4.     cmd: python ../run.py --rust scan-marc -L -o viaf.parquet ../data/viaf-clusters-marc21.xml.gz
    5. 

  5.     deps:
    6. 

  6.     - path: ../data/viaf-clusters-marc21.xml.gz
    7. 

  7.       md5: 2f1af5262584d38f7331d333dec81cc3
    8. 

  8.       size: 10662471024
    9. 

  9.     - path: ../src/cli/scan_marc.rs
    10. 

  10.       md5: 12a9d5d9854034367c903c5abb99d543
    11. 

  11.       size: 3557
    12. 

  12.     - path: ../src/marc
    13. 

  13.       md5: db6eef7d87b56283dbbcdeea71a69a37.dir
    14. 

  14.       size: 16783
    15. 

  15.       nfiles: 5
    16. 

  16.     outs:
    17. 

  17.     - path: viaf.parquet
    18. 

  18.       md5: 4e274ee04004e80f8da0f1168f123ff2
    19. 

  19.       size: 10561644263
    20. 

  20.   author-names:
    21. 

  21.     cmd: python ../run.py --rust fusion author-names.tcl
    22. 

  22.     deps:
    23. 

  23.     - path: author-names.tcl
    24. 

  24.       md5: d0ebf76118c885cd85ce01ddc87cdb64
    25. 

  25.       size: 223
    26. 

  26.     - path: viaf.parquet
    27. 

  27.       md5: 4e274ee04004e80f8da0f1168f123ff2
    28. 

  28.       size: 10561644263
    29. 

  29.     outs:
    30. 

  30.     - path: author-names.csv.gz
    31. 

  31.       md5: e825c6458652609163ea3194fd9197ea
    32. 

  32.       size: 418433891
    33. 

  33.   author-genders:
    34. 

  34.     cmd: python ../run.py --rust fusion author-genders.tcl
    35. 

  35.     deps:
    36. 

  36.     - path: author-genders.tcl
    37. 

  37.       md5: 53a3022f70f9a5b5662a0b97501c891b
    38. 

  38.       size: 183
    39. 

  39.     - path: viaf.parquet
    40. 

  40.       md5: 4e274ee04004e80f8da0f1168f123ff2
    41. 

  41.       size: 10561644263
    42. 

  42.     outs:
    43. 

  43.     - path: author-genders.parquet
    44. 

  44.       md5: cbd6871e6ba229881aed5f58126fd652
    45. 

  45.       size: 102791330
    46. 

  46.   index-names:
    47. 

  47.     cmd: python ../run.py --rust index-names author-names.csv.gz author-name-index.parquet
    48. 

  48.     deps:
    49. 

  49.     - path: ../src/cli/index_names.rs
    50. 

  50.       md5: 06ccbf23dca07e00e87d89dec625287a
    51. 

  51.       size: 2363
    52. 

  52.     - path: author-names.csv.gz
    53. 

  53.       md5: e825c6458652609163ea3194fd9197ea
    54. 

  54.       size: 418433891
    55. 

  55.     outs:
    56. 

  56.     - path: author-name-index.csv.gz
    57. 

  57.       md5: b3aabf1eadec3499fec1791e6404e2d3
    58. 

  58.       size: 497228706
    59. 

  59.     - path: author-name-index.parquet
    60. 

  60.       md5: 663023a30c3d2f93ca4dea7d26793e9d
    61. 

  61.       size: 483832252
    62. 

  62.   schema@author-name-index:
    63. 

  63.     cmd: python ../run.py --rust pq-info -o author-name-index.json author-name-index.parquet
    64. 

  64.     deps:
    65. 

  65.     - path: author-name-index.parquet
    66. 

  66.       md5: 663023a30c3d2f93ca4dea7d26793e9d
    67. 

  67.       size: 483832252
    68. 

  68.     outs:
    69. 

  69.     - path: author-name-index.json
    70. 

  70.       md5: 51182fb692c729e004c9c2f35b24de25
    71. 

  71.       size: 351
    72. 

  72.   schema@viaf:
    73. 

  73.     cmd: python ../run.py --rust pq-info -o viaf.json viaf.parquet
    74. 

  74.     deps:
    75. 

  75.     - path: viaf.parquet
    76. 

  76.       md5: 4e274ee04004e80f8da0f1168f123ff2
    77. 

  77.       size: 10561644263
    78. 

  78.     outs:
    79. 

  79.     - path: viaf.json
    80. 

  80.       md5: a3a19c00b21edb6eab06585afc95be57
    81. 

  81.       size: 1059
    82. 

  82.   schema@author-genders:
    83. 

  83.     cmd: python ../run.py --rust pq-info -o author-genders.json author-genders.parquet
    84. 

  84.     deps:
    85. 

  85.     - path: author-genders.parquet
    86. 

  86.       md5: cbd6871e6ba229881aed5f58126fd652
    87. 

  87.       size: 102791330
    88. 

  88.     outs:
    89. 

  89.     - path: author-genders.json
    90. 

  90.       md5: 4a450d88e5c99af2e6ca3d850bf27f71
    91. 

  91.       size: 352
    92.
Prev
Next
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Write Preview
Loading...
Integrate Google Cloud Storage
Use Google Storage
Select bucket
Upload key
Finish

Use Google Cloud Storage!

Browsing data directories saved to Google Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your Google Storage bucket

Service Account Key

Congratulations!

Bookdata-tools is now integrated with Google Cloud Storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate AWS S3
Use S3 remote
Select bucket
Access key
Finish

Use AWS S3 as storage!

Browsing data directories saved to S3 is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Bookdata-tools is now integrated with AWS S3!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate S3 compatible storage
Use S3 like remote
Select bucket
Access key
Finish

Use any S3 compatible storage!

Browsing data directories saved to S3 compatible storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your S3 bucket

Access key (If needed)

Congratulations!

Bookdata-tools is now integrated with your S3 compatible storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes
Integrate Azure Cloud Storage
Use Azure Storage
Select bucket
Set key
Finish

Use Azure Cloud Storage!

Browsing data directories saved to Azure Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!

Specify your Azure Storage bucket

Access key (If needed)

Congratulations!

Bookdata-tools is now integrated with Azure Cloud Storage!

Delete Storage Key

Are you sure you want to delete this access key?

No
Yes