Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel
..
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
6dc0053681
fixed clerical error
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago
ed8916da2a
Added AWS Project Files
1 year ago

README.txt

You have to be logged in to leave a comment. Sign In
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
  1. #------ AWS User:
  2. ACNT ID: <ID>
  3. Name: <USER_NAME>
  4. arn: arn:aws:iam::<ID>:<USER_NAME>
  5. Access Key1: <TOKEN>
  6. #----- Code Commit: (AWS SERVICE)
  7. REPO: <REPO_NAME>
  8. HTTPS: https://git-codecommit.us-east-1.amazonaws.com/v1/repos/<REPO_NAME>
  9. SSH: ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/<REPO_NAME>
  10. #----- AWS S3: (AWS SERVICE)
  11. Bucket Name: <BUCKET_NAME>
  12. #----- AWS EC2: (AWS SERVICE)
  13. Instances: 1
  14. t2.micro | 8Gib | Canonical, Ubuntu, 22.04 LTS, amd64 jammy image build on 2024-03-01
  15. Name: mflow.demo-server
  16. Key Pair Name: <KP Name>
  17. ID: i-00000000000000000
  18. Connection:
  19. 1. Select Instance (Instance management Page)
  20. 2. Click Connect (Instance Summary Page)
  21. 3. Click Connect (Connect to instance page)
  22. #------- Get Security Credentials (for current IAM User):
  23. *Click 'Account/Security Credentials' drop down (upper right)
  24. *Access Keys Section
  25. *Select "Application running on AWS compute service"
  26. *Copy Access key info (This is the ONLY time retrieve it)
  27. *Enter Infomation into EC2 instance command line:
  28. Access Key: <TOKEN>
  29. Secret Key: <SECRET>
  30. *Default
  31. #------ Setup ENV:
  32. $ sudo apt update
  33. $ sudo apt install python3-pip
  34. $ sudo pip3 install pipenv virtualenv
  35. $ mkdir MLflow.Demo
  36. $ cd MLflow.Demo
  37. $ pipenv install mlflow awscli boto3 setuptools
  38. $ pipenv shell #activates virtual environment
  39. $ aws configure
  40. #Get AWS credentials and enter them
  41. #----- Setup MLFlow:
  42. $ mlflow server -h 0.0.0.0 --backend-store-uri sqlite:///mlflow.db --default-artifact-root s3://<BUCKET_NAME>
  43. #----- Open Port on EC2 Instance:
  44. 1. Go to EC2 > Instances > Instance
  45. 2. Click "Security" tab
  46. 3. Click Secuirty Group Link (ex: sg-000000ab000000c0d - launch-wizard-2)
  47. 4. Click "Edit inbound rules"
  48. 5. Click "Add Rule"
  49. - TYPE: Custom TCP
  50. - Port: 5000
  51. - Source: Anywhere-IPv4
  52. 6. Click "Save Rules"
  53. 7. Go back to instance page
  54. 8. Copy "Public IPv4 DNS" (ex: ec2-aa-bbb-ccc-ddd.compute-1.amazonaws.com)
  55. 9. Enter DNS followed by port#:
  56. ec2-aa-bbb-ccc-ddd.compute-1.amazonaws.com:5000
  57. 10. MLflow should appear
  58. #-------- Clone the REPO:
  59. 1. Go to "Code Commit"
  60. 2. Open Repo
  61. 3. Click "Clone URL" dropdown
  62. - Select: HTTPS
  63. ex: https://git-codecommit.us-east-1.amazonaws.com/v1/repos/<REPO_NAME>
  64. 4. Go to "Security" tab
  65. 5. Click "AWS CodeCommit credentials"
  66. 6. Click "Generate Credentials"
  67. - Download Credentials
  68. - User Name: <USER_NAME>-at-<ID>
  69. - PW: <PW>
  70. 7. Create a local folder to work on project
  71. - ~/MLflow.Demo
  72. 8. Open gitbash: Navigate to local folder:
  73. - ~//MLflow.Demo
  74. 9. Type:
  75. $git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/<REPO_NAME>
  76. 10. Enter Credentials: user name / pw
  77. #---------- Setup Repo localy using local IDE: (VSCODE)
  78. 1. create data folder
  79. - ./MLflow.Demo/data
  80. 2. Copy Train & Test data to folder
  81. 3. Create a "data.py" file to prep data
  82. - fill with provided code
  83. 4. Create a "train.py" file to trian model
  84. - fill w/ provided code
  85. 5. Create a "parms.py" file to store ParameterGrid options
  86. - fill w/ provided code
  87. 6. Create a "utils.py" file to store helper evaluation functions
  88. - fill w/ provided code
  89. 7. Create a "MLproject" file
  90. - fill w/ provided information
  91. 8. Create a "conda.yaml" file
  92. - fill w/ provided informaiton
  93. 9. Create a "run.py" file
  94. - fill w/ provided code
  95. 10. Create local conda environment (using conda.yaml)
  96. 11. Activate local conda environment:
  97. $conda activate env_MLflow
  98. 12. Launch Mlflow server:
  99. $mlflow server --backend-store-uri sqlite:///mlflow.db --default-artifact-root ./mlflow-artifacts --host 127.0.0.1 --port 5000
  100. #----- Commit to Git:
  101. git bash:
  102. $git add *
  103. $git commit -m "add code for mlflow demo"
  104. $git push
  105. Result is pushing to AWS code commit repo:
  106. https://git-codecommit.us-east-1.amazonaws.com/v1/repos/<REPO_NAME>
  107. #------- Amazon Sage Maker:
  108. 1. Open sagemaker
  109. 2. Open Git Repository List
  110. Left Bar > Notebook > Git Repositories
  111. 3. Click Add Repository
  112. 4. Click "AWS CodeCommit"
  113. - Option: "Use Existing Repository"
  114. - Repository: MLflow.Demo (see above)
  115. - Branch: main
  116. - Sagemaker Repository name: <REPO_NAME>
  117. #Errors would show when creating saying regex patterns not matched
  118. #Refreshing page fixed the error
  119. URL: https://git-codecommit.us-east-1.amazonaws.com/v1/repos/<REPO_NAME>
  120. ARN: arn:aws:sagemaker:us-east-1:<ID>:code-repository/<REPO_NAME>
  121. 4. Create a notebook instance:
  122. Name: MLflow-Demo-nbi
  123. Instance Type: ml.t3.large
  124. 5. Create a role (do not close notebook instance tab)
  125. - Click Security (upper right) > Left bar > Roles
  126. - Create Role
  127. - Entity Type: AWS service
  128. - Use Case: Sagemaker
  129. - Select "Next"
  130. - Select "Next"
  131. - Enter Values:
  132. - Role Name: <ROLE_NAME>
  133. - Select "Create Role"
  134. - Select Role (<ROLE_NAME>)
  135. - Select "Add Permisions"
  136. - Select "Attach Policies"
  137. - Search for "CodeCommit"
  138. - Select "AWSCodeCommitFullAccess"
  139. - Search for "S3"
  140. - Select "AmazonS3FullAccess"
  141. - Search for "ECR"
  142. - Select "EC2InstanceProfileForImageBuilderECRContainerBuilds"
  143. - Select "Add Permissions"
  144. 6. Select Role in Notebook instance tab
  145. - Name: <ROLE_NAME>
  146. - Option: Enable
  147. 7. Select Git Repositories
  148. - Select code commit
Tip!

Press p or to see the previous file or, n or to see the next file

Comments

Loading...