DDA Challenge

Direct Data Access Dec 14, 2022

We’re excited to share the launch of our very own challenge - the DDA Challenge! Think of it as Hacktoberfest with a Dagshub twist🎃🌪.

With every Hacktoberfest, the Dagshub community rises to the occasion and makes amazing contributions to the ML field. Now that the event is over, let’s harness that creative energy and channel it into Direct Data Access (DDA).

Those who have previously participated in challenges will tell you that it can be tough, but it's also a terrific opportunity to learn, step out of your comfort zone, and communicate with people in the community.

Quick Start to Contribution

Sign-up for DagsHub.
Choose a open-source project or dataset from GitHub or DagsHub

DagsHub Direct Data Access

In case you missed it, earlier this month, we launched a new capability that has been the center of our efforts here at DagsHub - Direct Data Access (DDA). DDA lets you stream your data from, and upload it to, any DagsHub project. Even more mind-blowing, DDA doesn’t require you to make any changes to your project or dataset. 🤯

Of course, we think this is a game-changer for the ML field, but admit we are biased in our thinking 😆 So we decided to open up this challenge for all of you to support ML practitioners in reaping the benefits of DDA in their projects.

Where do I begin?

You don’t need to start from scratch. You can use a repository you have already worked on or choose another open-source project from GitHub or DagsHub.

If using a GitHub project:

Connect from Github to DagsHub
Add a dataset to DagsHub
Add DDA to the script
Run a training process and create a new experiment logged with MLflow or DagsHub logger.
Upload the new model to DagsHub using DDA.
Add the Label DDA challenge to the project.
Open a PR to the original project and mention you added the DDA capability.
Share the PR on our Discord server under the 💬share-your-project channel

Contribute DDA projects to DagsHub:

Fork the project you wish to contribute to and add the Label DDA challenge to the project.
Incorporate DDA into the code to stream the data.
Run a training process and create a new experiment logged with MLflow or DagsHub logger.
Upload the new model to DagsHub using DDA.
Open a PR to the original project and mention you added the DDA capability.
Share the PR on our Discord server under the 💬share-your-project channel

Here are some resources to further help you in your DDA implementation:

What you'll get if you take part in the DDA challenge

DagsHub will send swag to all participants who successfully contribute DDA projects to the DagsHub community and share their experience and project in the Discord community.

1 project - DagsHub Stickers
2 projects - stickers and T-shirt
5 Projects - full swag Package

Optional: Post and share your project on your social media accounts

Photo courtesy of Uktarsh Tomar who previously earned DagsHub swag! (Disclaimer: exact DagsHub swag will vary) — Photo courtesy of Uktarsh Tomar who previously earned DagsHub swag! *(Disclaimer: exact DagsHub swag will vary)*

FAQ’s

How does data streaming work?

Data streaming via DDA has two main implementations, Python Hooks and Mounted Filesystem, each valid for different cases. We recommend the Python Hooks method.

How to use Data Upload?

After installation you can use the upload function for both Git and DVC-tracked files. You can upload single files using the DagsHub CLI, or upload single or multiple files using the Python Client.

Do I need to know DVC to use DDA?

You don't need to know how to use DVC. You can version data, upload it to DagsHub and stream it to another machine.

Can you provide more help getting started?

Here you have some materials that will guide you through the DagsHub onboarding process.
- Get started tutorial - Covers the usage of Git & DVC with DagsHub.
- Connect an existing GitHub repo to DagsHub.
- Track experiments with either Git or MLflow