SkyNet
Decentralised ML across the interent, powered by ZK.
Created on 10th December 2023
•
SkyNet
Decentralised ML across the interent, powered by ZK.
The problem SkyNet solves
As said by Forbes, the future of AI will lie in the hands of the one that has access to the most powerful computing resources. More and more users will have the need to train more and more advanced models.
What if you could have a decentralised method to train the same machine learning model, with the epochs and data split across different machines, across the internet. SkyNet aims to solve that, enhanced with the power of zk.
Exisitng solutions like hivemind have a steep usage curve and are restricted to just a local network of machines. Federated learning is again restricted to a local network (unless the private IPs are exposed) and is more focused on data privacy.
SkyNet has a the following flow, coupled with the power of zero knowledge proofs:
- Choose or upload Data
- Select a Model from Marketplace (For Inference) or Upload Model Architecture for training
- Specify the computing resources requirements and budget.
- A Push Notification is sent to all interested participants
- When Requirements are met, the task is assigned to all participants and logged in the smart contract.
- The workflow is managed using Waku's P2P stack to ensure a reliable mode of communication which enables end-to-end delivery confirmations and uniquely identifies nodes in the subnet. It is used in the exchange of weights during the training process, without revealing the same to anyone else.
- Every model trained has a zkProof Verifier Contract generated. Proofs are generated for every worker running the model. These proofs can be verified by the contract deployed on the Scroll zkEVM.
Challenges we ran into
During the course of our project development, we encountered several challenges that required thoughtful resolution:
Peer-to-Peer Connection with Waku:
Establishing a reliable peer-to-peer connection with Waku proved challenging due to intermittent failures in the relay server. This hurdle necessitated a thorough examination of the server's reliability issues and the implementation of robust measures to ensure consistent and dependable connections.
Integration of Project Components:
Efficiently consolidating various project components posed a significant challenge.
File Upload Using Lighthouse:
Overcoming difficulties in file uploads through Lighthouse emerged as a pivotal concern. Thanks to mentorship, we successfully addressed this challenge by leveraging valuable guidance.
Tracks Applied (7)
Arbitrum Track
Arbitrum
Filecoin Track
Filecoin
Waku Track
waku
Alliance Track
Alliance
Push Protocol Track
Push Protocol
Lighthouse.storage Track
lighthouse
Scroll Track
Scroll
Cheer Project
Cheering for a project means supporting a project you like with as little as 0.0025 ETH. Right now, you can Cheer using ETH on Arbitrum, Optimism and Base.