Data collection - During the process of retrieving transaction data via RPCs, it became evident that the speed was notably slow and many RPCs were unreliable. To address this, the team developed a framework enabling uninterrupted dataset downloads, parallelizing the process across CPU cores and team members' computers.
Employing the DeepWalk algorithm on our acquired dataset posed computational complexities due to extensive simulations required. Despite its compatibility with sparse matrices, it consumed much RAM space, exacerbating the slowdown. However, for smaller datasets, it proved more feasible and user-friendly.
Another hurdle emerged with the application of default Louvain and Connected Components methods on large sparse matrices, as these methods were ill-suited for such data. To circumvent this, allocating greater computational resources became necessary. Crafting a tailored framework for sparse matrices wasn't pursued due to substantial time demands. Data preprocessing techniques were algorithm-dependent, employed to minimize RAM usage.
The protracted convergence time of the Louvain algorithm prior to project submission posed challenges. Running it over the last two days of the hackathon, even with the complete data, might not ensure convergence due to its O(n*log(n)) time complexity. Although we executed the algorithm on a manageable sample, reliable outcomes demand a run on the full dataset, highlighting the method's nature.
One uncompleted task pertains to translating deepwalk-generated embeddings into user addresses ineligibly participating in the airdrop. Despite being a common task, time constraints hindered resolution before the submission deadline. Additionally, decoding input data for smart contract communication, and analyzing External Owned Account (EOA) calls while applying ML methods to potential actions (swaps, liquidity provision, etc.), was more time-intensive than anticipated. These actions are equally popular among airdrop hunters.
Tracks Applied (2)
Polygon
Discussion