Lens Substream
A Rusty way to stream Lens data using Substreams
Created on 10th June 2023
•
Lens Substream
A Rusty way to stream Lens data using Substreams
The problem Lens Substream solves
Decentralized social media generates a huge amount of on-chain data. This on-chain data needs a low-latency interface for it to remain accessible for all, along with adding the ability to perform real-time analysis and monitoring.
Substreams are a data solution developed for The Graph Network. They allow developers to write Rust modules composing data streams with low-cost caching and archiving of blockchain data, high throughput processing, and cursor-based reorgs handling.
This project builds a (basic) Substream in Rust to monitor on-chain Lens protocol data and provide a low-latency data streaming interface. Potential uses of such a tool could be for real-time analysis and monitoring of Lens data. Since the data is streamed, it can be directly injected into a variety of databases (BigQuery, Clickhouse, Kafka etc.), opening up a lot of distinct analytics use cases.
A few unrelated citations:
Challenges I ran into
- Substreams are an extremely new technology. The most actively developed implementation by the team (uniswap-v3) is still in beta. Therefore, the documentation is extremely limited. I worked around this by simple trial and error, playing around with things until they worked.
- Lens protocol is complex and involves multiple moving parts, this implementation only covers a small fraction of it. However, given the modular nature of the substream, this functionality can be added in the future.
- Given the novelty of working with Substreams, I would particularly like to thank Alex and Ørjan from The Graph for helpful discussions.
Tracks Applied (1)
Social Impact
Technologies used