The problem Base Indexer solves

🧩 Problem

Accessing real-time, structured on-chain data is still a challenge for teams building on networks like Base. Existing solutions often suffer from slow processing, incomplete data (e.g. missing contract interactions), and limited scalability when trying to index an entire chain from genesis.

✅ What This Solves

This solution enables:
• Real-time tracking of contract deployments and interactions with sub-2s latency
• Full backfill from the first block to capture historical context
• Structured storage in PostgreSQL for fast querying and integrations
• Scalable architecture using workers and threading to handle both past and live data concurrently

It provides a reliable, developer-friendly foundation for powering analytics, builder reputation, usage insights, and ecosystem monitoring — all in real time.

Challenges we ran into

Building a real-time blockchain indexer for the Base network came with several technical challenges. Here’s a breakdown of what we encountered and how we tackled each one:

⏱️ 1. Processing Blocks Fast Enough (Sub-2s)

Base produces a new block roughly every 2 seconds, which meant our indexer had to process blocks at or below that speed to avoid lagging behind.

Initial Approach: Fetching transaction receipts one-by-one using eth_getTransactionReceipt quickly became a bottleneck.

Solution: We switched to using eth_getBlockReceipts, which allows us to fetch all receipts for a block in a single call. This dramatically reduced latency and allowed the indexer to stay in sync in real time.

🧵 2. Handling Historical Backfill + Real-Time Sync

We needed the system to backfill all historical data from genesis while simultaneously processing new blocks in real time.

Challenge: Without parallelism, syncing from block 0 would take days or block real-time processing.

Solution: We introduced a worker/threaded architecture to split responsibilities:
• One worker handled real-time sync via WebSockets
• Another performed historical backfill in chunks
• Each worker operated independently but safely shared access to the database

This concurrency model helped us achieve a full sync without sacrificing real-time accuracy.

☁️ 3. Deploying to AWS with Terraform

Automating deployment and infrastructure setup via Terraform proved more complex than expected.

Challenge: Managing state, secrets, VPCs, and networking (especially for WebSocket reliability) involved trial and error.

Solution: With careful module design, environment separation, and ECS + RDS setup, we eventually deployed a robust indexer stack. Terraform now handles:
• PostgreSQL provisioning
• ECS service deployment
• Environment secrets and networking

Though tricky at first, this setup gives us a solid foundation for scaling and maintaining the service.