Problem 1 : CPU and Queue Congestion Under Load
The Hurdle: During high-traffic simulations (testing 1,000 requests in batches), the system hit a specific bottleneck in "Batch 5." A temporary slowdown occurred where the maximum response time spiked to 4.37 seconds due to CPU and queue congestion.

Solution :

We implemented a Parallel Backpressure-handling algorithm using Sinks.Many.

This allowed the reactive buffer to be dynamically reconfigured and resized during runtime to adapt to the fluctuating workload.

By leveraging Non-Blocking I/O and this reactive backpressure, the system was able to recover quickly in subsequent batches, stabilizing response times back to the 30–70 ms range.

Problem 2: Complexity of Blockchain Integration

The Hurdle: We initially explored Hyperledger Fabric for its modular design, but we ran into issues with its complexity, which hindered the lightweight, reactive integration we aimed to achieve.

Solution:

We pivoted to BigchainDB, which offered a lighter framework better suited for reactive integration.

To further mitigate the slowness of direct blockchain queries, we designed a hybrid caching layer using Redis. Instead of querying the blockchain for every read, we cached transaction URLs in Redis mapped to user IDs, allowing for immediate retrieval.

Problem 3: Handling Real-Time Data Updates

The Hurdle: Traditional REST API polling was too slow and inefficient for the low-latency demands of modern applications (e.g., healthcare/finance), causing delays in data propagation.

Solution:

We built a Redis Watcher Service combined with WebSockets.

This service monitors specific User IDs for changes in the cache. When a change is detected, it triggers an immediate HTTP request to BigchainDB and streams the result directly to the client via WebSocket, eliminating the need for repeated client polling.