In the process of building our project, one of the issues we encountered was the time-consuming process of fetching up-to-date data from our database. When there's a significant number of blocks accumulated in our database, synchronization becomes a challenge, resulting in a delay in fetching the latest data.
We overcame this hurdle through two strategies:
- Logical Solution: When a user makes a request to inquire about a particular protocol, our server first checks the synchronization status by comparing the latest block in our database with the current block. This Boolean information about synchronization status is then relayed to the user. Subsequently, our server initiates a synchronization process to update the database with the latest data. Once this process is complete, a signal is sent to the frontend which was waiting using the 'await' keyword. Meanwhile, on the frontend, we have devised a system to periodically refresh the data every minute to prevent any user inconvenience caused by UI changes due to data updates.
- Batch Processing: We also implement batch jobs on our server. Anticipating frequent requests for certain protocols and regular calls from the explorer, we perform batch jobs to preload data related to these protocols. This way, we can minimize the load when queries are made for these often-accessed protocols.
Going forward, post-hackathon, we have some plans for further enhancement. Given the nature of WalletMatrix, we can access the information of wallets that have initiated transactions on a particular protocol. By collating and ranking protocols based on the number of transactions made from all wallets, we can schedule batch jobs in advance for those protocols that are accessed most frequently. We believe that this will significantly improve the user experience.