perdedune
Automagically parse all queries from a dashboard to reduce duplication and optimise your code!
Created on 18th September 2022
•
perdedune
Automagically parse all queries from a dashboard to reduce duplication and optimise your code!
The problem perdedune solves
You're a Dune Wizard and have written lots of SQL. However, Dune runs every query atomically - meaning much repeated code.
Waiting for your dashboard to queue and run costs you and your users valuable time. Further, long, repeated queries quickly damages both usability and maintainability.
Automagically de-duplicate SQL statements across your dashboard with our tool!
perdedune tool demo
perdedune query validation demo
Architecture diagrams
Challenges we ran into
We overcame a number of hurdles:
-
To begin with, though the Dune API is effective at fetching query results, it did not have dashboard nor SQL endpoints. Thus, we had to reverse engineer the frontend requests to extract the necessary data.
-
Many SQL parsers did not make the cut for our needs. We tested a number of parsers, including sqlparse and pglast, before settling on sqlglot.
-
Lastly, Dune does not yet allow the querying the results of other queries . Therefore, we extracted a sample dataset using the Dune API, and carried out the local validation using docker and postgresql.