The primary issue that plagued this idea was to find a way to choose the correct files to prompt LLM's, paying respect to the context size that would otherwise destroy any naive approach of passing the entire codebase for a prompt, once we had that figured out by just recursively calling Openhermes-API, the problems were more of the general sort, ranging from thinking about adding session management for the first time in our developer journey without adding a login screen, or finding a way to speed up the git files metadata extraction process(which improved by over 20x compared to the initial naive method we employed in the initial stages).
Concurrency was also an issue, and while its not built up yet for a complete production level of perfection, it still manages to hold up against everything we could throw at it (which was about ten of our friends all spamming questions from their laptops ).
While there are still a lot of issues we might fix, we are confident this might just be good enough