TeachinAssistant
AI teaching assistant chatbot understands voice queries in 50+ languages via Whisper-1. It processes input through a LLM, considering chat history and topic knowledge using RAG pipeline.
Created on 6th October 2024
•
TeachinAssistant
AI teaching assistant chatbot understands voice queries in 50+ languages via Whisper-1. It processes input through a LLM, considering chat history and topic knowledge using RAG pipeline.
The problem TeachinAssistant solves
This project features an interactive AI-powered teaching assistant chatbot capable of understanding voice input of student queries in over 50 languages, using Whisper-1 model, which also removes background noise for clearer communication. This input is then processed by a LLM tailored to match specific Teaching assistant personalities, considering chat history and Topic knowledge stored in a vector database via Retrieval Augmented Generation (RAG). The result is a text response that ensures engaging interactions with Students.
Challenges I ran into
Issue: The combination of real-time voice transcription using Whisper-1, language model processing, and retrieval from the vector database introduced delays that pushed the response time beyond acceptable limits. Our goal was to reduce latency to under 500ms to ensure smooth, real-time interaction.
Solution: In the Retrieval Augmented Generation (RAG) pipeline, the retrieval step was one of the bottlenecks. To mitigate this, we optimized the vector database by implementing indexed search with asynchronous pre-fetching. This allowed the system to fetch relevant knowledge in parallel with the LLM processing, thereby reducing wait time.
Technologies used