Challenges I ran into
Real-time WebRTC Implementation Issues
The biggest nightmare was getting WebRTC to work consistently across different browsers and network conditions. Initially, the video calls would randomly drop or fail to establish connections, especially when students joined from mobile devices or poor network areas.
Solution: Had to implement a fallback mechanism with TURN servers and added network quality detection. Spent 3 days debugging why Chrome and Safari were handling getUserMedia() differently for camera access.
Facial Recognition Accuracy Problems
The face recognition system was giving too many false positives and negatives. Students with glasses, different lighting conditions, or when they changed hairstyles - the system would fail to recognize them properly.
Solution: Switched to a multi-angle training approach and added confidence scoring. Had to collect more diverse training data and implement a backup verification system that asks students to remove glasses or move to better lighting.
MongoDB Aggregation Pipeline Performance
When generating reports for large datasets (1000+ students), the queries were taking 15-20 seconds to load, making the admin dashboard practically unusable during peak usage.
Solution: Redesigned the database schema with proper indexing and implemented data caching with Redis. Also broke down complex aggregation queries into smaller, parallel operations. Performance improved from 20s to under 2s.
Socket.io Memory Leaks in Live Sessions
During long live classes (2+ hours), the server memory would keep increasing and eventually crash when handling 50+ concurrent users. Took forever to figure out that event listeners weren't being cleaned up properly.
Solution: Implemented proper cleanup mechanisms in useEffect hooks and added connection pooling. Had to add heartbeat checks to detect disconnected clients and clean up their resources automatically.
Gemini API Rate Limiting Chaos
The AI chatbot would randomly stop working during high usage periods because we were hitting Google's API limits without proper queue management. Students would get frustrated when their doubts weren't getting answered.
Solution: Built a smart queueing system with request prioritization and implemented response caching for common queries. Added fallback responses when API limits are reached and upgraded to a higher tier plan.
CORS and Authentication Headaches
Spent 2 full days fighting CORS errors when trying to implement file uploads with authentication. The frontend couldn't send files to the backend with JWT tokens properly.
Solution: Had to configure express-cors properly and implement custom middleware for handling multipart/form-data with authentication. Also switched from storing tokens in localStorage to httpOnly cookies for better security.