The problem memo.ai solves

Inefficient Information Processing:
In the digital age, individuals often struggle with processing extensive and diverse forms of information efficiently. This leads to wasted time and cognitive overload, hindering effective learning and comprehension.

Learning Resource Integration Challenge:
Learners face difficulties in integrating diverse learning resources like PDFs, videos, and audio files into a coherent study plan. This disintegration leads to a fragmented learning experience and hampers knowledge retention.

Neglect of Physical Health During Study:
Extended periods of study or work without proper ergonomic practices can lead to health issues like poor posture. This problem is often overlooked, but it can have long-term consequences on physical well-being.

Poor Time Management in Self-Study:
Self-paced learners frequently struggle with time management, leading to procrastination and ineffective study sessions. Without structured time allocation, maintaining focus and productivity becomes a significant challenge.

Challenges we ran into

For our hackathon project, we encountered a series of significant challenges that tested our technical skills and creativity. Here's how we navigated through each obstacle:

1. Rapid Video Transcription - To figure out how to transcribe video content swiftly, we decided on using the medium Whisper model for its efficiency. Deploying it on a T4 GPU, we achieved a balance between speed and accuracy, enabling fast transcription without compromising on quality.
2. Pose Detection Implementation - Integrating a pose detection model into the frontend presented a unique set of challenges. We benchmarked various models to find the most suitable one, given that running inference and AI models directly in browsers demands extensive computational resources. Ultimately, TensorFlow.js with PoseNet emerged as the optimal choice due to its performance and feasibility for real-time pose detection.
3. Posture Algorithm - Designing an algorithm to accurately detect bad posture was intricate. We developed a method that combines the coordinates of the eyes and neck to calculate the distance by which the neck deviates from a neutral position. A threshold was established to determine when slouching occurs, triggering an alert to correct posture.
4. Generation of Properly Formatted Notes - To automate the creation of well-structured notes, we leveraged GPT-4. Through trial and error with various prompting techniques, we crafted prompts that elicited responses in a correctly formatted markdown, ensuring the notes were both informative and well-organized.
5. Backend Deployment and Frontend Integration - Perhaps the most daunting challenge was to fully deploy the backend and integrate it with the frontend within a mere two days. Fuelled by six Red Bulls and relentless determination, we managed to achieve a seamless integration, bringing our project to life.
   Each challenge pushed us to our limits, but through innovative solutions and teamwork, we overcame them, learning valuable lessons.