The problem AI pal solves

Mental health support remains inaccessible for many due to stigma, high costs, or lack of availability, especially for students, individuals in remote areas, or those afraid of being judged for seeking therapy. Traditional wellness apps often lack empathy and human connection.

AI Pal bridges this gap by acting as a 24/7, voice-enabled mental health companion that listens, understands, and supports in real time.

What sets AI Pal apart is its multimodal input system, it doesn’t just listen to your voice, but also reads facial expressions and emotional cues through video. Using real-time emotion detection, it mirrors the user’s emotional state, making responses feel more human and validating.

From guided breathing and movie-based therapy to empathetic voice interactions and access to professional help, AI Pal offers an accessible, emotionally intelligent alternative to traditional care.

Research shows that being seen and heard,both emotionally and visually can greatly reduce stress and improve mental resilience, and AI Pal delivers exactly that.

Challenges we ran into

Building AI Pal came with its fair share of challenges. Initially, we didn’t have access to testnet ETH, which made it hard to simulate transactions, so we pivoted to using the Base testnet as a workaround.

On the voice AI side, latency and a lack of emotional tone made the experience feel robotic. To address this, we experimented with different transcriber models, tested various voices from providers like ElevenLabs, switched between LLMs on the backend, and explored real-time streaming solutions like WebSockets and WebRTC.

Implementing real-time emotion detection in the browser was especially difficult. Ensuring smooth and continuous emotion capture required strategic timing, so we implemented a 3-second timer to analyze user emotion and sliced image frames every 2 seconds to maintain accuracy while minimizing performance issues.

Research was another major hurdle: gathering relevant literature to validate the effectiveness of AI-driven therapy was time-consuming, but we were able to compile meaningful insights through focused research and case studies.