The problem Sparsh Mukthi solves

In today's digital world, human-computer interaction is still heavily dependent on traditional input methods like keyboards, mice, and joysticks — tools that haven't changed fundamentally in decades. These interfaces can be inefficient, non-intuitive, inaccessible to physically challenged users, and limiting in immersive or hands-free environments.

This project aims to transform the way users interact with computers by replacing conventional input devices with a gesture-based, adaptive, AI-driven system that runs fully on the edge. Built using affordable hardware (Arduino, MPU6050 sensor, and webcam), this system enables users to control any computer with natural body movements and hand gestures — offering a frictionless, touchless, and immersive control experience.

✅ Problems It Solves:
Accessibility: Provides a powerful control option for users with physical disabilities who struggle with keyboards or mice.
Gaming & Entertainment: Replaces joysticks and keypads with body motion and hand gestures, creating a more immersive and dynamic gameplay experience.
Industrial or Sterile Environments: In places like medical labs or manufacturing floors where physical contact is unsafe or infeasible, this system offers a hygienic, gesture-based interface.
Cost Barrier: Offers Vision-Pro-like control at a fraction of the cost (₹5000 vs ₹3.5L+), making it accessible for students, researchers, and developing regions.
Personalization: Users can customize gestures, making the system adapt to personal comfort and efficiency over time.
Offline and Secure Usage: The system runs 100% on-device without any internet dependency, ensuring total privacy and functionality in offline or restricted settings.
In short, this project reimagines everyday digital interaction through affordable innovation, edge AI, and adaptive gesture recognition, solving real-world usability, cost, and accessibility challenges in a futuristic yet practical way.

Challenges we ran into

⏱️ Time Pressure:
Working within a limited timeframe meant rapid prototyping, debugging, and decision-making. Balancing ambition with feasibility was key, especially while integrating multiple complex features like AI adaptation and VR simulation.
🧠 Building Adaptive AI:
Designing an AI system that could learn from and personalize itself to user behavior locally (without relying on cloud models) was one of the most technically demanding parts. It involved careful data handling, feedback tuning, and lightweight training compatible with edge devices.
💻 Code Integration Across Systems:
Since team members worked on different parts of the project on separate laptops, merging the code into one cohesive, functioning system involved dealing with conflicting dependencies, version mismatches, and last-minute integration bugs.
🧩 Combining Modular Components:
The project was divided into modules — hardware sensing, computer vision, gesture-action mapping, and UI customization. Bringing all the independently developed pieces together into a seamless, unified experience required tight coordination and active collaboration.
🧪 Training and Testing AI Models:
Gathering usable training data for gestures, ensuring model reliability under varying lighting/background conditions, and optimizing real-time inference without lag were all significant hurdles.
Despite these challenges, through continuous testing, brainstorming, and agile collaboration, we were able to build a working prototype that met our core goals and maintained our vision of building a low-cost, immersive, AI-powered gesture interface.