IRIS - Intelligent Real-world Interaction System

IRIS is a Digital AI Assit for Blinds

Created on 27th December 2025

•

Visually impaired individuals face three critical gaps that existing solutions fail to address affordably:

Problem: Traditional canes detect only ground-level obstacles within 1m
IRIS Brain 1: YOLOv8 detects 80+ objects up to 5m with directional spatial audio
"Person approaching, 3 meters, right ear alert"

Problem: Limited awareness of their surroundings and face challenges in navigation
IRIS Brain 2: Gemini Flash provides contextual scene descriptions
"Busy intersection. Auto-rickshaw passing left. Vegetable cart ahead right"

Problem: GPS fails indoors; no affordable wayfinding for malls, offices, homes
IRIS Brain 3: ARCore breadcrumb system—explore freely, voice-guided return
No pre-mapping. No internet. Just independence.

Barrier	IRIS Solution
Cost: Smart devices ₹40K-₹3L	Free—works on existing Android phones (₹8K+)
Language: English-only apps exclude 80%+	On-device Tamil, Hindi, Telugu, Kannada translation
Connectivity: 300M+ without stable internet	Offline detection + translation (only Q&A needs internet)
Infrastructure: Limited tactile paving, audio signals	Self-sufficient system, no infrastructure dependency

IRIS doesn't replace the white cane—it enhances it. Like a GPS for cars that already have steering wheels.

Challenge	Problem	Solution	Impact
⚡ Performance Conflict	YOLOv8 (15 FPS) + Gemini (2-5s) caused frame drops	Dual pipeline: Brain 1 continuous, Brain 2 on-demand	Detection never pauses—safety first
🎧 Useless Alerts	"Car detected" without direction	Stereo panning: Left ear = left object, right ear = right	Intuitive spatial awareness
🧭 ARCore Drift	Position errors accumulated over 5m+ walks	Yaw-based turns: "Left 90°" not coordinates	Navigation stays accurate
🗣️ Robotic Tamil	Default TTS hard to understand	Gemini (EN) → ML Kit (on-device) → ElevenLabsTTS (TA)	Natural language, zero latency

Core Learning: Safety-critical features (detection) must never depend on slow operations (LLM calls). Decouple intelligently.

Tracks Applied (2)

We use ElevenLabs as the unified audio feedback engine across Brain 1 (Safety), Brain 2 (Understanding), and Brain 3 (Me...Read More

Eleven Labs

Gemini is used as the core reasoning engine in IRIS to perform descriptive scene analysis and contextual understanding f...Read More

Gemini

Technologies used

ARCore

Android Studio

C++

Kotlin

YOLOv8

Elevenlabs

Google Gemini 2.0 Flash