The problem Jyoti: Assistant for visually impaired solves

India has the largest population of blind people, 15 million and makes up almost 40% of the worlds blind population. Amongst these 15 million 2 million are childern below 10 and amongst these 2 million only 5% are literate. Further, these people are more prone to accidents and hence living a normal life becomes even more tedious.
Through Jyoti, we mainly aim to tackle both of these problems. Jyoti is basically an assistant which guides them through life. Jyoti comprises of a microphone, a camera and speakers on a portable headwear. Through the camera in this device, we perform tasks like "Scene Description", "Object Detection", "Optical Character Recognition", etc.
We further used Natural Language Processing (NLP) to make the interactions wuth the assistant as human as possible. All these things combinely form a visual representation of the world around us hence aiding the visually impaired.

Challenges we ran into

1. Preprocessing in OCR
2. Pretrained model for Scene Description
3. Delay less speech to text conversion and vice versa
4. High training and processing time