The problem Samarpit App (समर्पित) solves

👵👴 Many elderly people in tier-2,3 rural towns and villages can't read and write english. And they are rely on someone else's assistance for reading important documents like Bank Notices, Government Schemas, etc. Also, there is a lack of educational programs catering to this age group, leaving them dependent on external assistance for their reading needs.

Solution & Value Proposition

📱 Samarpit is a mobile application dedicated to the elderly people who can't read & write. It offers a digital assistant in their pocket, capable of reading English text and conveying in their native language and facilitating verbal 1 on 1 interaction with the content. With Samarpit, elderly users can easily access important information and have their questions answered without relying on others.

Key Features

✨ OCR: Samarpit uses Google's ML Kit OCR to extract English text from images, such as letters or documents.

✨ Generative AI Model (ChatGPT): The extracted text is processed by a generative AI model (ChatGPT) to generate a simplified and easy-to-understand summary for conversations.

✨ Translation Service & Text-to-Speech: Samarpit leverages Google Cloud Translation to automatically translate the summary into the user's native language.

✨ Speech-to-Text Interaction: Samarpit enables users to ask follow-up questions about the contents of the paper using their native language. The app employs Google's ML Kit for Speech-to-Text conversion to transcribe spoken queries.

✨ Contextual Answer Generation: Samarpit uses generative AI models (LLM) to provide contextual answers in the user's native language, ensuring relevant and meaningful responses.

Challenges I ran into

While building the protocol toughest part was to integrating different services togather to make the full flow working, and additonally the support for multiple Indian languages took the most time.

Now the app supports engaging in contextual conversation to english texts with Hindi, Bengali, Gujarati, Kanada, Marathi, Telugu, Tamil, Urdu