The problem Audio to ISL Translator solves

According to 2011 census, it is known that an estimated 18 million people in India are deaf. In spite of this, it's a struggle for many deaf children to avail appropriate education in India, a country where deafness and disability in generally has been underreported and underserved. Although sign language is used across the world to bridge the gap of communication for hearing or speech impaired which depend mostly on sign language, there are not enough efficient models that convert audio to sign language. Indian Sign Language (ISL) is a predominant sign language in South Asia. The Deaf communities of the Indian subcontinent are still struggling for ISL to gain the status of sign language as an indispensable language considering the fact that majority of such people do not understand other sign languages and very limited work has been done in computerization of ISL. Our project focuses on creating an audio- sign language converter that recognizes and comprehends speech and can perform the recognition process through matching the parameter set of the input speech with the stored templates to finally display the analogous Indian sign language in image or GIF format. This project will help in reducing
dependency of deaf and hard of hearing people on interpreters.

Challenges we ran into

Record audio in any Indian language- We solved this issue by giving the user a list of languages and accordingly user can choose and give speech input in any Indian language.

For mapping the speech input we had database in English small case language. So we first translated the input in Indian English using python translate.

Corresponding word/sentence not present in dataset- In this case we processed the lowercase translated input and obtained words from sentences as well as letters from words. These words/letters are being displayed in carousel so that user understands the ISL even if given input is not present in dataset.
We used carousel because it
-It diaplays images/gifs within particular time interval automatically and also user can navigate through it
-It will contain all output in given sequence of input speech and users can navigate through it according to their comfort