SignBuddy
SignBuddy: Bridging the Senses, Empowering Lives. Our revolutionary app opens a world of communication for the deaf and blind, fostering inclusivity through intuitive interfaces and innovative feature
Created on 3rd March 2024
•
SignBuddy
SignBuddy: Bridging the Senses, Empowering Lives. Our revolutionary app opens a world of communication for the deaf and blind, fostering inclusivity through intuitive interfaces and innovative feature
The problem SignBuddy solves
Our app is a game-changer for both visually impaired and deaf individuals, helping them overcome significant communication barriers. We've built it using React Native, Expo, Google API, and Salesforce Blip Image Captioning Base API to ensure seamless performance and reliability.
Imagine being visually impaired and unable to fully grasp your surroundings. Our app steps in to provide a solution. By simply snapping a picture, users receive instant verbal descriptions of what's around them. Whether it's identifying objects, understanding signs, or describing landscapes, our app empowers visually impaired individuals to navigate with confidence and independence.
But we didn't stop there. We recognized the struggles faced by the deaf community in verbal communication. So, we integrated a speech-to-text feature that converts spoken words into text in real-time. This feature facilitates smoother communication in various settings, from everyday conversations to important meetings and public announcements.
In essence, our app is all about inclusivity and empowerment. By harnessing advanced technologies like machine learning and natural language processing, we're breaking down barriers and making the world more accessible for everyone. And with React Native and Expo, we've ensured that the app is user-friendly across different devices and platforms.
The Google API adds another layer of accuracy to our app's functionalities, ensuring precise image recognition and speech-to-text conversion. We're committed to ongoing improvement, constantly refining our app to meet the evolving needs of users and push accessibility standards forward.
In conclusion, our app isn't just about technology—it's about making a real difference in people's lives. By providing vital assistance in understanding and navigating their surroundings, we're helping create a more inclusive and equitable society for all
Challenges we ran into
One significant challenge we encountered during the development of our project was integrating the Salesforce Blip Image Captioning Base API seamlessly into our application. Initially, we faced difficulties in configuring the API endpoints and ensuring smooth communication between our app and the Salesforce platform. To overcome this hurdle, we engaged in thorough documentation review and reached out to the Salesforce support team for assistance. Through diligent troubleshooting and collaboration, we identified the root cause of the integration issue, which was related to authentication parameters. By adjusting the authentication settings and fine-tuning our API requests, we successfully resolved the issue and achieved seamless integration of the Salesforce Blip API into our application.
Furthermore, we encountered performance issues with the speech-to-text feature, particularly in handling large volumes of audio data in real-time. This resulted in delays and occasional inaccuracies in transcribing spoken words. To address this challenge, we optimized our speech-to-text implementation by implementing buffering techniques to manage incoming audio streams more efficiently. Additionally, we fine-tuned the parameters of the speech recognition model and optimized network communication to reduce latency. These optimizations significantly improved the performance and accuracy of the speech-to-text feature, enhancing the overall user experience.
Technologies used