Bonetalk
When Voice Fails, BoneTalk Speaks.
The problem Bonetalk solves
BoneTalk addresses the communication barrier faced by people who cannot produce normal speech due to conditions such as larynx removal, ALS, throat injuries, or other voice-related disabilities.
It uses muscle signals captured through a wearable sensor, processes them with an ESP32 and machine learning, and converts intentional muscle movements into text or voice commands such as YES, NO, HELP, and WATER.
This provides a hands-free, low-cost and potentially portable communication method, helping users communicate more independently without relying on conventional speech.
Challenges we ran into
One of our biggest challenges was hardware reliability during the hackathon. Due to an accidental incident, our ESP32 prototype fell to the ground and continued working for a few hours before eventually stopping. This limited the amount of real-time hardware testing and improvement we could complete.
We also faced challenges with collecting reliable sensor data and achieving consistent ML predictions. Since muscle signals can vary between users and even between different attempts by the same user, calibration and data quality were important hurdles.
To overcome these challenges, we focused on improving the software pipeline, ML workflow, data processing, and user interface, while continuing to test the system wherever possible. Although we could not achieve everything we planned within the hackathon, these challenges helped us identify the key areas that need improvement for the next version.
Tracks Applied (1)
Hardware
Technologies used