V

Virtual Jukebox

Leveraging computer vision to convert facial expressions to drum beats to help visually challenged train their expressions and feel better with the help of music.


The problem Virtual Jukebox solves

Around 300 million people around the world are visually challenged. Facial expressions play a powerful role in social interactions from birth to adulthood. Music is something that is loved by everyone, we wanted to help the visually challenged people to express their emotions in a better way of using music. One of the problems the visually challenged face is to train themselves for these emotions. Babies who are born visually impaired, has no way to understand how to show emotions. This music jukebox is to help train their emotions for better expressiveness. If a visually impaired person is happy, the drum beats will help him/her understand the intensity of their emotions.

Challenges we ran into

  • Speed of processing the facial expression in a web browser
  • Creating a visually appealing user interface along with app
  • Streaming the screen of another user to enable the collaborative music creation

Technologies used

Discussion