Created on 30th December 2020
•
There are often instances when we see a food item and want to recreate it at home. The names of these items may not always be known to us; instead, we have a picture we took of the dish when we saw it. One would then search the internet using different keywords until we finally find the dish after going through numerous pages. Rather than taking so much effort to find out even the name of the dish, let alone the recipe, wouldn’tit be much easier to simply send a photo when we see the dish and receive a list of links which have the recipes? This project does exactly that. The main characteristic which makes it so user friendly is the use of a WhatsApp chatbot for interacting with the user.
Everyone is well versed with WhatsApp, and use it constantly to communicate with friends and family. Companies have also started using WhatsApp bots to receive orders from customers. The advantage of WhatsApp is its popularity and ease of use. Almost everyone uses WhatsApp which adds to its network and reach. However, less people are as acquainted with the internet or how to search for something. A lot of time is wasted scouring different websites, looking through articles, blogs and YouTube videos to find a simple recipe. For example, suppose you taste a slice of cake which you want to bake at home. Rather than searching the net for the name of the cake and going to different websites to find the recipe, life would be a lot easier if you could just snap a picture of the cake and send it to a WhatsApp number which would return the top or most popular recipes. Not only do you save time reading through different pages, but the effort required is also reduced.
The biggest challenge we faced was getting a good dataset containing food images and training it to a decent accuracy. When we initially tried to create a CNN model from scratch, it only gave a mere 47% accuracy so we thought of using a pretrained model and used transfer learning to get a model with a better accuracy. This model gave an accuracy of 84% which even though is not so brilliant, wasn't bad either as this meant it gave the right answer 8 out of 10 times.
Another difficulty we faced was getting a suitable free third party api to connect our backend to whatsapp. Twilio was the only decent one which we came across. Even twilio asked for payment if we wanted to make full use of it but for the purpose of this hackathon the open source free version was good enough.