The problem Envision Buddy solves

The problem:

Usage of AR in education is limited but it is expanding at a faster pace. It’s the age of learning more by visualization and less by reading. Nowadays educational institutions have upgraded their teaching methods to employ more AVs in the classroom. But these are in 2D and do not give freedom to the student to visualize it in the way they understand, a mindset that differs from student to student. Some may argue that he/she can refer to videos on YouTube or similar websites or even search on the Internet for the same. But at the same time, one must note that one cannot rotate or resize the 3D models alias the video on Youtube according to their wish. They must move the video back and forth to achieve this which can sometimes be frustrating. In this app, all a student needs to do is use his/her fingers to zoom (pinch) or rotate (swipe).

Our solution:

The prime objective of this project is to help students learn and understand concepts in a much better and streamlined manner (beyond what was taught using conventional classroom methods), quashing any doubts that linger in the minds of students due to the lack of visualizing capacity. We are going to achieve this using Augmented Reality and what better platform can we deploy it in than smartphones, which just isn’t a do-away these days. Students can visualize any model in 3D and also in any orientation that they wish using the app. While there may be many AR apps that can achieve this, the feature that makes our app stand out is that it can identify which model you may require for a particular concept. All you need to do is point the phone towards the book so that it can identify the concept and display the relevant AR model. No searches needed, no video scrubbing needed…all that's between you and your 3D model is just a single scan.

Challenges we ran into

The main challenege that we faced while implementing this app is the making of the Text Classification machine learning model. We first tried to make it ourselves using Natural Language Processing, but then we had to adopt a pre-trained BERT model and add our code to generate the API for the model. The second major problem that we faced was the hosting of the flask application on any of the cloud base services due to its large size. So, we had to finally settle with hosting the application on a localhost server using ngrok and had to update the API link every two hours.