The problem Lecturizer solves

Problem:

In the current scenario, the world is moving fast towards much more online existence. Students are taking classes online on video calls. They take classes from far situated teachers. In this online busy schedule students tends to miss classess. The pending classes gets piled up everyday due to some other work or connectivity issues. Same happens with faculties where they miss certain conferences and external lectures. This video recordings can be 2-3 hrs long wasting long much time of a student and teacher when they go through them.

Idea:

Lecture recordings are most probably available on the educational suites and can be easily downloaded. The student will spend 3 hrs on a 2 hr videos. Whereas he/she can read the textual insights of the lectures in about 30-40 minutes. And if it content is more important than can take 1 hr maximum to study. So by using textual insights of video lecture,1-2 hrs can be saved.

Technical Solution:

Lecturizer is a easy to use desktop application where student or teacher can upload the lecture video(mp3+) recordings. In couple of seconds a textual report is generated which contains useful insights of lectures. This textual insights can be :

• Transcripton of the lecture recording
• Summary of the lecture
• Important/hot terms used in the lecture

This platform comes as a desktop application which uses flask to open a local server in any browser. All the machine-learning models run on the local machine. Only google speech-to-text library for transcription ueses external APIs. We used data science concepts like bag of words, cosine similarity, Tf-Idf, etc. and coded them all using python.

Future Aspects :

There will be focussed on developing algorithms for adding features to the report like :

• Technical defination of important words with blogs link
• Question answers discussed in the lecture
• External questions for practise based on the lecture.

Challenges we ran into

Speech-to-text :

We need to convert the video lecture's audio to text format so that we can analyze them and create meaningfull insights. For that we struggled with many external APIs like AWS transcribe, GCP speech-to-text, speechly, IBM watson, etc. This services were good but paid with less free trial. After searching a lot we decided to use SpeechRecognition library in python which uses google pre-trained models. It was free but medium level accuracy. But it worked for our project. For good accuracy we may use gcp Speech-to-text API which is paid.

Server and Storage :

We decided to make desktop application. Before that we struggled with deploying machine learning models to servers and video storage because a single video can be upto 100-200 mbs which can exhaust our free trial heroku servers and even aws s3 storage.

Framework Decision :

For desktop application we first tried Electron + Nodejs. But we ran into inte process commuinication problems which were taking too long to solve. Given the time constraint we quickly took a decision to try flask and it worked for us. Desktop integration was hard as we were not able to generate .exe file of our flask app. We solved this by generating batch files for installation and application running.