The problem Audience Data Analytics (ADA) solves

Existing approaches to know whether the user is present and active during a conference or class is to either switch on audio or video of the user, or using chat system. Switching on video is not that scalable to large groups and it require very large bandwidth, and difficult to monitor all the audience.

• The soultion we come up is our project Audience Data Analytics (ADA). This will track the user from the video feed locally. No video stream will be transmitted to video call. The local camera feed of the user is given to a pretrained ML model to analyse user presense. We are able to track, the presense of user, head movements of the user, and activity of the user. These data is then send to the host and we are able to see the metrics about the audience in a pie chart. Thus the active participation of users can be tracked and monitored in realt-time with the meeting without cost of extra bandwidth.

• With our project, conferences, talks, classes, exams could be monitered in realtime. We can create alerts, when the user is not present for atleast 2 minutes, see how much percentage of users are active and looking on screen, track their head movement.

• The project could be improved by adding more metrics and analytics features.

• We also made the android app with webview to support our web app.

Challenges we ran into

Major problems we face was to integrate ML model and video call facility side by side and work in realtime. We solved it finally. We use jitsi (open source video conferencing solution) and tensorflow js to work realtime. There where issues to integrate the metrics about user in realtime with data visualisation. We solved those, and used Firebase Real-Time Database to store and retrieve data in realtime for data visualisation.