The problem VideoDex solves

The problem we started out to solve was to be able to provide the ability to search through a video through it's content. This would require semantic knowledge about the contents of the video. This would improve general search in video platforms. One critical applications of such a system would be in realtime security monitoring, where the video can be analysed at it streams and the relevant authorities can be alerted based on any suspicious acitivities. However, another interesting application of understanding the semantic meaning of frames in the video is that we can pair ads based on the current content in the video. This can improve the impact of ads on the users due to the apt placement of the ads. So for example if the video currently contains a person eating a pizza, it can be a perfect opportunity for a pizza seller to place the ad there. This can help making ads more relevant and also significantly increasing impact of the ads since they are placed with content that is relevant to them.

Challenges we ran into

One of the major challenges that we ran into was coming up with a robust and efficient algorithm that can analyze these videos to extract the information from videos. We went through several iterations of the algorithm to reach a reasonably fast and accurate solution. Another challenge was to align the advertisements to the specific position in the video. And along with these there were a number of associated technical challenges to get it working in a reasonable amount of time.