Meeting Paw
Meeting Paw is a meeting assistant that can keep track of key points in real-time, engage and respond with audience and shared screen, and provide a virtual whiteboard for ease of showing id
Created on 9th November 2024
•
Meeting Paw
Meeting Paw is a meeting assistant that can keep track of key points in real-time, engage and respond with audience and shared screen, and provide a virtual whiteboard for ease of showing id
The problem Meeting Paw solves
Compared with other AI conference assistants on the market, Meeting Paw has made many innovations.
Firstly, Meeting Paw's unique real-time key points display and meeting summary can enable participants to better catch up on the progress of the meeting and be aware of the ideas of the meeting content. Meeting Paw will be a powerful meeting partner for the participants of large-scale cross-disciplinary meetings.
Secondly, Meeting Paw innovatively allows AI assistants to participate in the meeting as participants, integrates the powerful analytical and computational capabilities of the AI model and rich Internet resources, and provides opinions and references from multiple angles and fields. Meeting Paw can often play a role in brainstorming.
Thirdly, Meeting Paw integrates the virtual whiteboard function into the shared screen for the first time. Meeting Paw can quickly and accurately identify the gestures of hand, which can be shared with other participants and AI meeting assistants along with the shared screen. The virtual whiteboard function can help participants to mark the contents of the meeting in various environments.
In addition, Meeting Paw also cares about the experience of people with disabilities and is committed to providing them with the simplest and most efficient AI conference assistant. For participants with disabilities or attention deficits, Meeting Paw provides real-time follow up of meeting contents and key points summary, and the built-in AI meeting assistant can communicate through voice, which is convenient for participants in the meeting to the greatest extent.
We believe that the appearance of Meeting Paw will provide participants with a brand-new meeting experience, which will make your meeting easier and more efficient.
Challenges we ran into
In the process of project progress, the solution of difficulties has promoted the perfection of our project.
In the training part of the gesture recognition model of a virtual whiteboard, we originally planned to use the distance between the thumb and index finger to determine whether the pen is down. However, when the trained model is imported into Python, the key points are often too close to determine. So we changed our idea and trained the model that the index finger is the key point to make the gesture recognition more accurate and rapid and use the space bar as the control of if the pen is down.
The challenge we encounter also includes establishing an effective connection between search AI and voice AI. Because search AI does not support audio functions, we use another relay agent to solve this problem. In addition, the use of voice recording API is also very challenging, as voice is continuous data instead of chunking data.
In the future, we plan to make several improvements to our project. We aim to make our virtual whiteboard faster and smoother for users, and we also want to improve the accuracy of keypoint detection. Originally, we intended to enable the model to input multimodal data through retrieval-augmented generation (RAG) to enhance the AI's ability to retrieve data within the company's database. However, we have found it challenging to integrate this with the original AI agent.
Tracks Applied (2)
Best use of GitHub
GitHub Education
Best Project Built Using Gemini API
Google For Developers