ChatGuardAI
because every user deserves a guardian
Created on 17th December 2024
•
ChatGuardAI
because every user deserves a guardian
The problem ChatGuardAI solves
ChatGuardAI addresses the growing need for safer and more responsible online communication by providing real-time detection of harmful or suspicious messages across multiple platforms. It supports both English and Malay, including modern slang and cultural nuances, ensuring accurate monitoring. Unlike existing solutions, ChatGuardAI can analyze overall context, detect illegal activities such as suspicious transactions or dangerous threats, and integrate with Malaysian laws for proper reporting. It also generates detailed reports to help users understand why content is flagged as harmful. By offering cross-platform monitoring for posts, comments, and direct messages, ChatGuardAI promotes safer online spaces and responsible digital communication.
Challenges we ran into
Initially, the AI was unable to detect certain modern inappropriate words and slang curse words during our testing. These words, often used in informal chats, were not part of the pre-trained dataset. To resolve this, we trained the AI model further by feeding it updated datasets containing modern curse words and slang. We optimized the prompting in JamAI Base to ensure it could accurately interpret and detect these words in real-time conversations. This enhanced the detection accuracy significantly.
Next we faced challenges in getting the Telegram bot to send user messages as formatted strings to JamAI Base for analysis. This issue arose because of difficulties with message formatting and passing data through the bot's API. To overcome this, we referred to YouTube tutorials and GitHub repositories to find examples of Telegram bot integration. By importing and configuring Python’s logging module, we ensured that the bot correctly formatted incoming user messages as strings suitable for input into JamAI Base.
Technologies used