CLUTCH
Every word counts: BET ON IT
The problem CLUTCH solves
CLUTCH brings real-time betting to live streaming. Currently, betting on dynamic, split-second in-game events (like player kills on Kick streams) through visual and audio clips which then are relied on centralized, or manual resolution. CLUTCH automates this entirely:
Challenges we ran into
Our biggest challenge was the audio capture pipeline — we initially used Silero VAD (Voice Activity Detection) running client-side with ONNX Runtime, but it consistently detected 0ms of speech due to model input dimension mismatches and the Web Speech API failing to receive audio data from tab sharing. On top of that, React's useEffect cleanup cycle was destroying our bet resolution timer every time a Kick chat message arrived, meaning the timer to resolve bets literally never fired in busy chatrooms. We solved the audio problem by replacing the entire VAD pipeline with a simple MediaRecorder that records raw tab audio and sends it to OpenAI Whisper for server-side transcription, and we fixed the timer by switching from a fragile setInterval inside useEffect to a single setTimeout with useRef-based state access that's immune to React re-renders. Finally, we hit a Socket.IO silent payload drop where the 1.5MB audio blob exceeded the default 1MB maxHttpBufferSize, causing the client to silently disconnect — a one-line config fix that took significant debugging to identify.
Use of AI tools and agents
CLUTCH uses a multimodal, multi-agent AI pipeline to ensure cost-efficiency and on-chain verifiable accuracy:
Whisper: Transcribes the 10-second audio chunks on-chain to detect streamer commentary or game audio cues (e.g., "I got him!").
Then, an LLM uses the transcription and determines whether the user mentioned a certain word, gpt 5.2 mini.
Tracks Applied (1)