The problem Chain Speak solves

Navigating web3 can be intimidating for non-technical users, requiring familiarity with complex interfaces, transaction fees, and the nuances of DeFi protocols. Chain Speak simplifies this by allowing users to perform actions via speech, such as querying protocols, transferring tokens, or swapping/trading assets.

With Chain Speak, users can:

• Effortlessly query information about protocols or entire ecosystems (e.g., Polygon Stack, Coinbase Stack).
• Execute token transfers using simple voice commands, even referencing ENS or base names.
• Perform token swaps, bridging, and liquidity provisioning without needing to understand underlying mechanisms.
• Access DeFi functionalities like staking or providing liquidity on protocols like Aave or Uniswap.

This makes web3 accessible to everyone, removing technical barriers and enabling faster adoption.

Challenges I ran into

1. Using multiple LLMs: We utilized GPT-4o-realtime for user speech-to-speech interaction and instant responses, while leveraging the CDP SDK for AI agent calls. Orchestrating these models to work seamlessly posed significant challenges.
2. Custom Contract Calls: Integrating custom smart contract calls into the tool was tricky. The context within the LLM would occasionally get lost when processing larger instruction sets, requiring fine-tuning and optimization to ensure reliability.

Additional Features

Nothing was built before the hackathon, here are some of the features that we have built during the hackathon

• Real-time Polygon stack queries.
• Speech-to-speech integration using advanced transcription engines.
• Built custom adapters for using Uniswap to swap assets, bridging, and an adapter to integrate a custom contract.
• Added multi-language support for diverse user bases.