The problem Forest AI solves

AI innovation is measured using a set of well-known benchmarks and a publication of a model that does considerably better on them can wipe out even a $1tn from the stock market. That's what happened when DeepSeek-R1 benchmarks were published. Yet these benchmarks, that everybody so heavily relies on, our fundamentally flawed (public benchmarks get overfitted on, and private benchmarks can't be trusted not to be manipulated by controlling entities). That's why news headlines like that appear systematically:

• "Chatbots Are Cheating on Their Benchmark Tests" (The Atlantic),
• "Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates" (Cornell University),
• "AI agent benchmarks are misleading, study warns" (VenctureBeat) or
• "Meta exec denies the company artificially boosted Llama 4's benchmark scores" (TechCrunch).

Non-robust and non-transparent benchmarking clearly throttles AI innovation progress.

Forest AI is a Web3 Ratings Agency for AI with a market of competing AI providers that get rewarded for finding product-market-fit and for having high quality / performance.

This is accomplished by creating a permisionless way for people to define and create their own competitions (called Protocols) that compete for Forest AI emissions based on revenue they're bringing in. Each Protocol consists of an Owner (who created and defined the rules of the competition), Providers (who offer AI Services inline with Protocol's definitions) and Validators (who check the quality of Providers' offering as per Protocol's requirements). Those who do the best job get the biggest shares of Protocol's emissions.

The core of the Validators' system is a novel benchmarking framework developed with collaborators from top AI institutions (e.g. MIT, Princeton, Near AI) that doesn't recycle data but rather continuously creates new datasets so overfitting on previously seen data is not possible. The system provides full transparency. The Actors in the system are incentivised with Forest AI's tokenomics layer.

Such a system has a real chance of accelerating AI innovation and producing real value while preventing tokenomics pitfalls that allow oligarchies of large token holders to control networks to their benefit like in Bittensor.

Proof of Mainnet Deployment + 1 TX:

• https://basescan.org/tx/0x066ebed5d77f9d0704aec9fabc053846af58c47b1f00acef2713a569748b58b1
• https://basescan.org/tx/0x47064d9762283875b3d074d72f80a177a6831bc9f1f277d1b9f4344ea9f77d11
• https://basescan.org/tx/0x437079f637f62a220df1b40a007fd239044711357ef0722551bb5abbda21020b
• https://basescan.org/tx/0x0c64627e973b742b74bd1f7e5199d97786c0af5438674fb4f360d5440fd7a783
  Links to testnet contracts available in the github repo as part of the README:
• https://github.com/Forest-Protocols/forest-AI/tree/main/contracts#base-sepolia

Challenges we ran into

The biggest challenges we faced were around iterating on a number of ideas before we found this one which is actually worth building.

However it gives us a bunch of satisfaction seeing the excitement in people's eyes when we pitch Forest AI because it clearly shows how many people care about fixing AI benchmarks. Numbers speak for themselves as well. Developers immediately started integrating with our tech: Bittensor developers, Web2 AI startups, academic researchers. We already have 12 Protocols with 16 Providers live and growing.

Protocol design is also intellectually stimulating and not easy at all. Some of the attack vectors we needed to take into account were e.g.:

• Sybil Attack: Providers may try to cheat the token reward mechanism by creating many fake accounts and buying from themselves cycling the money to increase the Protocol's revenue artificially.
• Validator Vote Copying: Some might try to game the system to get emissions without actually doing the work of evaluating results (like creating new test data) but instead simply waiting for other validators to do this work and then copying their vote vector.
• Validator-Provider Collusion: A group controlling both a Validator and a Provider may attempt to extract unfair token rewards by scoring themselves unfairly. In prior AI Crypto projects this kind of self-dealing was rampant and made it unattractive for fair players to compete.

For Sybil Attack we are utilising the following methods to counter this possibility: Shannon Entropy over Purchase Vector, Off-chain Verifiable Identifiers, Proof of B2B Transaction and New KYB Onboarding, Chain Analysis Clique Detection.

For Validator Vote Copying we are utilising a Commit-Reveal mechanism that forces Validators to first commit to benchmark results and only later reveal the results together with all other Validators. In that way the results of others and not visible to the one wanting to copy results over as his own.

For Validator-Provider Collusion we mitigate this issue by requiring all Validators to keep a publicly auditable log of every score they have given for every prompt to every Provider. If unfair scoring of one Provider is suspected any token holder can request an audit and a DAO vote to slash the Validator's collateral and simultaneously render the Validators' votes ineffective.

Proof of Mainnet Deployment + 1 TX:

• https://basescan.org/tx/0x066ebed5d77f9d0704aec9fabc053846af58c47b1f00acef2713a569748b58b1
• https://basescan.org/tx/0x47064d9762283875b3d074d72f80a177a6831bc9f1f277d1b9f4344ea9f77d11
• https://basescan.org/tx/0x437079f637f62a220df1b40a007fd239044711357ef0722551bb5abbda21020b
• https://basescan.org/tx/0x0c64627e973b742b74bd1f7e5199d97786c0af5438674fb4f360d5440fd7a783
  Links to testnet contracts available in the github repo as part of the README:
• https://github.com/Forest-Protocols/forest-AI/tree/main/contracts#base-sepolia