Equilibrium Infra Bulletin #23: Verifiable Inference, Sui's New Consensus-Mechanism, Near Chain Signatures, and more...

Equilibrium Labs builds the state-of-the-art of decentralized infrastructure. We are a global team of ~30 people who tackle challenges around security, privacy, and scaling.

🔍 First On-Chain LLM - Ethereum Speaks Its First Words

⚡️ Topic Summary

• Modulus Labs recently announced that they had generated a ZK proof of GPT2-XL - a 1.5bn parameter model - and verified it on Ethereum. The one billion parameter threshold is generally considered the complexity regime where LLMs begin to be useful, which the GPT2-XL exceeds. Its architecture is also relatively straightforward with 48 decoder blocks, making it easier to turn into circuits and an overall good LLM to try to ZK prove. While previous attempts have focused on far smaller models or skipped the on-chain settlement altogether (LLM results have never been settled to Ethereum), Modulus wanted to prove that this was possible.

  

  The two questions that Modulus set out to answer (source)

• A key part of proving a 1bn+ parameter scale model was using a custom prover. While most previous attempts have tried to prove models in Risc0, a custom-built prover yields much better results. Modulus used Halo2 (a widely used generic ZK prover) and initially attempted to prove each decoder block separately (48 proofs), but just one of these ~30M parameter blocks already overwhelmed the Halo2 prover. To make proving feasible, they had to break down the blocks into 3 sub-blocks (total of 48 * 3 = 144 sub-blocks) each representing ~10M parameters worth of computation. Recursive aggregation was used to compress results and make the final on-chain settlement cheaper.

• The proving was done on a 128-core CPU and 1TB RAM machine. Some stats regarding performance:

  • It took over 91 hours to generate the proving keys on a single machine and these 144 proving keys occupied a disk space of over 10TB.

  • The total proving time of the 144 sub-blocks took just under 90 hours (when run on the same single machine).

  • The total time for the whole operation from start to finish was 200+ hours

  • Compared to inference alone, the overhead from ZK proving was 1,125,761x (yes, more than 1m x higher).

🤔 Our Thoughts

• This is a big moment for verifiable inference as it’s the first time a result of a model of this size has been settled on Ethereum (an order of magnitude larger model than previous attempts). What Modulus proved is that it’s feasible to do verifiable LLM inference for large models and verify the output on Ethereum, but the cost of proving and the time it took is far from ideal. To scale verifiable AI, we need further specialization and there is a lot of optimization to do. Still, this is a big step in the right direction and a commendable achievement!

• At first glance, combining LLMs with ZKPs seems like a bad idea due to the size of the LLM models and the overhead of ZK proving. However, verifiable AI inference is important as it gives a (cryptographic) guarantee that a specific model was used for inference (more information here). This has implications for many different use cases:

  • Real-world: With more AI models advancing and being deployed in a wider capacity across industries (legal, education, medical…), it’s becoming increasingly important to enable verifying that the correct model was used. In addition, more ML-powered APIs increase the importance of verifiable inference.

  • Crypto-specific: Verifiable inference can help smart contracts access AI without requiring a third party to monitor it (such as a watcher network). This enables things like smarter defi with automated rebalancing or risk management.

💡 Research, Articles & Other Things of Interest

🤓 Presentation about Mysticeti, Sui’s new DAG-based consensus with lower latency (based on this paper released earlier).

📚 An overview of the current state of Anoma and how it can interact with Ethereum (and other existing networks).

📚 Chain Signatures launch on NEAR, which enable transactions on any blockchain from a NEAR Account.

🎧 A debate on cryptographic (ZK) vs cryptoeconomic security (summary here).

🎧 Whiteboard sessions about relevant blockchain concepts (Blobstream, Slinky, etc.)

🔥 News From Our Partners

⛴️ Gevulot Launches Its Devnet: 16,000+ whitelisted users can now start deploying arbitrary provers & verifiers and run workloads to generate & verify proofs. You can still sign up and get access to the devnet (instructions here). This is a big step forward for decentralized proving and an important testing phase for Gevulot to guide further development as they continue to build towards an incentivized testnet later this year (and eventually mainnet). Gevulot was incubated within Equilibrium Labs.

🚀 Membrane Finance Launches EUROe Ramp: EUROe Ramp provides a simple API for creating bank transfers to and from users' non-custodial wallets, providing regulated DeFi protocols with seamless connectivity to banking rails. EUROe is fully compatible with the upcoming Markets in Crypto-Assets (MICA) Regulation. Membrane Finance is part of Equilibrium Group and a sister company to Equilibrium Labs.

🤌 Personal Recommendations From Our Team

📚 Reading: The Dispossessed - Ursula K. Le Guin: If you’re into sci-fi which describes civilizations with totally different socio-economic structures than our own ("anthropological" sci-fi?), this might be for you. Originally recommended in a Slack thread several years ago that recently resurfaced.

🎧 Listening: Snarky Puppy Keyboardist Hears Dua Lipa For The First Time: What happens when Justin Stanton (keyboardist for Snarky Puppy) hears Dua Lipa’s "Don't Start Now" for the first time (with only drums and vocals)? Pure magic if you ask us!

💡 Other: Smart Words From Smart People: A collection of quotes with timeless wisdom, curated by Morgan Housel.