. . AI: Mistral Launches APIs, EU AI Act advances, and much more (12.11.23)

Friends, it was a big weekend in AI. Mistral's official launch of its Mixture of Experts model and APIs are dominating the conversation today, but there are quite a few other important stories being discussed as well! All of these are worth a scan.

There's some powerful stories bubbling up already for tomorrow too, so watch this space!

As always, this is a collection of the stories most-talked-about according to a weighted analysis of AI community engagement. I hope some of these stories will help advance, refine, or inspire your work. Please help spread the word about our little newsletter, I really appreciate you doing so - and welcome to everyone who’s joined recently!

Now here’s the news,

Marshall Kirkpatrick, Editor

EU Strikes Landmark AI Regulation Deal

First impacted: Industry legal analysts
Time to impact: Short

The European Union says it has reached an agreement on the AI Act, a law designed to regulate AI. The law, which encompasses risk classification, transparency rules, and financial penalties for noncompliance, requires tech companies to disclose data and conduct rigorous testing, particularly for high-risk applications, and is expected to be approved by the European Parliament before May, with full implementation taking two years. Yann LeCun comments on the open source angle on X. [E.U. reaches deal on landmark AI bill, racing ahead of U.S.] Share by email

Mistral AI Launches High-Performing Mixtral 8x7B Model

First impacted: AI developers, AI researchers
Time to impact: Short

Mistral AI has launched Mixtral 8x7B, an open-weight model that they say outperforms Llama 2 70B and matches or surpasses GPT3.5 in most benchmarks. Licensed under Apache 2.0, the model is said to have 6x faster inference, supports multiple languages, and can be fine-tuned to follow instructions, achieving a score of 8.3 on MT-Bench, a tool used to evaluate the conversational and instruction-following abilities of models. [Mixtral of experts] Share by email

Mistral AI Launches Open Generative Models Platform

First impacted: AI Developers, Beta Testers
Time to impact: Medium

Mistral AI has launched its initial platform services, which include three chat endpoints for text generation and an embedding endpoint, according to the company's blog post. The platform, which includes endpoints mistral-tiny, mistral-small, and mistral-mid, each with different performance and price trade-offs, is currently open for beta testing, with a wider release planned. [La plateforme] Share by email

HuggingFace Explainer: Mixture of Experts Models

First impacted: AI researchers, AI engineers
Time to impact: Medium

The launch of Mixtral 8x7B, a Mixture of Experts (MoEs) model, has sparked interest in the AI community due to its claims of faster pretraining and inference compared to dense models. HuggingFace has published an explainer blog post, including discussing tradeoffs: MoE's face challenges such as high memory requirements and difficulties with fine-tuning, as all experts are loaded in memory. [Mixture of Experts Explained] Share by email

RunwayML Launches General World Models Research

First impacted: AI developers, video content creators
Time to impact: Medium to Long

RunwayML has launched a long-term research project focused on developing General World Models (GWM), an AI system designed to understand and simulate the visual world and its dynamics. The GWM, which builds upon the Gen-2 video generative system, aims to create an internal representation of an environment, simulate future events within it, and address the complexities of camera or object movements. There's a cute 3 minute video explaining the ambitious program. [Introducing General World Models.] Share by email

Edinburgh University's Yao Fu Proposes Path to 100x Cost Reduction for LLMs

First impacted: AI researchers, AI engineers
Time to impact: Medium to Long

Yao Fu from the University of Edinburgh has published a long blog post of methods he says could decrease the deployment cost of LLMs by a factor of 100. His approach involves an in-depth analysis of transformer inference optimization, considering hardware specifications, MLSys methods, model designs, and decoding algorithms, with particular emphasis on the importance of GPU architecture and improving flop utilization for increased efficiency. [Towards 100x Speedup: Full Stack Transformer Inference Optimization | Built with Notion] Share by email

Lean Co-pilot Launched for Mathematical Proof Collaboration

First impacted: Mathematicians, Computer Scientists
Time to impact: Medium to Long

Prof. Anima Anandkumar and her team at Caltech have launched Lean Co-pilot, a tool designed to facilitate collaboration between humans and LLMs in writing formal mathematical proofs. The Lean package, which is integrated with Lean's VS Code workflow, uses LLMs to suggest proof strategies, allows for human intervention and modification, and is expected to initiate a positive feedback loop where automated proofing contributes to improved data, subsequently enhancing the mathematical capabilities of LLMs. [LLMs as Copilots for Theorem Proving in Lean] Share by email

"Hallucination Problem" in Large Language Models: A Feature, Not Bug

First impacted: AI developers, AI researchers
Time to impact: Medium to Long

Andrej Karpathy, in a long tweet, addressed the so-called "hallucination problem" in LLMs, arguing it's not a flaw but a feature. "In some sense, hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful." Karpathy says there are ways to mitigate these "hallucinations", like using Retrieval Augmented Generation (RAG) to more strongly anchor the responses in specific, data, but ultimately it's not the LLM but the application built on top of it that will need to account for any inaccuracies. [via @karpathy] Share by email

That’s it! More AI news tomorrow.