. . AI: Claude 2.1, Orca 2, Lookahead decoding and more

Author

Marshall Kirkpatrick
November 21, 2023

Friends, it was an exciting day in AI today! A number of big announcements were made of cool new technology advancements. The new version of Claude is especially interesting. Even OpenAI shipped something new today.

As always, this is an analysis of the top stories being discussed by the AI community, summarized into a few sentences and edited by me, Marshall Kirkpatrick.

I hope you’ll consider sharing some of these stories with the data analysts or software developers or videographers in your life. If you click the “share by email” link at the end of each story, this is what pops up. An email ready for you to address to a friend, write your own thoughts at the top of, and send! Let me know how that works for you, I’d love your feedback, and your help growing our community of readers and thinkers.

And now, here’s today’s news.

First impacted: Data analysts, AI researchers
Time to impact: Short

Anthropic has introduced Claude 2.1, an AI model that the company says can handle a 200K token context window, equivalent to processing over 500 pages of data. According to Anthropic, the updated model has halved its hallucination rates, added a beta feature for tool use by API (add a calculator, web or database search, taking simple actions in other software), and comes with revised pricing. [Introducing Claude 2.1] Share by email

First impacted: AI researchers, Software developers
Time to impact: Short

Microsoft Research has introduced Orca 2, an open-source language model they assert is more effective at reasoning than earlier versions. They claim the model equals or outperforms models 5-10 times its size on intricate tasks, based on 15 varied benchmarks consisting of 100 tasks and over 36,000 unique prompts. [Orca 2: Teaching Small Language Models How to Reason] Share by email

First impacted: AI developers, Data scientists
Time to impact: Medium

A new decoding algorithm called lookahead decoding has been introduced by the Large Model Systems Organization, which claims the algorithm speeds up LLM inference. It does this by breaking the sequential dependency in autoregressive decoding (it "looks ahead" in parallel) and reducing latency by 1.5x to 2.3x. The team responsible for this development has also developed an implementation that works with HuggingFace's transformers, enabling users to improve performance with a few lines of code. [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding | LMSYS Org] Share by email

First impacted: Language learners, AI community members
Time to impact: Short

OpenAI has announced that its ChatGPT with voice feature is now available to all free users. Many people commended the company for shipping product admist recent chaos. Many other people note, and OpenAI has aknowledged, that ChatGPT and the API are experiencing a lot of errors right now. [via @OpenAI] Share by email

First impacted: AI developers, Video content creators
Time to impact: Medium to long

Stability AI has introduced Stable Video Diffusion, its first base model for generative AI video, and is currently seeking feedback to fine-tune the model before its official launch. The company also revealed a new Text-To-Video interface (waitlist open now) and continues to grow its collection of open-source models. [Introducing Stable Video Diffusion — Stability AI] Share by email

That’s it! More AI news tomorrow.