. . AI: Google Unlocks 'Infinite Attention' (and Robots Playing Soccer) (4.11.24)

Friends, today's newsletter highlights Google's Infini-attention method, a technique they claim allows Transformer-based LLMs to handle unlimited inputs. We also share updated footage from Google of robots playing soccer, which comes highly recommended. Additionally, there's a story on Meta's efforts to evaluate a model's understanding of spatial settings. That's two robotics stories in one day!

Below we’ve got summaries of the top 3 stories, then headlines only for 3 more important ones. As always, these are the AI news stories the AI community is talking about most. Thanks for reading!

-Marshall Kirkpatrick, Editor

Google Introduces Infini-attention for Scaling LLMs

First impacted: AI researchers, AI developers
Time to impact: Medium

Google has launched a method, dubbed Infini-attention, to scale Transformer-based LLMs to handle unlimited inputs, even with limited memory and computation. The company states that this technique, which incorporates a compressive memory in the attention mechanism and includes both local and long-term attention mechanisms, has been effective in tasks such as long-context language modeling benchmarks, 1M sequence length passkey context block retrieval, and 500K length book summarization tasks with 1B and 8B LLMs. [Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention] Explore more of our coverage of: Google AI, Transformer-based LLMs, Infini-attention method. Share this story by email

Meta AI Launches OpenEQA to Evaluate AI's Spatial Understanding

First impacted: AI researchers, AI developers
Time to impact: Medium

Meta AI has introduced OpenEQA, a tool designed to assess an AI's understanding of physical environments through open-ended questioning. Dhruv Batra, a researcher at Meta AI, says that current vision and language models (VLMs) struggle with tasks requiring spatial comprehension. He hopes that OpenEQA will stimulate further research into improving AI's perception and articulation of the world it perceives. [fb.me] Explore more of our coverage of: Meta AI, OpenEQA Tool, Vision+Language Models. Share this story by email

Google's AI Trains Robots to Play Soccer Using Deep Reinforcement Learning (Again)

First impacted: Robotics Engineers, AI Researchers
Time to impact: Medium

Google's DeepMind team shared footage of robots playing soccer, leveraging a deep reinforcement learning-based system that they say allows humanoid robots to play soccer. If this footage looks familiar, it's because it is similar to a post from this time last year about DeepMind's efforts in the space. This time around, the robots seem to be much more agile and have a research paper accompanying the video. The robots trained in a simulated environment using deep reinforcement learning techniques performed better than those with a pre-programmed controller, while also showing the ability to anticipate ball movements and block shots from opponents. [Learning agile soccer skills for a bipedal robot with deep reinforcement learning] Explore more of our coverage of: Google AI, Reinforcement Learning, Humanoid Robots. Share this story by email

More headlines…

Cohere Launches Rerank 3 to Boost Enterprise Search

Explore more of our coverage of: Cohere Rerank 3, Enterprise Search, Retrieval Augmented Generation. Share this story by email

Y Combinator's W24 batch highlighted 247 companies presenting AI innovations, with Voice Agents, ChatGPT for Business, and Video Generation as top categories.

Explore more of our coverage of: Y Combinator W24, AI Innovations, Patient Handoff Enhancement. Share this story by email

Karpathy Explains His GPT2 LLM Training Process

Explore more of our coverage of: Andrej Karpathy, Large Language Models, PyTorch Optimization. Share this story by email

That’s it! More AI news tomorrow.