. . AI: Apple Announces New Models and We Find Out About An Acquisition (3.15.24)

Friends, in today's AI news, Apple is finally making waves in the AI world with an acquisition and the announcement of new multimodal models. We also cover an open-source dataset that could disrupt the web development world, new reasoning frameworks and an insightful report on the open source AI ecosystem.

We work hard to bring these stories on weekdays in mere hours. If your peers like being this close to the frontier, please share this newsletter with others who might also appreciate it!

Now here’s today’s news,

Marshall Kirkpatrick, Editor

Apple's Team Develops Advanced AI Models

First impacted: Apple developers, AI researchers
Time to impact: Medium

Apple's team says it has developed a new set of Multimodal Large Language Models (MLLMs) named MM1, which they reported excelled in pre-training metrics and showed strong performance after supervised fine-tuning on various multimodal benchmarks. Senior co-author Alexander Toshev said on X, "more importantly, we give insights that hopefully will help folks out there build such models." According to their research, the key to optimal results lies in its architecture. By integrating image captions, mixed image and text data, and text-only information, the system can capture and analyze a wide spectrum of inputs. This multimodal approach allows the architecture to leverage the strengths of each data type, enhancing its ability to understand and generate more accurate and contextually relevant responses. [MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training] Explore more of our coverage of: Apple AI, Multimodal Language Models, Machine Learning Performance. Share this story by email

Apple Quietly Acquires DarwinAI

First impacted: Apple developers, AI researchers
Time to impact: Short

Apple Inc. quietly acquired Canadian AI startup DarwinAI earlier this year to enhance its artificial intelligence capabilities and is speculated to be under $100 million. DarwinAI specialized in computer vision but hadn't launched any products yet. Apple has recently been scrutinized in the media for its lack of presence in the AI sphere and this acquisition could be seen as a way to shore up its position in the competitive tech industry. [Apple Acquires AI Startup DarwinAI] Explore more of our coverage of: Apple Acquisition, AI Startup, Computer Vision. Share this story by email

Quiet-STaR Enhances Language Models' Reasoning Abilities

First impacted: AI researchers, AI developers
Time to impact: Medium

"When writing and talking, people sometimes pause to think." That's the inspiration for the creators of Quiet-STaR, an enhanced version of the Self-Taught Reasoner (STaR), who have developed a language model that they say can reason more effectively and on a larger scale. According to the developers, this model has improved zero-shot performance on GSM8K (5.9%→10.9%) and CommonsenseQA (36.3%→47.2%) without any specific fine-tuning for these tasks. Zero-shot learning is important because it enables models to operate effectively in problems they have never encountered before or have no training data on. [Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking] Explore more of our coverage of: Language Models, AI Reasoning, Quiet-STaR. Share this story by email

WebSight Dataset Helping Turn Screenshots to HTML Code

First impacted: Web Developers, AI Researchers
Time to impact: Short

Hugging Face has launched WebSight, an open-source dataset comprising 2 million pairs of HTML codes and corresponding screenshots, to support research in Vision Language-Models (VLMs). The dataset's creators say it has been used to fine-tune a basic VLM and the model can now convert webpage screenshots into functional HTML code. [From screenshots to HTML code: Introducing the WebSight dataset] Explore more of our coverage of: Open-Source Dataset, VLM Research, Web Development Efficiency. Share this story by email

Open Source AI Ecosystem Analysis and Trends

First impacted: AI developers, Analysts
Time to impact: Short

Chip Huyen has posted a sweeping analysis of the open source AI ecosystem and has concluded that it's seeing a surge in application creation, with over 20,000 developers contributing to 845 generative AI projects on GitHub. The study revealed that the most popular types of open-source AI applications are coding, workflow automation, and information aggregation, with more than half of these applications being hosted by individuals rather than organizations. Check out the link for the graphs and breakdowns across the various segments. [What I learned from looking at 900 most popular open source AI tools] Explore more of our coverage of: Open Source AI, AI Development, China's AI Ecosystem. Share this story by email