. . AI: Two New State of the Art Models, Build Your Own Robot and Models Get Emotional (3.28.24)

Friends, in today's edition, we're diving into some fascinating advancements in the world of mid-sized models, where the blending of different model architectures is not only challenging established leaders (like Mixtral) but sparking a wave of innovation reminiscent of natural evolution. (Except we're leaping forward by thousands of years in a matter of months!)

Other highlights include an open-source project that lets you build your own robot, significant strides in making models more compact and efficient, and the latest developments in AI that can detect and even respond to human tones (well, kinda). Enjoy!

-Marshall Kirkpatrick, Editor

Databricks Launches DBRX, Claims Superiority Over Other Open-Source Models

First impacted: Data scientists, AI researchers
Time to impact: Short

Databricks has launched DBRX, an open-source LLM that the company says surpasses all other established open-source models in standard benchmarks. Built on the MegaBlocks research and open-source project (which is a lightweight library for mixture-of-experts (MoE) training), DBRX is an MoE model that Databricks says allows for faster processing and the training of larger models, outperforming models such as LLaMA2-70B, Mixtral, Grok-1, and GPT-3.5 in most tests. The model is available to test in chat on HuggingFace and is being benchmarked in the blind test at LMSYS. [Announcing DBRX: A new standard for efficient open source LLMs] Explore more of our coverage of: Databricks DBRX, Open-Source LLM, Mixture-of-Experts Model. Share this story by email

AI21 Labs Launches Jamba, a Hybrid SSM-Transformer LLM

First impacted: AI developers, AI researchers
Time to impact: Short

AI21 Labs has unveiled Jamba, a cutting-edge hybrid model that integrates the benefits of Structured State Space Model (SSM) technology with the Transformer architecture, enhancing the model's ability to process large sequences of data efficiently. This combination allows Jamba to tackle complex tasks that were previously challenging for either LLMs or SSMs alone. Claiming to be much bigger, better, and cheaper than Mixtral, Jamba operates with 12 billion active parameters in a Mixture of Experts framework, allowing it to manage up to 140K tokens of context on a single 80GB GPU and making it a versatile tool in the landscape of machine learning models as it offers both high efficiency and a broad context windows for advanced text processing tasks. [ai21labs/Jamba-v0.1 · Hugging Face] Explore more of our coverage of: AI21 Labs, Language Learning Model, Jamba. Share this story by email

K-Scale Labs Develops Affordable, Open-Source Humanoid Robot

First impacted: Robotics researchers, 3D printing enthusiasts
Time to impact: Short

K-Scale Labs has launched an open-source humanoid robot, Stompy, which can be constructed by anyone with a 3D printer. It's 4 feet tall and has a claw gripper. The company says the robot, costing under $10,000 to produce, is designed to address the lack of large, real-world datasets in robotics. [Launch YC: K-Scale Labs 🤖 Open-source humanoid robots for everyone | Y Combinator] Explore more of our coverage of: Open-Source Robotics, 3D Printed Robots, Robotics Datasets. Share this story by email

Mobius Labs Launches HQQ Model Quantizer

First impacted: AI developers, AI researchers, Data scientists
Time to impact: Short

Mobius Labs has launched a model quantizer, Half-Quadratic Quantization (HQQ), with promising results around speed and accuracy. Quantization is a compression technique used to reduce the model's size and increase processing speed, without greatly compromising its accuracy. The company also states that HQQ is compatible with PyTorch 2, can be installed via pip, and supports a range of models including Llama, Mistral, Mixtral-8x7B, Phi + Phi_opt, and ViT-CLIP. [GitHub - mobiusml/hqq: Official implementation of Half-Quadratic Quantization (HQQ)] Explore more of our coverage of: AI Model Quantization, Mobius Labs, Pytorch 2 Compatibility. Share this story by email

Hume's Empathic Voice Interface Designed to Understand User's Tone

First impacted: Developers, Speech Developers
Time to impact: Short

Hume is set to launch its AI, Empathic Voice Interface (EVI), in April, which the company says can comprehend and respond to users' tone of voice. Check out the demo of it as a work in progress, it's both impressive and weird. [Hume AI] Explore more of our coverage of: Hume AI, Empathic Voice Interface, API Development. Share this story by email

That’s it! More AI news tomorrow.