. . AI: Gemma Gets Huge Context Window and Eleven Labs Creates Music (5.9.24)

Today's news was a little light, but the highlight was Eleven Labs showcasing a new tool that transforms your lyrics into music across various genres. We also see two new tiny open-source models pushing the frontier forward.

— Sasha

Gemma 2B Model Expands Context Window to 10M Tokens!

First impacted: AI researchers, data scientists
Time to impact: Short

The Gemma (Google's open-source tiny model initiative) developers shared that they have launched Gemma 2B, which features a 10M context window and operates on less than 32GB of memory, as detailed in their blog post. They attribute this enhancement to the use of Infini-attention and activation compression techniques, along with a method that divides attention into local blocks with recurrence, efficiently managing the larger context. [mustafaaljadery/gemma-2B-10M · Hugging Face] Share this story by email

Eleven Labs Launches AI Music Model

First impacted: songwriters, music producers
Time to impact: Short

Eleven Labs has launched a new AI music model that creates songs from user-submitted lyrics and prompts, as demonstrated in a video they shared. The model produced a variety of musical styles and we recommend you check it out as it is surprisingly good! [via @ammaar] Share this story by email

Alignment Lab AI and Hive Digital Launch Buzz-8b-Large Model

First impacted: AI researchers, data scientists
Time to impact: Short

Alignment Lab AI (a global team dedicated to advancing AI through ethical alignment and open-source innovation), in collaboration with Hive Digital Technologies (a data center operator), has launched Buzz-8b-Large, a new language model designed to enhance the efficiency of reusing and optimizing pre-trained language models. They utilize a novel iterative fine-tuning methodology that incorporates "grounding" distributions from previous epochs, enhancing the model's learning efficiency and performance capabilities. The release on HuggingFace states that "Buzz hopes to be a proof of concept, and a toolkit to demonstrate and enable the community in the pursuit of efficient and effective locally run, personally owned, language models." [H-D-T/Buzz-8b-Large-v0.5 · Hugging Face] Share this story by email