. . AI: People Driving a Collaborative AI Ecosystem (2.12.24)

TheProfessor will help you brainstorm

Author

Marshall Kirkpatrick
February 12, 2024

Friends, the stories highlighted in today's newsletter showcase how the efforts of scientists, engineers, and researchers are collectively enriching the AI ecosystem in several key ways. It includes tools, resources and insights from the community, including the creation of a multilingual dataset aimed at overcoming language barriers in AI to cutting-edge tools that are reshaping research, engineering practices, and theoretical understanding.

Welcome to our new subscribers over the weekend; many of you came in from Geoff Livingston’s community. We’re happy to have you with us.

Now here’s today’s news,

Marshall Kirkpatrick, Editor

First impacted: Researchers, Brainstorming Teams, Academics
Time to impact: Short

AbacusAI has launched TheProfessor-155b, a big AI model engineered to boost brainstorming and research. Combining technologies like MergeKit with inputs from models such as Dolphin and SynthIA, TheProfessor-155b is tailored for a wide array of tasks—from aiding in dissertation writing to generating innovative ideas in science and math. A cool feature of TheProfessor is its ability to interpret and explain complex physics equations from numerous theoretical domains, such as gauge theories, emergent gravity, and cosmic inflation, showcasing its broad scientific capabilities. It's not available on Poe or Perplexity Labs yet but we're excited to try this one! [abacusai/TheProfessor-155b · Hugging Face] Explore more of our coverage of: AI Brainstorming Tools, Advanced AI Models, AbacusAI Innovations. Share this story by email

First impacted: AI researchers, Neural network engineers
Time to impact: Medium

Jascha Sohl-Dickstein, a research scientist at Google, revealed his discovery of a fractal pattern in the hyperparameter landscapes of neural network training. (Parameters are derived from training data; the hyperparameters are externally imposed during configuration and optimization.) Sohl-Dickstein suggests that this fractal behavior can demonstrate the challenges of meta-learning, as minor adjustments in hyperparameters could lead to considerable changes in training dynamics. [Neural network training makes beautiful fractals] Explore more of our coverage of: Neural Network Training, Hyperparameter Optimization, Meta-Learning Challenges. Share this story by email

First impacted: AI researchers, Machine Learning Engineers
Time to impact: Medium

Scientists worldwide have developed the Aya Dataset, which they say will help tackle language differences in AI using instruction fine-tuning (IFT). This dataset, which features 513 million instances spanning 114 languages, was gathered from instructions in different languages, templated and translated datasets. The initiative involved almost 3000 participants from 119 countries. [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning] Explore more of our coverage of: Aya Dataset, Instruction Fine-Tuning, Multilingual AI. Share this story by email

First impacted: AI Engineers, Open-source Developers
Time to impact: Short

Stas Bekman has revamped his Machine Learning Engineering guide, offering an organized collection of methodologies and tools for training large language models (LLMs) and multi-modal models. The book is aimed at engineers and operators and is filled with practical scripts and commands, drawing on Bekman's experience with projects like the BLOOM-176B and IDEFICS-80B models. [GitHub - stas00/ml-engineering: Machine Learning Engineering Guides and Tools] Explore more of our coverage of: Machine Learning, Language Models, AI Engineering. Share this story by email

First impacted: AI researchers, AI developers, Job Seekers
Time to impact: Short

Sebastian Ruder's latest post examines the AI and NLP fields, highlighting the blend of fundamental and applied research due to recent innovations. It discusses the evolving AI job market, promoting startups as practical alternatives to PhDs for cutting-edge work. Additionally, the shift from open-source to closed-source in ML development is critiqued, suggesting it could restrict innovation and knowledge sharing. This is a great read if you're evaluating the landscape for a career change. [Thoughts on the 2024 AI Job Market] Explore more of our coverage of: AI Job Market, NLP Research, Closed-Source Critique. Share this story by email

That’s it! More AI news tomorrow!