6 remarkable things in AI today

Think of emergence, as a phenomenon. It’s hard to picture anything else when you see today’s stories: Like AI models being fed examples and training data until they hit a tipping point and start “grokking” general principles. Or when you think about millions (?) of developers getting one-click AI multi-node supercomputing powers, browser-based AI-assisted coding, and expert notebooks for deploying LLMs in solving complex science challenges.

These stories are awe-inspiring. And the first one is one you’ll probably end up citing in conversations about ChatGPT for a while.

Thanks for being here, feel free to share these stories with a friend,

-Marshall

(P.S. if you want to get these stories on LinkedIn or Instagram and support this project over there, I just created those accounts yesterday! Brand new :)

ChatGPT Crawlers Can Now be Blocked From Websites
OpenAI announced a new protocol for blocking its crawler with a robots.txt file it promised to respect going forward. Blogging developer Gergely Orosz prominently said he'll block the crawler because of ChatGPT's failure to cite its sources: "it's a one-way relationship." [GPTBot]

NVIDIA, Hugging Face Partner for AI Supercomputing
NVIDIA and Hugging Face are partnering to integrate NVIDIA DGX Cloud into Hugging Face's platform. This means Hugging Face "will be integrated into DGX Cloud for one-click multi-node AI supercomputing platforms," tech lead Philipp Schmid explained. Hugging Face will launch a new service, Training Cluster as a Service, powered by NVIDIA DGX Cloud to simplify generative AI model development. The integration is expected to be available in the coming months. [NVIDIA and Hugging Face to Connect Millions of Developers to Generative AI Supercomputing]

Google AI Explores 'Grokking' Phenomenon
"In 2021, researchers made a striking discovery while training a series of tiny models on toy tasks. They found a set of models that suddenly flipped from memorizing their training data to correctly generalizing on unseen inputs after training for much longer. This phenomenon – where generalization seems to happen abruptly and long after fitting the training data – is called grokking and has sparked a flurry of interest." [Do Machine Learning Models Memorize or Generalize?]

Anthropic Explores Influence of Examples on LLMs
Anthropic's newest research paper reveals the use of influence functions to pinpoint training examples impacting a model's output. The study found that larger models show more abstract generalization patterns as model scale grows. The research also indicates that influence is spread across numerous training examples, suggesting models don't just rely on memorization. My observation: this may make source citation all the harder, or it could be a step in that direction. [Studying Large Language Model Generalization with Influence Functions]

Google Announces New AI-Assisted Coding Platform idx.dev
Google's new browser-based code environment, idx.dev, uses AI for code-generation and explanation, and supports modern JavaScript frameworks. It simplifies full-stack, multiplatform app development. Users can join a waitlist to try it and give feedback. [idx.dev]

Getting Started With LLMs
Google's data science contest platform Kaggle is hosting a new competition called the LLM Science Exam and AI community leader Jeremy Howard has published a Kaggle notebook demonstrating the use of LLMs for solving challenging multiple choice science questions. [Getting started with LLMs]