. . AI: Bard Beats GPT-4 (But not Turbo!)

Influencers boosting academic citations, new RAG

Author

Marshall Kirkpatrick
January 26, 2024

Friends, 4 stories lead the AI conversation today. Google’s Bard beat GPT-4 in one contest (though not GPT-4-Turbo yet), there are enough places you can get Mixtral-powered inference as a service that a new leaderboard tracks their cost and performance, a startup has launched a new way to search the web and your own documents, and (perhaps unsurprisingly) influencers online are impacting academic citation counts for AI research.

As always, these are the top stories by weighted analysis of AI community engagement. Thank you for reading and sharing with others - every time someone shares this newsletter, it makes my day.

More AI news on Monday!

-Marshall Kirkpatrick, Editor

First impacted: AI developers, Chatbot Users
Time to impact: Short

Google's chatbot, Bard, powered by the Gemini Pro model, has climbed to the second position on the LMSYS leaderboard, outdoing GPT-4, trailing only GPT-4-Turbo. There has also been chatter about the possible launch of an improved model soon called 'Gemini Ultra' on various social networks with some speculating it could be as early as next week. [Chatbot Arena Leaderboard - a Hugging Face Space by lmsys] Explore more of our coverage of: AI Chatbots, Google's Bard, lmsys Leaderboard. Share this story by email

Influencers are Boosting AI and ML Paper Citations

First impacted: Academic researchers, Social media influencers
Time to impact: Short

A recent study from UC Santa Barbara researchers reviewed 8,000 AI and ML papers shared on Twitter suggests that papers boosted by social media influencers receive 2-3 times more citations than those in a similar control group. The study underscores the increasing influence of social media in academia, particularly in raising the profile of research papers and expanding the range of authors featured.

  • The study suggests the emergence of a 'TikTokification of academia', where social media influencers could impact how research shared, read and ultimately adopted.

First impacted: AI developers, API inference product users, Scaling Companies
Time to impact: Short

Martian has launched a leaderboard for LLM Inference Providers and is providing daily updates on metrics such as output token length, cost and service load. The leaderboard not only measures the performance of LLM inference providers, but tracks these metrics over time, providing a comprehensive overview of the providers' reliability and performance. So far the top 5 are all providers of Mixtral, from DEEPINFRA to ANYSCALE. [LLM Inference Provider Leaderboard] Explore more of our coverage of: LLM Inference Providers, API Performance Metrics, Martian Leaderboard. Share this story by email

First impacted: Data Scientists, Information Retrieval Professionals
Time to impact: Short

AI startup Metaphor has rebranded to 'Exa' and is introducing a suite of content retrieval features using new models trained for RAG. "We're building the future of search," the company's website says. "The Exa API retrieves the best content on the Internet using embeddings-based search." Exa says its access to A100 clusters enables them to create state-of-the-art self-supervised search models. [Exa API] Explore more of our coverage of: AI Startups, Content Retrieval, RAG-Trained Models. Share this story by email

Have a good weekend, more AI news coming on Monday!