A new and mysterious Gemini model appears at the top of the leaderboard, but is that the full story? I dig behind the headline to show you some anti-climactic results, give some context with leaks in the last 48 hours of diminishing returns to scaling, and add the response of Altman, OpenAI and co. The future is about to look a lot stranger...


80,000 hours Podcast and Channel: https://open.spotify.com/show/2WzJwXWBDnn4iZ7odKwDib
https://www.youtube.com/@eightythousandhours/videos

You can now gift memberships to AI Insiders (my Patreon w/ exclusive vids, network): https://www.patreon.com/AIExplained/gift


‘There is no wall’: https://x.com/sama/status/1856941766915641580

https://x.com/vedantmisra/status/1857148554105544708

Gemini Ranking: https://lmarena.ai/?leaderboard

API not yet up: https://x.com/OfficialLoganK/status/1857106844805681153

‘Just Die Chat’: https://x.com/koltregaskes/status/1856754648146653428

Google CEO tweet: https://x.com/sundarpichai/status/1857114106928718329

Sutskever Quote: https://www.reuters.com/technology/artificial-intelligence/openai-rivals-seek-new-path-smarter-ai-current-methods-hit-limitations-2024-11-11/

Another OpenAI Staffer Leaves: https://x.com/RichardMCNgo/status/1856843040427839804

Bloomberg Report: https://www.bloomberg.com/news/articles/2024-11-13/openai-google-and-anthropic-are-struggling-to-build-more-advanced-ai?s=09

Noam Brown on what OpenAI Researchers Believe: https://x.com/polynoamial/status/1855037689533178289

Clive Chan: https://x.com/itsclivetime/status/1855704120495329667

Chollet Responds to Altman: https://x.com/fchollet/status/1857060079586975852

https://x.com/sama/status/1856940152460869718

Altman Emails: https://x.com/TechEmails/status/1857285960997712356

Change of Heart: https://sd11.senate.ca.gov/news/senator-wiener-responds-openai-opposition-sb-1047

Amodei on ‘Empirical Regularities’: https://lexfridman.com/dario-amodei-transcript/

Verge Report: https://www.theverge.com/2024/10/25/24279600/google-next-gemini-ai-model-openai-december

OpenAI Agents in January: https://www.bloomberg.com/news/articles/2024-11-13/openai-nears-launch-of-ai-agents-to-automate-tasks-for-users?srnd=phx-ai

The last few days have seen two narratives emerge. One, derived from yesterday’s OpenAI leak in TheInformation, that GPT-5/Orion is a disappointment, and less of a leap than GPT-3 to GPT-4. The second comes from a series of 4 clips (shown in this video) from Sam Altman, regarding the ‘clear path’ to AGI. Let’s go beyond the headlines (and through papers like Frontier Math) to get closer to the ground truth…

Plus Universal-2, Sora comments, Claude 3.5 Haiku SimpleBench update, and a great new AI video.


Assembly AI Speech to Text: https://www.assemblyai.com/?utm_source=youtube&utm_medium=influencer&utm_campaign=ai_explained

00:39 – Bear Case, TheInformation Leak

04:01 – Bull Case, Sam Altman

06:20 – FrontierMath

11:29 – o1 Paradigm

13:11 – Text to Video Greatness and Universal-2

TheInformation Leak: https://www.theinformation.com/articles/openai-shifts-strategy-as-rate-of-gpt-ai-improvements-slows?rc=sy0ihq

Noam Brown Replies: https://x.com/polynoamial/status/1855453104394637444

Sam Altman Y-Combinator Interview: https://www.youtube.com/watch?v=xXCBz_8hM9w&t=1556s

Altman Reply: https://x.com/sama/status/1855100359511097828

https://simple-bench.com/

FrontierMath Paper: https://arxiv.org/pdf/2411.04872

Frontier Math Blog Post: https://epochai.org/frontiermath

Tao: https://x.com/EpochAIResearch/status/1854996368814936250

MMLU Are We Done (cites me!): https://arxiv.org/pdf/2406.04127

Universal-2 https://www.assemblyai.com/research/universal-2

Noam Brown ‘We don’t know’: https://www.youtube.com/watch?v=Gr_eYXdHFis

Anthropic Founder Response: https://x.com/jackclarkSF/status/1855485569998217231

Sora (Runway Comment): https://x.com/c_valenzuelab/status/1855026417354129455

Sora New Vid: https://www.youtube.com/watch?v=_iETa2KDRuw

Darri3D Video: https://www.reddit.com/r/ChatGPT/comments/1gn0n3z/can_you/

The Google destroyer, the Perplexity crusher? Or just hype? ChatGPT with Search is here, and simultaneously Altman and co did an AMA on Reddit, covering GPT-5, Sora, SearchGPT and a lot more. Plus, the biggest news of them all: Simple Bench is out.

ChatGPT with Search: https://openai.com/index/introducing-chatgpt-search/

Altman AMA (ask me anything): https://www.reddit.com/r/ChatGPT/comments/1ggixzy/ama_with_openais_sam_altman_kevin_weil_srinivas/

https://x.com/sama/status/1852041075793522911

Perplexity Ads: https://www.cnbc.com/2024/08/22/perplexity-ai-plans-to-start-running-search-ads-in-fourth-quarter.html

Perplexity: https://www.perplexity.ai/

https://simple-bench.com/

A new state of the art LLM (at least for creative writing and basic reasoning) but what lies behind the numbers that were put out? Is it for real, and are AI agents about to grab your mouse and shake your cursor?

Plus, results on my own Simple Bench, and new tools from Runway (Act-One), HeyGen (Zoom Calls) and an updated NotebookLM. AI, without the hype.

Weights and Biases' Weave: https://wandb.me/ai_explained