Episodes

Latest Episode
Name, image, and AI's likeness

Name, image, and AI's likeness

Episode 34 · · 09:03

Celebrity's power will only grow in the era of infinite content.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOrigi...

OpenAI chases Her

OpenAI chases Her

Episode 33 · · 12:28

ChatGPT leaves the textbox, and Google is building the same, and more, as practical tools.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolamber...

OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions

OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions

Episode 32 · · 14:05

Now we will have some grounding for when weird ChatGPT behaviors are intended or side-effects -- shrinking the Overton window of RLHF bugs.This is AI generated audio with Python and ...

RLHF: A thin line between useful and lobotomized

RLHF: A thin line between useful and lobotomized

Episode 31 · · 13:08

Many, many signs of life for preference fine-tuning beyond spoofing chat evaluation tools.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolamber...

Phi 3 and Arctic: Outlier LMs are hints

Phi 3 and Arctic: Outlier LMs are hints

Episode 30 · · 09:46

Models that seem totally out of scope from recent open LLMs give us a sneak peek of where the industry will be in 6 to 18 months.This is AI generated audio with Python and 11Labs.Sou...

AGI is what you want it to be

AGI is what you want it to be

Episode 29 · · 10:38

Certain definitions of AGI are backing people into a pseudo-religious corner.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnec...

Llama 3: Scaling open LLMs to AGI

Llama 3: Scaling open LLMs to AGI

Episode 28 · · 15:05

Meta shows that scaling won't be a limit for open LLM players in the near future.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interco...

Stop "reinventing" everything to "solve" alignment

Stop "reinventing" everything to "solve" alignment

Episode 27 · · 07:32

Integrating some non computing science into reinforcement learning from human feedback can give us the models we want.This is AI generated audio with Python and 11Labs.Source code: h...

The end of the "best open LLM"

The end of the "best open LLM"

Episode 26 · · 06:45

Modeling the compute versus performance tradeoff of many open LLMs.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOr...

Why we disagree on what open-source AI should be

Why we disagree on what open-source AI should be

Episode 25 · · 08:57

Last minute title change from: The tech industry can't agree on what open-source AI means. That's the process.How to read what multiple people mean by the word openness and see throu...

DBRX: The new best open LLM and Databricks' ML strategy

DBRX: The new best open LLM and Databricks' ML strategy

Episode 24 · · 16:33

Databricks' new model is surpassing the performance of Mixtral and Llama 2 while still being in a size category that's reasonably accessible.This is AI generated audio with Python an...

Evaluations: Trust, performance, and price (bonus, announcing RewardBench)

Evaluations: Trust, performance, and price (bonus, announcing RewardBench)

Episode 23 · · 12:40

Evaluation is not only getting harder with modern LLMs, it's getting harder because it means something different.This is AI generated audio with Python and 11Labs. Music generated by...

Model commoditization and product moats

Model commoditization and product moats

Episode 22 · · 10:56

Where moats are tested now that so many people have trained GPT4 class models. Claude 3, Gemini 1.5, Inflection 2.5, and Mistral Large are here to party.This is AI generated audio wi...

The koan of an open-source LLM

The koan of an open-source LLM

Episode 21 · · 23:06

A proposal for a new definition of an "open source" LLM and why no definition will ever just work.This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGe...

Interviewing Louis Castricato of Synth Labs and Eleuther AI on RLHF, Gemini Drama, DPO, founding Carper AI, preference data, reward models, and everything in between

Interviewing Louis Castricato of Synth Labs and Eleuther AI on RLHF, Gemini Drama, DPO, founding Carper AI, preference data, reward models, and everything in between

Episode 20 · · 01:26:28

Louis recently has been founding a new startup focused on synthetic data for alignment, Synth Labs, and is a researcher at Eleuether AI. This interview should speak for itself, and i...

How to cultivate a high-signal AI feed

How to cultivate a high-signal AI feed

Episode 19 · · 10:46

Basic tips on how to assess inbound ML content and cultivate your news feed.This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.Source code: https:/...

Google ships it: Gemma open LLMs and Gemini backlash

Google ships it: Gemma open LLMs and Gemini backlash

Episode 18 · · 17:17

Google rejoins the open model party and gets some backlash for a frequent problem for generative AI.This is AI generated audio with Python and 11Labs. Music generated by Meta's Music...

10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and more

10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and more

Episode 17 · · 14:58

10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and moreThis is AI generated audio with Python and 11Labs. Music generated by Meta...

Releases! OpenAI’s Sora for video, Gemini 1.5's infinite context, and a secret Mistral model

Releases! OpenAI’s Sora for video, Gemini 1.5's infinite context, and a secret Mistral model

Episode 16 · · 09:07

Emergency blog! Three things you need to know from the ML world that arrived yesterday.This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.Source co...

Why reward models are still key to understanding alignment

Why reward models are still key to understanding alignment

Episode 15 · · 07:44

In an era dominated by direct preference optimization and LLMasajudge, why do we still need a model to output only a scalar reward?This is AI generated audio with Python and 11Labs. ...

Alignment-as-a-Service: Scale AI vs. the new guys

Alignment-as-a-Service: Scale AI vs. the new guys

Episode 14 · · 10:19

Scale's making over $750 million per year selling data for RLHF, who's coming to take it?This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.Source ...

Open Language Models (OLMos) and the LLM landscape

Open Language Models (OLMos) and the LLM landscape

Episode 13 · · 09:28

A small model at the beginning of big changes.This is AI generated audio with Python and 11LabsSource code: https://github.com/natolambert/interconnects-toolsOriginal post: https://w...

Model merging lessons in The Waifu Research Department

Model merging lessons in The Waifu Research Department

Episode 12 · · 19:05

Note: some of the audio in the second half is a little wonky, but the general voice was upgraded so hopefully it's a little less "poppy" until then!I'm trying to fix little pronuncia...

Local LLMs, some facts some fiction

Local LLMs, some facts some fiction

Episode 11 · · 09:59

Local LLMs: the latency solution, Meta's open AGI, personalization myth, and moats X factorThe deployment path that'll break through in 2024. Plus, checking in on strategies across B...

Multimodal blogging: My AI tools to expand your audience

Multimodal blogging: My AI tools to expand your audience

Episode 10 · · 08:18

A fun demo on how generative AI can transform content creation, and tools for my fellow writers on Substack!This is AI generated audio with Python and 11LabsSource code: https://gith...

Multimodal LM roundup: Unified IO 2, inputs and outputs, Gemini, LLaVA-RLHF, and RLHF questions

Multimodal LM roundup: Unified IO 2, inputs and outputs, Gemini, LLaVA-RLHF, and RLHF questions

Episode 9 · · 15:59

A sampling of recent happenings in the multimodal space. Be sure to expect more this year.This is AI generated audio with Python and 11LabsSource code: https://github.com/natolambert...

Where 2024’s “open GPT4” can’t match OpenAI’s

Where 2024’s “open GPT4” can’t match OpenAI’s

Episode 8 · · 13:41

And why the comparisons don't really matter. Repeated patterns in the race for reproducing ChatGPT, another year of evaluation crises, and people who will take awesome news too far.T...

It's 2024 and they just want to learn

It's 2024 and they just want to learn

Episode 7 · · 09:57

The state of the ML communities big and small starting 2024. My general expectations for the year.This is AI generated audio with Python and 11LabsSource code: https://github.com/nat...

Interconnects year in review: 2023

Interconnects year in review: 2023

Episode 6 · · 14:45

The core themes of ML and the blog this year. What changes in 2024.This is AI generated audio with Python and 11Labs. Source code can be found here: https://github.com/natolambert/in...

Interviewing Tri Dao and Michael Poli of Together AI on the future of LLM architectures

Interviewing Tri Dao and Michael Poli of Together AI on the future of LLM architectures

Episode 5 · · 35:47

Michael Poli is a PhD student at Stanford and a researcher at Together AI. https://zymrael.github.io/Tri Dao is the Chief Scientist at Together AI and an incoming assistant professor...