Interconnects cover art

Interconnects

Interconnects

By: Nathan Lambert
Listen for free

This Season's Biggest Deal | $0.99/mo for 3 months + $20 Audible bonus

Auto-renews at $8.99/mo. after 3 months - terms apply.
Summary
 Audio essays about the latest developments in AI and interviews with leading scientists in the field. Breaking the hype, understanding what's under the hood, and telling stories.

www.interconnects.aiInterconnects AI, LLC Science
Episodes
  • Open-source AI without Jensen

    Open-source AI without Jensen
    Nov 14 2026
    The resignation of Nvidia CEO Jensen Huang is a major setback for the US development of an open model ecosystem. Nvidia still has the financial interests in supporting open models, which are to avoid concentration in power among a few customers who would develop their own chips and reduce Nvidia’s profits, but supporting the open ecosystem is a tricky puzzle more than an engineering problem to grind out a solution on.Nvidia’s business operations are famously extremely dependent on Jensen as an operator, so it is a consensus opinion that there are challenges ahead (or even more high-profile departures through chip-smuggling investigations or plain-old sharks circling more poachable talent). Nvidia’s number one goal is to stabilize the ship, deliver to customers in the next 24 months, and prevent implosion. Open models have been one of Jensen’s more ambitious, founder-style bets. He has been flooding the zone with funding in a way that will very likely dry up.Interconnects AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.In this period, the open model economy is in its nascency. These are crucial years for establishing use-cases, revenue flow, and motives to continue training the next generation models. I discussed the sensitivity of these early years in my recent talk at Fireworks Forge (slides, recording). The core of the argument is one upside for open models, a reason to be optimistic, and one downside.The positive case is that the open model economy has correctly latched onto “ownership of intelligence” and enterprise post-training as a crucial lane. What is to be determined is exactly how big the downstream market is.The downside is that the true long-term benefit of open models, as truly commoditized intelligence, is very far from realization with compute in such short supply. In the near term, where inference startups are also struggling to get enough chips (because Anthropic and OpenAI are buying so much capacity), there’s little margin competition on open-weight models. If open-source models are also served at giant inference margins, as we learned with DeepSeek’s DeepInfer release, they’re far from a commodity. In this paradigm, the closed labs with better economies of scale and integration from model to datacenter and chips will win. I expanded on this challenge in my recent piece on the open model economy.Together, a lot still needs to go right for open models to be a meaningful economic engine. Jensen’s support, and protection, gave the ecosystem time to figure this out.Jensen was more than a cheerleader. He would stop a lot of s**t from going wrong once early winds were stirring. He was willing to use his power on open models.There’s a major swell in Democratic government funding towards candidates that want to regulate the AI industry, fostering “safety”. Many of these people feel a need to do something and have general opinions that open models are unsafe, while not being educated on the complexities of their benefits. My biggest near-term worry is sloppy regulation at the state level, followed by national regulation. We’ve been one, strong Chinese open-weight model away from a crisis since this summer. It still holds – especially as we enter a new, turbulent equilibrium of cybersecurity. We have not seen a major hack driven by a popular Chinese open-weight model, but it is a matter of when, not if. We need someone to encourage caution and an ecosystem perspective when the “open-weight hawks” come hunting. Knocking over the open-source AI domino today is easy — if done, unwinding the downstream concentration of power, reduction in safety research, and stifling of long-term innovation is far harder. On public data, it looks like balancing long-term innovation versus short term risks, but this is due to how horribly hard it is to measure the open model economy (e.g. the most valuable aspects of it run on private infrastructure). It’s a recurring challenge facing the ecosystem and my work.As a brief aside, the extent to which the media ecosystem is revolving around timing today, rather than exact availability of information, is incredible. All of these Jensen affairs were known, but getting them out at the right time, right after the midterm political swing, took down the king.The same goes for the Jacob Coxon brouhaha this summer. All of that safety culture around top labs was well known, but the right timing around datacenters and the Overton window made it a national issue. This could very well happen at some point for open model safety, as it tends to be harm, risk, or safety-related narratives that drive the modern media algorithms. I cannot imagine my work on open-science driving such a media cycle, because it will never happen.The only lasting solution to this is to incubate a truly frontier open model player in the US. For example, I agree completely with Dylan Patel, CEO...
    Show More Show Less
    7 mins
  • Was I wrong about open models & cyber risks?

    Was I wrong about open models & cyber risks?
    Oct 30 2026
    Last week coined the term “Black October“ after an onslaught of critical vulnerability disclosures, including PHP2Shell and a long-standing GitLab bug, signified the moment when the cyber risks of AI models started to materialize. It seemed like the news cycle had stabilized a bit when I started writing this, but then there was news about another major crypto security incident at Binance. The crypto industry has a very long history of hacks, so it’s not the best industry to indicate vulnerabilities translating into new societal harm. Still, we are very much still living through the implications of this transition.The best guess for what is happening in the cybersecurity space is that the behind the scenes work has only accelerated, bringing more people and resources in to try and patch the internet — the Wall Street Journal reported that cyber first responders are strained, which seems like the system working. We’re living through this plot of an exponential rise in vulnerabilities, which will take a long time to level off at the top of the sigmoid.A few weeks ago, before the rapid acceleration in vulnerability disclosures and the uptick in reported hacks, I wrote the following take. It’s been getting a pretty consistent barrage of feedback, some hopeless dunk attempts and some reasonable discussion, so I wanted to unpack where I stand.Open-weight Mythos in April would’ve been fine?When Claude Mythos was announced, it was previewed as if it was a new class of cyber weapon, where if it ended up in the wrong hands it would’ve caused mass societal destabilization.It seems like this was wrong.By all measures, GLM-5.3 is the model that crosses that threshold of capability, and there’s little public evidence that much has changed, and we’re over a month out of the model weight release.In fact, I would go further. If Claude Mythos was accidentally released as open-weight, it seems like the world would have been more or less fine. Yes, there would be a clear increase in cybersecurity incidents and it would be bad for society if the model leaked, but it would be a scenario that looks more like an acceleration than a step change in risk.The leading proponents of this new wave of open-weight fear mongering have effectively been making a falsifiable prediction — that the current open-weight models are going to cause substantial, not seen before AI harms by crippling our cyber infrastructure.Based on all the evidence I have, which I’ve unpacked above, they’re set up to be wrong. At least now, after so many years of debating open weight risks, we will start to get some real answers.To start the reflection, I wanted to do more due diligence and track down some receipts on the claim that the AI safety community was reacting as if open-weight Mythos would be an actual disaster, and not just a painful headache. This was distinctly my impression having lived through the online discourse. I’ve gotten criticism that the frontier AI community never said it would be a “true disaster,” but regardless of exactly what they said, it is always about how people hear and internalize your information.Here are some examples of the sort of language that I felt was too exaggerated and from prominent people, which spilled into the public and more importantly federal government perspective that this was a cyber weapon include:* Rob Wiblin, host of the 80,000 hours podcast, on his own podcast (April 10, 2026): “If it fell into the hands of criminals or hostile state cyber actors, it would be an actual disaster.”* Anthropic, in the Project Glasswing announcement blog (April 7, 2026): “Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely. The fallout—for economies, public safety, and national security—could be severe.”For one that is actually more reasonable, to illustrate the fine lines between correct and exaggerated, from Zvi Mowshowitz (who I don’t agree with a ton of things on) on April 9, 2026: “Claude Mythos would give attackers a cornucopia of zero-day exploits for essentially all the software on Earth … It would be chaos.” To date, this chaos has been confined to those running large-scale digital products, which is good, but could change. I largely still stand by my above writing a few weeks ago, but I’m nervous about it. I’m not a cyber expert, I’m working hard to make the most educated predictions I can. It’s a fine line between “all of our tech companies need to work extra hard to keep us secure” and “public infrastructure is actually going down.” It is very clear as of writing this that there are way more critical vulnerabilities which are being found in popular tools and fixes are being built.For example, Epoch AI recently published a nice post on their internal work using GLM-5.3 to try and find vulnerabilities and chain them into what could be an attack. ...
    Show More Show Less
    10 mins
  • Artificial Analysis v5 and the benchmarker's dilemma

    Artificial Analysis v5 and the benchmarker's dilemma
    Oct 28 2026
    Hill-climbing is easy, knowing what to hill-climb is hard. Making scores go up is expensive, but we have a much clearer playbook for it than we do for creating meaningful benchmarks at the true frontier of AI today.Independent evaluation of language models is under-appreciated in how hard of a task it is. These groups are not just taking current models and certifying their scores, which is a technically involved task. This is their job if AI progress is slow. When AI progress is as fast as it is, the technical details of the job are the routine work part of trying to put a single number on what makes for a good AI model today. The goal is to have a single number that is representative of current models, matching vibe checks, community sentiment, and scientific analysis, while being forward looking enough so that it’s not outdated in 3 months.Everyone building a benchmark faces a sort of paradox. You need your benchmark to reduce to one single number, so that it is easy to discuss, use in reports, and compare models, but every method of aggregating scores comes with meaningful trade-offs. The best scientists need to do this, everyone does. It’s a matter of creating aggregation metrics that you think are most fair, knowing there will always be imperfections.With the amount of money in the AI ecosystem today, the weight of independent evaluation is even higher – you dictate the winners and losers of a trillion dollar market. You also influence some of the models themselves, steering the shape of progress. This is a lot of pressure!My prompt for this article is Artificial Analysis’s recent update to their Intelligence Index, v5. The AA index is the most prominent independent intelligence metric for the latest frontier models, followed in a distant second by Epoch AI’s Capabilities Index (ECI). To date, all of these have been incredibly useful for the community and massively flawed!Interconnects AI is a reader-supported publication. Consider becoming a subscriber.Language models are capable of so much today, and this scope of capabilities is growing quickly. Our era of singular intelligence indices should be phased out soon, but it will take a long time.A summary of AAv5 is below, from their blog post, where they describe it as “reflecting real-world usage today:”Intelligence Index v5 weights:➤ Knowledge Work (25%): AA-Briefcase (15%), AutomationBench-AA (5%), and GDP.pdf (5%)➤ Coding (25%): FrontierCode 2.0 (15%) and Terminal-Bench 4.0 (10%)➤ Science (25%): Terminal-Bench-Science (15%) and CritPt v2 hard subset (10%)➤ General (25%): AA-Omniscience v1.1 15% (10% accuracy, 5% non-hallucination) and HLE-Diamond (10%)Here’s the new ranking of models:The problem with trying to benchmark real-world usage today is that it is extremely jagged, just like the models. The most proven use-case is coding, where open models are far more competitive and likely to keep up. In knowledge work, I’m less confident in open models.At first glance, in aggregate, their model ranking definitely looks reasonable. Still, it shows signs of strain and confusion, as any benchmark will, with GPT 6.1 Terra beating Claude Fable 5.1 on AAv5. Today, some benchmarks measure the absolute frontier of capabilities, playing nicely in open-ended tasks through your favorite harness, which often does take a large model. Some other benchmarks measure certain skills that will be needed of a model in repetitive agentic tasks. I get the feeling that AA is tuning their benchmark more towards the former, a general agentic collaborator, but all of these behaviors overlap.This takes us to the plot going around that prompted me to write this, where they showed the balance of open vs. closed models on the new benchmark. This plot shows 8 closed models beating the best to-be open-weight model, Kimi K3.1 Pro (which isn’t colored blue). This benchmark is taking the opinion of trying to measure where frontier models are moving into, which grows the gap to open models. This is an okay decision to take, but over time as existing workflows of open models become key factors of the economy, it could become a benchmark that misses where the core economic engine of the industry is.Open models’ demise from this seems greatly exaggerated, but I do think the previous index was flattering the open weight models too strongly. Consider this plot from the Kimi K3.1 Pro launch — yes, the open models are strong and useful in some cases, but the best OpenAI and Anthropic models have been a major step ahead in their flexibility and reliability.Some of the evaluations in AAv5 that open models are most behind on are “science” evals like terminal-bench-science and CritPT v2. Open models are behind, but the value proposition was never to compete truly at the frontier. The open model economy will be about* Reducing domains with high market demand for inference to a commodity price of hardware.* Allowing ownership of intelligence, specializing it to ...
    Show More Show Less
    11 mins
No reviews yet
In the spirit of reconciliation, Audible acknowledges the Traditional Custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to their elders past and present and extend that respect to all Aboriginal and Torres Strait Islander peoples today.