Gutenberg Lab — Morning Paper

Three AI Researchers Debate Technical Barriers to Recursive Self-Improvement

Participants

Dwarkesh Patel (host) × Baron Millich (CTO, Zyphra), John Schulman (Chief Scientist, Thinking Machines; former OpenAI co-founder), Charlie O'Neil (Head of Model Training, Base Ten)

Bottom line

AI reaching self-improvement loops is estimated at 2-10 years, but whether current methods (transformers + RL) can reach the "global optimum" of intelligence remains unresolved.

Three-Line Summary

• The CTO of Zyphra, a former OpenAI researcher, and Base Ten's training lead debated whether AI recursive self-improvement (RSI) will arrive by 2036
• All predict major AI progress, but disagree on timelines for automating AI research itself (2-10 years) and achieving superintelligence across all cognitive domains (3-10 years)
• Technical cruxes center on three questions: whether the current paradigm requires fundamental discontinuities, whether objective-setting can be automated, and whether continual learning can be solved

Three Key Points

1. Distillation as anti-centralizing force: John Schulman noted that "anything learned through RL can be distilled very easily" from small amounts of data, which fights consolidation among model providers. However, this requires the right prompt distribution—Chinese companies are reportedly using router services (proxies allowing China to access US models) to collect and sell usage data, providing "perfect prompt distribution" for distillation

2. Mid-training does 80% of the work: Baron attributed RL's success to "very, very good mid-training data," explaining that "this actually takes the model almost 80% of the way to the final RL checkpoint often." RL's bits carry "extremely high signal" because the objective ignores all reasoning tokens and updates only on answers—versus supervised learning where signal was "drowned out in noise"

3. Timeline prediction variance: Hiring AI as drop-in remote workers for white-collar work is predicted at 1-3 years, 10x productivity uplift for AI researchers at 2-10 years, and superintelligence across all cognitive work at 3-10 years. The most optimistic, John, sees "two years" for automating AI research, while Charlie predicts 5-10 years, citing his "capacity to absorb information and make Bayesian optimal decisions on next experiments" as the bottleneck

Editorial Perspective

The question of how close current deep learning paradigms are to the "global optimum" of intelligence mirrors the structure of Moore's Law—which required many discrete innovations to sustain. The core insight is the separation between technical cumulative improvements and the philosophical/normative problem of objective specification.

Source

Dwarkesh Podcast, "AI researchers debate how close we are to recursive self-improvement" (September 11, 2026)

https://www.dwarkesh.com/p/john-beren-charlie

AI Disclosure: Produced with AI assistance; facts and analysis reviewed by our editorial team.

• --

Crypto's Regulatory Failure and AI's Slowdown Debate Reveal the Fragility of Tying Industries to Political Regimes

Featuring

Kade Mark (Host of Uneasy Money, Synthetix/Infinex founder) × Tommy (Founding Partner, Delphi Ventures) × Taylor Monahan (Security expert, formerly MetaMask)

Bottom line

Relying on administrative discretion instead of laws for regulatory clarity creates a house of cards that collapses overnight when regimes change.

3-Line Summary

• Despite $100M+ lobbying and a pro-crypto administration, the Clarity Act failed to reach 50 Senate votes. Trump's memecoin launch created ethics concerns that killed the bill
• OpenAI/Anthropic's "AI slowdown" proposal is about profitability, not safety—both are "horrifically unprofitable," subsidizing compute, and trying to build a regulatory moat before Chinese open-source models (90% intelligence at 1/10th cost) take their market
• Both industries share a structure where founders trade with regulators for survival, creating a new vulnerability: dependence on political regimes. The "safety" camp is running a marketing operation, not solving the actual problem

3 Key Points

1. Gensler era was existential, current era is merely annoying — Kade: "I'd rather do it now—100% less likelihood a crackhead lunatic kicks my door down and shoots me in the head. I'm talking about Gensler." Tommy: "We'd talk to brilliant founders… and it's like, but they might go to prison. So weird under Gensler." The trade-off: less existential risk, but more competition (Nasdaq-listed entities now in crypto) and no legislative clarity

2. Frontier labs have no idea what they're doing — DeepSeek switched everyone to 4.1 Flash, claiming "we've discovered AGI." Two days later: "Sorry, it's really dumb." Kade: "These labs have no idea what they're doing." Taylor: "They're way more innovative but also have no idea. That scares me more than the safety thing—you guys truly don't." Implication: if labs can't control their own models, they can't control AGI

3. Anthropic/OpenAI can read every prompt you send — Taylor on Anthropic's threat actor report: "They can literally read every single job you have in full." Example: CCP officials using a Chinese model (Anthropic backend) to surveil citizens—Anthropic saw it, published it. "This is also happening for US companies/enterprises. At some point that's a problem." Privacy risk is existential for enterprise adoption

Editorial Perspective

Crypto escaped Gensler—the "crackhead lunatic"—by trading with the regime, but got a fragile victory: dependence on administrative discretion, not laws. AI is walking the same path. OpenAI/Anthropic call for "slowdown" while actually trying to build a regulatory moat before Chinese open-source models take their market. Both industries share a structure: founders trade with regulators for survival, creating a new vulnerability—dependence on political regimes. History shows that when innovation moves faster than governance, founders choose between compliance risk and competitive extinction. The "safety" camp in both domains is likely running a marketing operation, not solving the actual problem.

Source: Uneasy Money "Uneasy Money: What Crypto's Gensler Years Say About the AI Slowdown Debate" (September 17, 2026)

https://unchainedcrypto.com

Spoken Source: Unchained Crypto "Uneasy Money: What Crypto's Gensler Years Say About the AI Slowdown Debate" (September 17, 2026). Guests are Tommy (Founding Partner, Delphi Ventures) and Taylor Monahan (Security expert, formerly MetaMask)

AI Disclosure: Produced with AI assistance; facts and analysis reviewed by our editorial team.

• --

24-Year-Old Independent Journalist Exposes Government Fraud; California Passes Law to Stop Him

Featuring

Jason Calacanis × Nick Shirley (Independent Investigative Journalist/YouTuber)

Bottom line

Citizen journalism can now deliver the government accountability traditional media failed to provide—but power structures respond by legislating it out of existence.

3-Line Summary

Nick Shirley, a 24-year-old self-taught journalist, exposed Minnesota daycare fraud ($4 million) and California's catastrophic high-speed rail failure (18 years, $15 billion spent, zero operational track) through YouTube videos that led to arrests and political consequences. California responded by passing AB 2624, legislation specifically designed to prevent Shirley from filming government-funded facilities, threatening $4,000+ fines for violations. Shirley is now suing the state for First Amendment violations, while his funding model—audience subscriptions on X—proves independent investigative journalism can be economically viable without institutional backing.

3 Key Points

1. California's high-speed rail represents government failure at historic scale: After 18 years and $15 billion, California has built zero operational track. The original $33 billion estimate has ballooned to $236 billion—on track to become the third most expensive infrastructure project in world history after the Interstate Highway System and International Space Station, except those projects actually work. An expert calculates California could subsidize free one-way flights between San Francisco and Los Angeles for 150 years with the money already spent, or 300 years with the full project cost. One-third of spending ($5+ billion) went to "change order delays"—paying contractors $170,000/day ($5 million/month) when they can't work due to government planning failures.

2. Government creates legislation to suppress journalism exposing its own fraud: California passed AB 2624 (the "Stop Nick Shirley Act") allowing immigration support service providers to block Shirley's videos with a demand letter, threatening $4,000+ fines and legal fees for violations. The bill's co-sponsor CHIRLA received $80 million in government funding, and the bill's author Mia Bonta is married to California's Attorney General Rob Bonta—the official responsible for prosecuting fraud. This creates a structural conflict where the prosecutor's spouse legislates to make fraud harder to expose.

3. X amplifies accountability journalism 47x more than YouTube: Shirley's Minnesota fraud video got "a few million views" on YouTube but 140 million views on X, reaching decision-makers who then had to act. The video generated 4 billion views in 7 days, leading to the fraud ringleader's arrest (caught fleeing to London with a one-way ticket), guilty plea, Governor Tim Walz dropping out of reelection, and 12 individuals charged with $10 million in fraud on the day of this podcast recording. Shirley credits Elon Musk's platform changes for making this reach possible.

Editorial Perspective

Government accountability has shifted from institutional journalism to direct citizen investigation. Shirley's success provokes hostility from traditional journalists not because of his methods, but because he embodies what they wanted to be—idealistic, brave, uncaptured. They became politicized and picked sides while Shirley didn't. They criticize minor fact-checking issues while missing that he finds real fraud leading to arrests. Platform democratization (X, YouTube) and audience funding (X subscriptions) have made institution-independent investigative journalism economically viable—but power structures respond by legislating it out of existence. This tension will define press freedom for the next decade. The distinction between outright fraud (daycare schemes) and legal but wasteful corruption (high-speed rail settlements to billionaires) matters less than the systemic unaccountability both represent.

Source: All-In "Nick Shirley: Exposing Government Fraud, Suing California & Taking on the Media" (September 18, 2026)

https://allinchamathjason.libsyn.com/nick-shirley-exposing-government-fraud-suing-california-taking-on-the-media

AI Disclosure: Produced with AI assistance; facts and analysis reviewed by our editorial team.

• --