Naval puts the two standard bullish arguments to Deutsch: enough compute produces emergent general algorithms, and multimodal embodiment gives the model context the way infants get it. Deutsch rejects both. The way the models do maths resembles students who never got it but have more compute, picking up the feel of what tests look like without learning any mathematics. Adding digits indefinitely is increasingly futile, and it never knows what multiplication is.

Test-feel through practice is not learning mathematics
Learning maths is not learning to execute an algorithm
It can recite an encyclopedia definition of multiplication and still not do it
Deutsch's falsifier: prove the Riemann conjecture and he is wrong

“I think it's precisely not how human babies learn”
11:30

“never does it know what multiplication is”
12:30

#agi#learning#mathematics#embodiment

Hot Take· 2

Hot Take05:30

It never was intelligent: why Deutsch sees no threshold being crossed

Asked whether the models are getting smarter, Deutsch says the question is malformed because they never were intelligent. He rates GPT-4 slightly better than 3.5 and calls it a phenomenal chatbot he expected to take decades. His explanation for why people react as if a line was crossed is that chatbots did not improve incrementally, so the sudden jump in quality bowls people over. There is no understanding underneath, of their own output or of the human's.

A large quality jump is not a threshold crossing
Compares the jump to an electric car's sudden acceleration
No understanding of its own output, and none of the prompt
Surprised chatbots got this good this fast, and still calls it a chatbot

“is it getting more intelligent it never was intelligent”
05:30

“they don't understand what they themselves have just said they certainly don't understand what the human says to them”
06:30

#agi#llms#understanding#ai-skepticism

Hot Take45:00

Dictators, AI and the weather are not the existential threat

Deutsch refuses to prophesy, then rules almost everything out. Threats from outside the civilisation, dictators, terrorists and any AGI or ASI that appears, he cannot take seriously, since the first AGIs would presumably be part of Enlightenment culture and enhance it. Climate scare stories are about expense on long timescales, not existence. The one candidate that could be existential is the civilisation of the Enlightenment making bad enough mistakes, through fads and ideologies that deny and hate it.

External threats do not scale to existential
The first AGIs would inherit our culture and extend it
Weather risks are cost problems on much longer timescales
The Enlightenment contained an anti-Enlightenment from day one, per Roy Porter

“I can't take seriously any threats to our civilization from the outside that is dictators terrorists”
45:00

“the only threat that could possibly be existential is if our civilization the civilization of the Enlightenment makes bad enough mistakes”
46:00

#existential-risk#enlightenment#civilisation#ideology

Explainer· 6

Explainer02:00

The formula that separates hard science fiction from lazy science fiction

Deutsch names Greg Egan as the best current science fiction author and states the formula for the genre: invent one fictional piece of science, then explore its ramifications in both science and society. The invented world has to hold together, with laws of physics, a coherent society and aliens that make sense, which is the hard-to-vary signature applied to fiction. He rates Neal Stephenson second, and separates Ted Chiang and Borges by how they play with time and infinity.

Invent one fictional science, then follow its consequences honestly
Egan gets the maths and physics right, including a universe with a different spacetime signature
Fiction becomes fantasy when the world stops making sense
Stephenson writes a different genre every book; Borges switches genre inside one story

“I think the best science fiction author currently is Greg Egan”
02:00

“you invent a fictional piece of Science and then you explore the ramifications of it both in science and in society”
02:30

#science-fiction#reading#worldbuilding#creativity

Explainer22:30

Why Deutsch says artificial superintelligence cannot exist

Silicon Valley researchers tell Naval that AGI is five to ten years out and ASI follows five to ten years after that, a self-improving system that recursively makes itself smarter. Deutsch denies the category. There is nothing beyond explanation, because explanatory universality rests on Turing universality, which rests on physics. Whatever an ASI was could be reverse-programmed down to the Turing level and back up to the explanatory level. An AGI could try to improve itself, but no more reliably than humans do.

Explanatory universality rests on Turing universality, which rests on physics
Any candidate ASI reduces to the same universal level
Self-improvement is available to humans too, and is not reliable for either
The ceiling is a physical fact, not an engineering limit

“I don't think there's such a thing as an ASI”
23:00

“there can't be anything beyond explanation because explanatory universality rests on touring universality and that rests on physics”
23:00

#asi#universality#agi#physics

Explainer26:30

Language came two million years before speech, and Deutsch buys it

Deutsch relays Daniel Everett's anti-Chomskian case that human ancestors had language two million years ago, with Homo erectus. The argument runs through evolutionary pressure: the fine motor adaptations for speech in the throat and mouth had to be selected for by something, and that something was language. Modern experiments teaching fire-making without words support the idea that symbolic communication predates speech. For Deutsch, speech is icing on the cake; the basic idea of language is symbols.

Speech adaptations need prior selection pressure, and language supplies it
Symbols need not be words or sentences
Graduate students can be taught fire-making with gesture and inarticulate sound
Deutsch is candid that he has watched one video and one failed criticism of Everett

“he reckons that humans had language or rather sorry that human ancestors had language two million years ago with Homo erectus”
27:00

“the basic idea of language is as ett says symbols and symbols need not be words or sentences”
28:30

#language#evolution#homo-erectus#linguistics

Explainer29:30

Universal explanation may have evolved exactly once, in Homo erectus

Naval asks whether the jump to explanatory universality happened once or repeatedly. Deutsch says it is an open question, then guesses once, possibly in Homo erectus, since every known instance is in apes and their descendants and the capacity must evolve in a meme-carrying animal. He rejects levels of personhood: earlier people may have had less memory or slower thought, but nothing essential was missing. The fluke could cut either way, and might bear on the Fermi paradox.

Meme-carrying animals are common; this capacity appeared in one lineage
No serious gradation of personhood, only hardware limits like memory and speed
Homo erectus lasted over a million years and could have founded a civilisation
They lacked not capacity but aspiration, and carried anti-rational memes

“my guess is it began once maybe in fact homoerectus is the place where it began”
30:30

“could be that hom rectus could have founded a civilization and that could be 2 million years old by now”
31:30

#human-evolution#universality#fermi-paradox#memes

Explainer37:00

Taking children seriously, and why we barely manage it with adults

Naval extends Deutsch's taking children seriously position and asks whether there is an equivalent for adults. Deutsch says at scale we do not yet know how. The institutions of the West, science, economics and politics, are the best that have ever existed at fostering creativity by letting people act voluntarily, and they still have gaping imperfections. Any coercion, including a state enforcing the rule of law, is a sign of something unfixed. Among friends, we already do it automatically.

No coercion, no testing, no pushing; follow the child's own curiosity
Western institutions are the best yet and still visibly imperfect
State coercion marks a problem nobody has solved, not a settled answer
Telling a friend what coat to wear reads as rude and irrational, which is the tell

“any coercion even as exerted by a state enforcing the rule of law is a sign of something imperfect”
38:00

“with one's friends let's say with with the people one knows one is automatically doing the taking them seriously thing”
38:30

#taking-children-seriously#coercion#institutions#freedom

Explainer52:30

How elite opinion flips overnight, and why nobody knows the trigger

Deutsch contrasts the German universities, where Nazism had its core, with Britain, where the anti-democratic tendency was on the left and the pacifism was on orders from Stalin after his deal with Hitler. Those people turned on a sixpence when Stalin told them to. The stranger case is the general flip: nobody objected to the Rhineland, Austria or Czechoslovakia, then Poland was suddenly unacceptable. His guess is a seeding process, thinkers changing their minds and the people relying on them following.

Student radicalism is not intrinsically bad; the direction is what varies by country
British communist pacifism tracked the Molotov-Ribbentrop deal exactly
No principled difference between the Rhineland and Poland, yet opinion phase-changed
Naval's frame: ideology is play until consequences arrive, then the top changes and the rest follow

“they were all pretending to be pacifists so they were against the war but that was because Stalin told them to be against the war…”
53:00

“suddenly he invades Poland and everybody's like this is unacceptable”
53:30

#ideology#information-cascade#universities#history

Story· 4

Story07:00

The Socrates and Plato picture that Deutsch could not correct

Deutsch tried to generate an illustration he had wanted for a book: Socrates with the young Plato and friends. The model produced a black-and-white image with Socrates enthroned above everyone. He asked for Socrates to be lowered to the same level and for Plato, a wrestler, to be built accordingly. It made Plato topless and ripped, kept Socrates taller, and after three or four further prompts he still could not get Plato clothed. Personalising the prompts made it worse.

Photorealistic was interpreted as black and white
Explicit corrections were not applied, or were applied once and then lost
Plato means broad, and Deutsch calls him a wrestler in The Beginning of Infinity
Once it went down the wrong track, he had no method to pull it back

“so I said put Socrates down the same level as everybody else and by the way make Plato a bit taller”
08:00

“I tried three or four more prompts I just couldn't get in to clothe Plato again after it had got that wrong the first time”
09:00

#image-generation#prompting#ai-limits#plato

Story35:30

Van Gogh and the mathematician who turned down a million dollars

Deutsch reaches for two cases that break the self-help frame. Van Gogh never sold a painting, refused the gallery job his brother offered and would have been good at, painted what and how he wanted, was hard to be around, and was recognised as a genius only after his death. The Russian mathematician who refused every award is the other. Deutsch's line: taking a million to work on the wrong thing is bad, being handed one for work you were doing anyway is not.

Van Gogh refused a job he would have been great at to paint his own way
Whether he helped himself is left as an open question
Accepting money for work you already chose is not a compromise
Refusing an unasked-for prize signals something unexplained is going on

“never sold a painting refused to take the job that his brother offered him in the art gallery”
36:00

“if he works on it for its own sake and then somebody offers him a million dollars why not take the million dollars”
37:00

#self-help#motivation#van-gogh#prizes

Story47:30

We are not in 1940, we are in the interwar period

Deutsch corrects the usual World War Two analogy. The match is the twenties and thirties: a massive loss of confidence in our own culture, and conventional wisdom drawing the wrong lesson from the Great Depression, that we needed less capitalism and stronger leaders. His favourite example is the Oxford Union carrying a motion that the house would not fight for king and country under any circumstances, allegedly giving Hitler ideas. A year later those same students were flying in the Battle of Britain.

The interwar loss of confidence, not the wartime resolve, is the analogy
Britain, France and America confirmed the fascist charge of decadence
The undergraduates who passed the motion were fighting within a year
Deutsch's phrase for the earlier posture: you piss on us we say it's raining

“the place where we're most analogous to is not the second world war it's the 30s 20s and 30s the interwar period”
47:30

“the motion was this house would not fight for king and country under any circumstances”
48:30

#history#interwar#oxford-union#appeasement

Story49:30

His mother knew the Allies would win on 3 September 1939

Deutsch asked his mother, a Holocaust survivor having a very bad time of it, when she became sure the Allies would win. He expected a late answer, since in September 1939 the world thought Britain was doomed and Joseph Kennedy cabled Washington that Britain was finished and to make an accommodation with the Nazis. She said 3 September 1939, the day Britain and France declared war, because that was the moment the policy of pretending it was raining ended.

The turning point was a declared policy change, not a battlefield result
Kennedy was withdrawn as ambassador after the British heard of his cable
Nobody could foresee the tactical route; some found the outcome obvious anyway
Britain flipped its whole set of beliefs on a dime

“she said September the 3rd 1939 the day that Britain and France declared war”
50:30

“that was the moment when they reversed their policy of saying it's raining and started the policy of actually standing up for civilization”
50:30

#history#world-war-two#conviction#britain

Takeaway· 2

Takeaway01:00

Deutsch uses ChatGPT many times a day and trusts it not even slightly

Deutsch's theoretical view of AI has not moved, but his behaviour has. He has used ChatGPT since March, uses it many times a day, and is still discovering new uses for it. Most of those uses are things Google could do but not fast enough to be worth it. At the same time it is often confidently wrong, so it cannot be relied on at all.

The utility is real and the theory of mind behind it is unchanged
Best use case: questions where a search would work but cost too much time
Hallucinates and is very sure while giving the wrong answer
Still in the discovery phase months after adopting it

“I use chat GPT all the time many times a day and it's incredibly useful”
01:00

“chat GPT is often very wrong it often hallucinates”
01:30

#chatgpt#ai#tools#productivity

Takeaway20:30

Good at extrapolation, bad at synthesis, and that is still worth a lot

Naval reports that in areas he knows well the model adds verbiage without information, and summarises badly because it keeps the wrong things. Deutsch's framing is that it is better at extrapolation than synthesis, and extrapolation is what a great deal of society already does: newspaper columns, term papers, padding to length. Compression requires understanding what is superfluous. The upside is that most of what humans do is not creative, so offloading it increases the creativity in the world.

Adding words is easy; reducing to the core requires understanding
It drops the important bits and keeps the wrong ones when summarising
A lot of human output is pragmatic filler, not creative work
Removing drudgery frees human creativity without the tool being creative

“I found it better at extrapolation than synthesis and extrapolation seems to be what a lot of society does”
21:00

“synthesizing reducing coming to the core of it I think is very difficult because it requires understanding”
21:30

#ai-tools#writing#synthesis#productivity

Frameworks in this episode

The Disobedience Test for Creativity

The Enhancement Factor Test for Relationships

The Fun Criterion for Choosing Problems

Insights & moments

⚡Myth Buster· 1

Emergent arithmetic and robot bodies are not how babies learn

◆Hot Take· 2