The Hard-to-Vary Test for Good Explanations
Judge an explanation by how little you can change it without wrecking it.
- Difficulty
- Advanced
- Time to result
- ~ongoing to results
- Steps
- 6
- Confidence
- 92%
Deutsch refuses to define explanation, but he will define a good one: it meets every criticism currently available, and it has no surviving rivals because you have found reasons to reject them. The operative property is hard-to-vary. A good explanation's parts are load-bearing — you cannot swap a component, or stretch its range to a neighbouring problem, and still have it work. Bad explanations (angels hold up the stars) are easy to come by and therefore easy to vary, which is why you can generate sixty an hour. The general test is criticizability, not falsifiability: mathematics and philosophy admit criticism without experiment, and any test in science must arise from within the explanation rather than being bolted on. Reach — explaining more than it was built for — makes an explanation better still, but most good explanations have none.
Origin
Deutsch's development of Popper's epistemology, set out in The Fabric of Reality and The Beginning of Infinity and restated here for a lay audience.
Core principles
- 01An explanation is good when it survives every criticism you currently have of it.
- 02If two live rivals both have something going for them, neither is yet the best explanation.
- 03Good explanations are hard to vary because they were hard to come by.
- 04Criticizability is the general test; experimental testability is its scientific special case.
- 05Reach is a bonus, not a requirement — most good explanations explain only their own narrow problem.
- 06A theory that immunizes itself against criticism has disqualified itself.
How to run it
- 1
Fix the problem before the answer
Write down precisely what the explanation is supposed to account for. Deutsch's own example is mundane: getting the delivery person to the right door. The problem sets the standard the explanation must meet.
Pro tip A vague problem statement lets any explanation look adequate.
- 2
Run it against every criticism you already hold
An explanation is good when it meets all the criticisms currently available. Enumerate the objections you can actually state and check each is answered rather than deferred.
Watch out Meeting today's criticisms is a moving bar — a better rival can appear tomorrow.
- 3
Kill the rivals explicitly
You only hold the best explanation once you have reasons to reject its rivals. If a competing account still has something going for it, you are holding two explanations, which means you hold none.
Pro tip List the rivals by name; unnamed rivals never get rejected.
- 4
Try to vary it
Attempt to swap a component for another, or extend its range to a neighbouring problem. If the explanation survives arbitrary substitution, its parts were doing no work and it explains nothing.
Pro tip Easy-to-come-by explanations are always easy to vary — that is the tell.
- 5
Test criticizability, not just falsifiability
Ask what forms of criticism the explanation exposes itself to. In science that includes experiment; outside science, criticism is still available. An account built so that no criticism can touch it is disqualified.
Pro tip Watch for immunising moves: mysterious ways, an incomprehensible simulator, critics not worth listening to.
Watch out Bolting an arbitrary testable prediction onto a theory does not make it testable in any useful sense.
- 6
Score reach as upside, not entry
If the explanation also accounts for things it was not built for, it is of a different order of goodness. But most good explanations have little or no reach, so absence of reach is not a defect.
Watch out Do not reject a narrow, hard-to-vary explanation for lacking grand implications.
In the wild
Asked why the stars do not fall down, Deutsch says he can generate sixty explanations an hour: angels hold them up, they are holes in the firmament, they are falling and we should take cover. Each is cheap and each is trivially variable. Then the planets turn up — points of light that move differently from night to night and year to year. An explanation that accounts for the planets as well was hard to come by, and its parts cannot be swapped. Once you hold it, the angel story is no longer available, because it explains nothing the new account explains.
→ The cheap rivals are eliminated not by taste but because they cannot be made to cover what the hard-won explanation covers.
Deutsch describes working with collaborators on constructor theory. The parts they have are very hard to change, which he treats as a good sign, but there is still no experimental test. He refuses to fix that by adding an arbitrary testable claim — attaching a prediction that the stock market will rise next year would be testable and would leave the theory no better as an explanation. The test they want has to arise from within the explanation, from what they have not yet discovered about it.
→ A theory can be provisionally good on hard-to-vary grounds while its genuine tests are still being worked out.
Common mistakes
Treating falsifiability as the whole test
Falsifiability is real but is the scientific special case of criticizability. Applying it outside science leads people to dismiss mathematics and philosophy as meaningless rather than criticising them.
Bolting on an arbitrary prediction
Adding a testable claim that does not arise from the explanation buys you a test and no explanatory power. The test has to make sense within the explanation.
Demanding reach from every explanation
Most good explanations solve only their own narrow problem and never generalise. Judging them by how much they explain elsewhere discards sound work.
Is it for you?
Best for
Anyone who has to choose between competing theories, strategies, or diagnoses and wants a criterion beyond consensus or gut feel.
Not ideal for
Situations demanding an immediate decision under time pressure, where no rival explanations can be surfaced or criticised.
From the transcript
“what makes an explanation good is that it meets all the criticisms that we have at the moment”
“therefore the good explanation is hard to vary it's hard to vary because it was hard to come by”
“criticizability you could say is the more General thing”
From the episode
David Deutsch: Knowledge Creation and The Human Race, Part 1