Experiment, Demonstration, Measurement
Only call it an experiment when two rival explanations are actually at stake.
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 5
- Confidence
- 82%
Deutsch splits the loose word experiment into three distinct activities. A demonstration shows that if you do this, you get that, with no rival theory in contention; pouring acid into base for a class of schoolchildren is a demonstration. A measurement refines a quantity inside a theory nobody is disputing; Cavendish weighing masses to get Newton's constant is a measurement, because Newton's theory was not in doubt before, during or after. A genuine experiment tests two rival good explanations against each other in a situation where you could not tell which is right without it. Which category a procedure falls into is set by the problem situation, not the equipment. Had someone proposed that the gravitational constant varies across the Earth, Cavendish's measurement would have become an experiment.
Origin
Deutsch sets up this three-way terminology on the Naval podcast while explaining why testability matters only within fields that can test explanations against each other, using acid and base for demonstration and the Cavendish experiment for measurement.
Core principles
- 01A test is only a test if it can change which explanation you hold.
- 02Showing that something happens is not the same as finding out why.
- 03Refining a constant inside an accepted theory settles nothing about that theory.
- 04The value of a procedure depends on the problem situation around it, not the apparatus.
- 05Most things called experiments are demonstrations or measurements in disguise.
How to run it
- 1
Write down the problem situation
Before classifying anything, state which explanations are currently live in the field and which of them are actually contested. Everything downstream depends on this list.
Pro tip Include explanations you think are wrong but that competent people still hold.
- 2
Count the rival good explanations
If the number is one, no test can choose between theories, whatever the apparatus looks like. If it is two or more, you may have a genuine experiment available.
Watch out Two rival explanations only count if both are good explanations, not one real theory plus a strawman.
- 3
Classify the procedure
Label it a demonstration if it exhibits a known effect, a measurement if it fixes a value inside an undisputed theory, or an experiment if the outcome selects between rivals.
Pro tip Say out loud what result would count as a defeat for your preferred explanation; if none exists, it is not an experiment.
Watch out Historical names mislead: the Cavendish experiment is a measurement by this terminology.
- 4
Match the claim to the category
Report a demonstration as an illustration, a measurement as a refinement of precision, and reserve confirmation or refutation language for genuine experiments.
Watch out Presenting a demonstration as a decisive test is how weak evidence acquires unearned authority.
- 5
Recheck when a rival appears
Because the classification depends on the problem situation, a new competing explanation can convert an old measurement into a decisive experiment. Revisit archived results when the theoretical landscape changes.
Pro tip Keep raw measurement data, since it may become experimental evidence later.
In the wild
Newton produced the theory of gravitation but could not separate the constant from the mass of the Earth, so masses of other bodies were expressed as multiples of the Earth's. Cavendish measured the gravitational force between two objects whose masses could be weighed directly against a standard, which fixed the constant. Deutsch points out that nobody doubted Newton's theory before, during or after, so despite its name this was a measurement rather than an experiment. Had anyone proposed that the constant varies across the Earth, the same procedure would have become an experiment.
→ A famous named experiment is reclassified as a measurement, showing that the category depends on which explanations were contested at the time.
Pouring acid into base in front of a class produces a colour change that reverses back and forth. Deutsch treats this as a demonstration, since the point is to show pupils an effect they would not have guessed. It only becomes an experiment when two accounts of the mechanism are on the table and the class asks how to distinguish between them, at which point the same apparatus is doing different epistemic work.
→ The identical bench procedure is a demonstration for one audience and an experiment for another, depending entirely on whether rival explanations are in play.
Common mistakes
Calling every procedure an experiment
Deutsch observes that many things now called experiments are really measurements, and many are really demonstrations. The loose label hides the fact that no explanation was ever at risk.
Testing against a strawman rival
An experiment needs two rival good explanations. Setting a real theory against a bad one produces a foregone conclusion dressed up as a decisive test.
Assuming precision equals confirmation
Tightening a constant inside an accepted theory improves precision but says nothing about whether the theory is right, because the theory was never in contention.
Is it for you?
Best for
Scientists, analysts and product teams who want to know whether a given test can actually settle the question they care about.
Not ideal for
Purely exploratory or teaching contexts where the aim is to show a phenomenon rather than adjudicate between explanations.
From the transcript
“I'd like to draw a distinction between experiments demonstrations and measurements”
“that is an experiment it's testing two different explanations against each other where you can't tell without the experiment which is the good explanation”
“that's not an experiment because there's only one explanation involved before during and after”
From the episode
David Deutsch: Knowledge Creation and The Human Race, Part 2