The Real Feedback Filter
Only nature and free markets give honest feedback, everything social is noise
- Difficulty
- Easy
- Time to result
- ~weeks to results
- Steps
- 7
- Confidence
- 78%
The filter sorts every feedback source into two buckets and discards one of them. Feedback from people is usually fake: awards are fake, critics are fake, kudos from friends and family are fake. They may be trying to be genuine, but it is lost in such a sea of fakeness that real signal cannot survive. Groups are worse than individuals, because a group that does not get along decoheres, so groups optimise for consensus while individuals search for truth, and the bigger the group the less useful the output. What survives the filter is feedback that cannot be socially negotiated: free markets, where people either buy it or they do not, and physics, where the rocket either launched or it did not, the drone either flew or it did not, the 3D printer either printed within tolerance, on time and on budget or it did not. Any goal that routes through a magazine cover or an industry award is a sign the filter has failed. You need customers. That is your real feedback.
Origin
Naval builds it out of a Schopenhauer point about permission to be exceptional: you do not get to say you are exceptional, other people do, and your mom does not count.
Core principles
- 01Individuals search for truth; groups search for consensus
- 02Real feedback comes from free markets and from nature
- 03You are not the one who gets to say you are exceptional
- 04The larger the group, the less honest the signal
- 05It is easy to fool yourself, easy to be fooled by others, impossible to fool nature
How to run it
- 1
Inventory what you are listening to
Write down every source of feedback that currently changes your decisions: friends, investors, press, award panels, customers, tests.
- 2
Sort into market, nature or social
Market feedback is a purchase. Nature feedback is a physical result. Everything else (praise, criticism, awards, consensus) is social.
Pro tip If the source could change its answer to keep the relationship pleasant, it is social.
- 3
Discount the social bucket
Stop treating praise or criticism from friends, family, critics and award systems as evidence about the work.
Pro tip Keep the relationships; just remove their verdicts from your decision inputs.
Watch out Discounting praise is easy; discounting flattering praise is the hard part.
- 4
Design one harsh test
Convert the open question into something binary that reality answers: did they buy it, did it fly, did it print within tolerance.
Pro tip Specify time and cost budgets in the test so a slow, expensive pass counts as a fail.
- 5
Run it and take the answer
Execute the test and record the result as given. Physics is harsh and free markets are harsh; that harshness is the value.
Watch out Explaining away a failed market test is exactly the self-deception the filter exists to prevent.
- 6
Kill the vanity objectives
Remove awards, magazine covers and industry recognition from your goals. Optimising for them means you are failing.
- 7
Prefer individuals over groups
When you do need human input, ask individuals privately rather than groups, because groups converge on getting along.
Pro tip Ask people with skin in the outcome; they are closest to a market test in human form.
In the wild
Naval draws the line sharply for companies: if you are optimising your company to end up on the cover of a magazine or to win an industry award, you are failing. Both of those are group verdicts, produced by people managing relationships and reputations, and both can be won by a company whose product nobody buys. The replacement objective is blunt: you need customers, that is your real feedback, and it has the property that no amount of goodwill can fake it.
→ Vanity objectives removed from the goal set and replaced with a signal that cannot be socially negotiated.
For anything physical, Naval reduces the question to a binary reality answers on its own. Did your rocket launch. Did your drone fly. Did your 3D printer print the object within the tolerances it was supposed to, in the time it was supposed to, in the cost budget it was supposed to. None of these can be argued with, softened, or improved by relationships. It is very easy to fool yourself and very easy to be fooled by others, but it is impossible to fool mother nature.
→ A pass or fail that no one can talk their way around, replacing an unfalsifiable sense of progress.
Common mistakes
Counting kudos as evidence
Friends and family may genuinely try, but their verdicts are lost in a sea of fakeness and carry no information about the work.
Asking a group for the truth
Groups optimise for consensus because a group that does not get along falls apart, so the larger the group the weaker the signal.
Self-certifying excellence
Investors constantly meet people who say they are great at something and are a little delusional. You do not get to say it; results and other people do.
Is it for you?
Best for
Founders and creators deciding whether their product or work is actually working.
Not ideal for
Domains with no market or physical test available, where informed human judgement is the only signal there is.
From the transcript
“Real feedback comes from free markets in nature. Physics is harsh. Either your product worked or it didn't.”
“Individuals search for truth. Groups search for consensus.”
“If you're optimizing your company to end up on the cover of a magazine or to win an industry award, you're failing. You need customers.”
From the episode
In the Arena