Unpredictable Rewards: Why the Variable Schedule Is the Hardest to Leave
A reward that arrives on a predictable count is easy to walk away from once you have it. A reward that arrives after an unpredictable number of tries is the pattern that keeps people going longest. Learning research has a name for it: the variable-ratio schedule.
Four ways a reward can arrive
Learning from consequences is studied under the heading of operant conditioning, and the timetable on which a reward turns up is called a schedule of reinforcement. Four basic schedules are usually distinguished.
- Fixed ratio. A reward after a set number of actions, like a free coffee on every tenth cup.
- Variable ratio. A reward after an unpredictable number of actions, which only averages out over a long run.
- Fixed interval. A reward for the first action after a set stretch of time, like a monthly salary.
- Variable interval. A reward that becomes available after unpredictable stretches of time, like checking whether a reply has arrived.
These schedules were mapped systematically in laboratory work in the 1950s associated with the American psychologist B. F. Skinner. The result that has lasted is comparative: of the four, the variable ratio produces the steadiest and most persistent behaviour, and it is the slowest to fade once rewards stop altogether.
Why the unpredictable version is the stubborn one
Three things are going on at once.
There is no place in the count to stop. On a fixed ratio you always know where you stand: the reward is three cups away, and after it the cycle visibly restarts. On a variable ratio every single try has the same standing as the last. No point in the sequence feels like a natural end, because none of them is one.
A dry run is not evidence of anything. If rewards are withdrawn completely, behaviour eventually stops; psychologists call that extinction. A variable schedule makes extinction slow, because long gaps with nothing are a normal feature of the schedule rather than a sign that it has ended. The experience of "nothing for a while, then something" has already been taught as the ordinary state of affairs.
The gap gets filled with a story. Suppose, purely as an illustration, that a game returns something on roughly one try in ten, scattered at random. Over any short stretch the returns will cluster: five in quick succession, then forty with nothing. Those clusters are what randomness looks like, but they read as warm and cold spells, and a run of nothing starts to feel like credit being accumulated towards a payout.
Where the pattern shows up
Any game of chance with repeated small stakes is a variable-ratio schedule by design. So are several things that are not games: boxes with randomised contents, a feed that is pulled down to see what has appeared, a notification badge checked between tasks. The schedule is not inherently sinister, since fishing and knocking on doors have the same shape, but recognising it explains why those particular activities are hard to put down at any specific moment.
Anticipation is the part that drives it
Work on reward signalling in the brain, much of it concerning dopamine, points to something counterintuitive: these systems respond strongly to the anticipation of a reward and to rewards that arrive unexpectedly, rather than simply to the pleasure of receiving one. Uncertainty is not a tolerated cost of the reward. Uncertainty is doing much of the work.
That is also why the moment after a payout is rarely the moment people leave. The payout is the end of one piece of anticipation and the start of the next.
What follows in practice
- Bring your own stopping cue. The schedule will not supply one, so the amount and the time have to be fixed before you begin and enforced from outside: an alarm, not a feeling.
- Distrust the sense of being owed. On a random schedule, a long run of nothing changes neither the odds of the next try nor what you are entitled to.
- Add friction. Anything that puts a gap between the impulse and the next try, whether that is logging out, removing a saved payment method or leaving the device in another room, works on the one part of the loop you control.
- Name it when you notice it. "This is a variable schedule and I am in the middle of a dry run" is a surprisingly effective thing to say to yourself, because it describes the design rather than your luck.
If stopping has stopped being a matter of rules, see the diagnostic criteria and help and resources.