Test what, not how

Think of it when you write a test, or a small change inside the code breaks many tests.

In everyday life

Like a food inspector who tastes the soup and checks its temperature, not which spoon the cook used. A new spoon should not fail the inspection.

One case

A ride-sharing app’s fare function has 12 tests. A developer rewrites its inside to make it faster, and every fare it returns stays the same. 9 of the 12 tests fail, because they checked which helper ran and in what order. The developer spends a day fixing tests that found no bug.

The fares did not change, and 9 tests failed

The case, cell by cell1 cell = 1 test

Should pass (no fare changed)

passpasspasspasspasspasspasspasspasspasspasspass

Result after the rewrite

passfailfailfailpassfailfailfailfailpassfailfail

Tests that failed

Fares that changed

Every red cell is a test that checked how the function worked, not what it returned.

Every case

A test is a claim about what the code does for its users. Judge it with two imagined edits. Break the result: a good test fails. Rewrite the inside and keep every result the same: a good test still passes. A test with an assertionThe line in a test that states what must be true, such as “the fare is $12.40”. about the inside does the opposite. It breaks on every rewrite and often misses real bugs, so people learn to fear change and to ignore red.

The word

Engineers call this

behavior testsTests that check what a user of the code can see: the result, the error, the saved record. They do not check how the code does the work inside.

Test what the code does, not how it does it. A good test fails when the result breaks and survives a rewrite of the inside.

Where this idea leads

lets you safelySame behavior, new shapeFoundations, How code is organized
goes in through theWhat you call vs how it worksFoundations, How code is organized
runs on every change inSame checks, every timeHabits, Before you ship

Skills that encode it

principle-test-behavior-not-implementation pstackJudge each test with two imagined edits: break the behavior and it must fail; change only the internals and it must pass.
tdd Matt PocockTests verify behavior through public interfaces. A test that breaks on a refactor with no behavior change is an anti-pattern.