In 1948 the pollster George Gallup wrote that the first question people asked about any poll was always the same: how many persons were included? In the same pages he wrote: "A sample of 10,000,000 improperly selected can be less accurate than a sample of 100 properly selected."

Think of soup. Leave the salt in one layer and a spoon from the top says it's all salt. Stir it, and one spoonful tells you the pot.

So we ran it. A pot of 10,000 dots for the town, a pot of 340 million for America, half of each orange. We scooped 1,000 at random, a thousand times from each. The two piles came out the same width: about 19 polls in 20 within 3 points of the truth. The size of the pot barely matters; the size of the spoon does. Statisticians call that plus or minus 3 the margin of error: 1.96 times the square root of (0.25 divided by 1,000) is about 3.1 points, and the size of the country isn't in it. The town's pile is a touch narrower, because there the spoon is a tenth of the pot.

The irony: Gallup's own interviewers chose whom to ask, filling quotas. That same year his final poll had Dewey ahead, Truman won, and pollsters moved to picking people at random.

The soup is our picture, not Gallup's words. His own comparison was a photograph, which "can be reduced in size and still remain a truthful and accurate portrait."

So, how many did you guess?

Sources: George Gallup, A Guide to Public Opinion Polls, Princeton University Press, 1948; U.S. Census Bureau, Vintage 2024 population estimates, December 2024; Andreas Philippou, "Why Do Polls Fail?", arXiv, 2021; our own simulation.
Sounds: our own, made in code (earshot, CC0).