Statistical power and underpowered statistics
Alex Reinhart’s book Statistics Done Wrong: The Woefully Complete Guide is available online for free. Excerpt from the Statistical power and underpowered statistics chapter, describing how to look at a power graph for a scientific study:
You can see that if the coin is rigged to give heads 60% of the time, and I flip the coin 10 times, I only have a 20% chance of concluding that it’s rigged. There’s just too little data to separate rigging from random variation. The coin would have to be incredibly biased for me to always notice.
But what if I flip the coin 100 times?
You can read the full chapter here, which once again goes to show how easy it is to make numbers say what you want them to say.
In today’s age of clickbait titles, instant articles and autoplay videos, it’s becoming increasingly easy to fall for (mistakenly or maliciously drawn) wrong conclusions about the huge amounts of data that are being collected all over the place. That trend is making reading books like Reinhart’s an increasingly vital part of being a healthily skeptical (or at least less gullible) citizen of the internet.
(via Failure Is Moving Science Forward, a FiveThirtyEight essay and also a great read on this topic.)