Created by Kristoffer Magnusson
Some say that a shift from hypothesis testing to confidence intervals and estimation will lead to fewer statistical misinterpretations. Personally, I am not sure about that. But I agree with the sentiment that we should stop reducing statistical analysis to binary decision-making. The problem with CIs is that they are as unintuitive and as misunderstood p-values and null hypothesis significance testing. Moreover, CIs are often used to perform hypothesis tests and are therefore prone to the same misuses as p-values.
Some points to consider:
The take home message is that we must accept that our data are noisy and that our results are uncertain. A single “significant” CI or p-value might provide comfort and make for easy conclusions. I hope this visualization shows that instead of drawing conclusions from a single experiment, we should spend our time replication results, honing scientific arguments, polish theories and form narratives, that taken all together provide evidentiary value for our hypothesis. So that we in the end can make substantive claims about the real world.
If you enjoy my work, or maybe even use it in your own teaching, please consider supporting me on Patreon. My visualizations will always be free to use, but you can show your support by donating a dollar or two for every new visualization I create.
Have any suggestion? Send them to me, my contact info can be found here.