Understanding p-values Through Simulations

An Interactive Visualization

Created by Kristoffer Magnusson

Share

P-values are often misinterpreted or misused. My goal with this page is to explain p-values through an interactive simulation. (This is an early release that is still under development!).

Cohen's d

Power Analysis: Pr(test T rejects H0; μ = 100.0) = 0.05

If we assume that the population mean is 100.0, then our test would reject the null hypothesis 5.0% of the time.

Drag

Observations per sample

n=5

Draw samples

# Draws: 0


Mean

Stats

Type I Error: 0.00

Power (true): 0.00

Effect size (true): 0.0

Effect size (sim): 0.0

Effect size (pub. bias): 0.0

Loading visualization

Written by Kristoffer Magnusson, a researcher in clinical psychology. You should follow him on Twitter and come hang out on the open science discord Git Gud Science.

Page under construction
This is an alpha version of this page, there might be some rough edges and missing features!

  • P HACK: add 1 observations to all samples that do not show a positive effect and reanalyze.
  • Severity Assessment: Mayo's severity assessment can be toggled by clicking on a sample. This only works when the sample distribution show "mean" values. The highlighted sample is draggable.

FAQ

Please report errors or suggestions by opening an issue on GitHub, if you want to ask a question use GitHub discussions

Contribute/Donate

There are many ways to contribute to free and open software. If you like my work and want to support it you can:

A huge thanks to the 119 supporters who've bought me a 268 coffees!

Jason Rinaldo bought ☕☕☕☕☕☕☕☕☕☕ (10) coffees

I've been looking for applets that show this for YEARS, for demonstrations for classes. Thank you so much! Students do not need to tolarate my whiteboard scrawl now. I'm sure they'd appreciate you, too.l

Pål from Norway bought ☕☕☕☕☕ (5) coffees

Great webpage, I use it to illustrate several issues when I have a lecture in research methods. Thanks, it is really helpful for the students:)

@MAgrochao bought ☕☕☕☕☕ (5) coffees

Joseph Bulbulia bought ☕☕☕☕☕ (5) coffees

Hard to overstate the importance of this work Kristoffer. Grateful for all you are doing.

@TDmyersMT bought ☕☕☕☕☕ (5) coffees

Some really useful simulations, great teaching resources.

@lakens bought ☕☕☕☕☕ (5) coffees

Thanks for fixing the bug yesterday!

@LinneaGandhi bought ☕☕☕☕☕ (5) coffees

This is awesome! Thank you for creating these. Definitely using for my students, and me! :-)

@ICH8412 bought ☕☕☕☕☕ (5) coffees

very useful for my students I guess

@KelvinEJones bought ☕☕☕☕☕ (5) coffees

Preparing my Master's student for final oral exam and stumbled on your site. We are discussing in lab meeting today. Coffee for everyone.

Someone bought ☕☕☕☕☕ (5) coffees

What a great site

@Daniel_Brad4d bought ☕☕☕☕☕ (5) coffees

Wonderful work!

David Loschelder bought ☕☕☕☕☕ (5) coffees

Terrific work. So very helpful. Thank you very much.

@neilmeigh bought ☕☕☕☕☕ (5) coffees

I am so grateful for your page and can't thank you enough!  

@giladfeldman bought ☕☕☕☕☕ (5) coffees

Wonderful work, I use it every semester and it really helps the students (and me) understand things better. Keep going strong.

Dean Norris bought ☕☕☕☕☕ (5) coffees

Sal bought ☕☕☕☕☕ (5) coffees

Really super useful, especially for teaching. Thanks for this!

dde@paxis.org bought ☕☕☕☕☕ (5) coffees

Very helpful to helping teach teachers about the effects of the Good Behavior Game

@akreutzer82 bought ☕☕☕☕☕ (5) coffees

Amazing visualizations! Thank you!

@rdh_CLE bought ☕☕☕☕☕ (5) coffees

So good!

@MichaMarie8 bought ☕☕☕ (3) coffees

Thanks for making this Interpreting Correlations: Interactive Visualizations site - it's definitely a great help for this psych student! 😃

Zakaria Giunashvili, from Georgia bought ☕☕☕ (3) coffees

brilliant simulations that can be effectively used in training

Someone bought ☕☕☕ (3) coffees

@PhysioSven bought ☕☕☕ (3) coffees

Amazing illustrations, there is not enough coffee in the world for enthusiasts like you! Thanks!

Cheryl@CurtinUniAus bought ☕☕☕ (3) coffees

🌟What a great contribution - thanks Kristoffer!

vanessa moran bought ☕☕☕ (3) coffees

Wow - your website is fantastic, thank you for making it.

Someone bought ☕☕☕ (3) coffees

mikhail.saltychev@gmail.com bought ☕☕☕ (3) coffees

Thank you Kristoffer This is a nice site, which I have been used for a while. Best Prof. Mikhail Saltychev (Turku University, Finland)

Someone bought ☕☕☕ (3) coffees

Ruslan Klymentiev bought ☕☕☕ (3) coffees

@lkizbok bought ☕☕☕ (3) coffees

Keep up the nice work, thank you!

@TELLlab bought ☕☕☕ (3) coffees

Thanks - this will help me to teach tomorrow!

SCCT/Psychology bought ☕☕☕ (3) coffees

Keep the visualizations coming!

@elena_bolt bought ☕☕☕ (3) coffees

Thank you so much for your work, Kristoffer. I use your visualizations to explain concepts to my tutoring students and they are a huge help.

A random user bought ☕☕☕ (3) coffees

Thank you for making such useful and pretty tools. It not only helped me understand more about power, effect size, etc, but also made my quanti-method class more engaging and interesting. Thank you and wish you a great 2021!

@hertzpodcast bought ☕☕☕ (3) coffees

We've mentioned your work a few times on our podcast and we recently sent a poster to a listener as prize so we wanted to buy you a few coffees. Thanks for the great work that you do!Dan Quintana and James Heathers - Co-hosts of Everything Hertz 

Cameron Proctor bought ☕☕☕ (3) coffees

Used your vizualization in class today. Thanks!

eshulman@brocku.ca bought ☕☕☕ (3) coffees

My students love these visualizations and so do I! Thanks for helping me make stats more intuitive.

Someone bought ☕☕☕ (3) coffees

Adrian Helgå Vestøl bought ☕☕☕ (3) coffees

@misteryosupjoo bought ☕☕☕ (3) coffees

For a high school teacher of psychology, I would be lost without your visualizations. The ability to interact and manipulate allows students to get it in a very sticky manner. Thank you!!!

Chi bought ☕☕☕ (3) coffees

You Cohen's d post really helped me explaining the interpretation to people who don't know stats! Thank you!

Someone bought ☕☕☕ (3) coffees

You doing useful work !! thanks !!

@ArtisanalANN bought ☕☕☕ (3) coffees

Enjoy.

@jsholtes bought ☕☕☕ (3) coffees

Teaching stats to civil engineer undergrads (first time teaching for me, first time for most of them too) and grasping for some good explanations of hypothesis testing, power, and CI's. Love these interactive graphics!

@notawful bought ☕☕☕ (3) coffees

Thank you for using your stats and programming gifts in such a useful, generous manner. -Jess

Mateu Servera bought ☕☕☕ (3) coffees

A job that must have cost far more coffees than we can afford you ;-). Thank you.

@cdrawn bought ☕☕☕ (3) coffees

Thank you! Such a great resource for teaching these concepts, especially CI, Power, correlation.

Julia bought ☕☕☕ (3) coffees

Fantastic work with the visualizations!

@felixthoemmes bought ☕☕☕ (3) coffees

@dalejbarr bought ☕☕☕ (3) coffees

Your work is amazing! I use your visualizations often in my teaching. Thank you. 

@PsychoMouse bought ☕☕☕ (3) coffees

Excellent!  Well done!  SOOOO Useful!😊 🐭 

Dan Sanes bought ☕☕ (2) coffees

this is a superb, intuitive teaching tool!

@whlevine bought ☕☕ (2) coffees

Thank you so much for these amazing visualizations. They're a great teaching tool and the allow me to show students things that it would take me weeks or months to program myself.

Someone bought ☕☕ (2) coffees

@notawful bought ☕☕ (2) coffees

Thank you for sharing your visualization skills with the rest of us! I use them frequently when teaching intro stats. 

Someone bought ☕ (1) coffee

Someone bought ☕ (1) coffee

Thanks a lot!

Someone bought ☕ (1) coffee

Reena Murmu Nielsen bought ☕ (1) coffee

Tony Andrea bought ☕ (1) coffee

Thanks mate

Tzao bought ☕ (1) coffee

Thank you, this really helps as I am a stats idiot :)

Melanie Pflaum bought ☕ (1) coffee

Sacha Elms bought ☕ (1) coffee

Yihan Xu bought ☕ (1) coffee

Really appreciate your good work!

@stevenleung bought ☕ (1) coffee

Your visualizations really help me understand the math.

Junhan Chen bought ☕ (1) coffee

Someone bought ☕ (1) coffee

Someone bought ☕ (1) coffee

Michael Hansen bought ☕ (1) coffee

ALEXANDER VIETHEER bought ☕ (1) coffee

mather bought ☕ (1) coffee

Someone bought ☕ (1) coffee

Bastian Jaeger bought ☕ (1) coffee

Thanks for making the poster designs OA, I just hung two in my office and they look great!

@ValerioVillani bought ☕ (1) coffee

Thanks for your work.

Someone bought ☕ (1) coffee

Great work!

@YashvinSeetahul bought ☕ (1) coffee

Someone bought ☕ (1) coffee

Angela bought ☕ (1) coffee

Thank you for building such excellent ways to convey difficult topics to students!

@inthelabagain bought ☕ (1) coffee

Really wonderful visuals, and such a fantastic and effective teaching tool. So many thanks!

Someone bought ☕ (1) coffee

Someone bought ☕ (1) coffee

Yashashree Panda bought ☕ (1) coffee

I really like your work.

Ben bought ☕ (1) coffee

You're awesome. I have students in my intro stats class say, "I get it now," after using your tool. Thanks for making my job easier.

Gabriel Recchia bought ☕ (1) coffee

Incredibly useful tool!

Shiseida Sade Kelly Aponte bought ☕ (1) coffee

Thanks for the assistance for RSCH 8210.

@Benedikt_Hell bought ☕ (1) coffee

Great tools! Thank you very much!

Amalia Alvarez bought ☕ (1) coffee

@noelnguyen16 bought ☕ (1) coffee

Hi Kristoffer, many thanks for making all this great stuff available to the community!

Eran Barzilai bought ☕ (1) coffee

These visualizations are awesome! thank you for creating it

Someone bought ☕ (1) coffee

Chris SG bought ☕ (1) coffee

Very nice.

Gray Church bought ☕ (1) coffee

Thank you for the visualizations. They are fun and informative.

Qamar bought ☕ (1) coffee

Tanya McGhee bought ☕ (1) coffee

@schultemi bought ☕ (1) coffee

Neilo bought ☕ (1) coffee

Really helpful visualisations, thanks!

Someone bought ☕ (1) coffee

This is amazing stuff. Very slick. 

Someone bought ☕ (1) coffee

Sarko bought ☕ (1) coffee

Thanks so much for creating this! Really helpful for being able to explain effect size to a clinician I'm doing an analysis for. 

@DominikaSlus bought ☕ (1) coffee

Thank you! This page is super useful. I'll spread the word. 

Someone bought ☕ (1) coffee

Melinda Rice bought ☕ (1) coffee

Thank you so much for creating these tools! As we face the challenge of teaching statistical concepts online, this is an invaluable resource.

@tmoldwin bought ☕ (1) coffee

Fantastic resource. I think you would be well served to have one page indexing all your visualizations, that would make it more accessible for sharing as a common resource.

Someone bought ☕ (1) coffee

Fantastic Visualizations! Amazing way to to demonstrate how n/power/beta/alpha/effect size are all interrelated - especially for visual learners! Thank you for creating this?

@jackferd bought ☕ (1) coffee

Incredible visualizations and the best power analysis software on R.

Cameron Proctor bought ☕ (1) coffee

Great website!

Someone bought ☕ (1) coffee

Hanah Chapman bought ☕ (1) coffee

Thank you for this work!!

Someone bought ☕ (1) coffee

Jayme bought ☕ (1) coffee

Nice explanation and visual guide of Cohen's d

Bart Comly Boyce bought ☕ (1) coffee

thank you

Dr. Mitchell Earleywine bought ☕ (1) coffee

This site is superb!

Florent bought ☕ (1) coffee

Zampeta bought ☕ (1) coffee

thank you for sharing your work. 

Mila bought ☕ (1) coffee

Thank you for the website, made me smile AND smarter :O enjoy your coffee! :)

Deb bought ☕ (1) coffee

Struggling with statistics and your interactive diagram made me smile to see that someone cares enough about us strugglers to make a visual to help us out!😍 

Someone bought ☕ (1) coffee

@exerpsysing bought ☕ (1) coffee

Much thanks! Visualizations are key to my learning style! 

Someone bought ☕ (1) coffee

Sponsors

You can sponsor my open source work using GitHub Sponsors and have your name shown here.

Backers ✨❤️

Pull requests are also welcome, or you can contribute by suggesting new features, add useful references, or help fix typos. Just open a issues on GitHub.

Webmentions

1922 404

What's this?

イリュージョン・オブ・エビデンス
#P値 #仮説検定 #信頼区間 に「数学」的な理解を要求してる人たちが真犯人かもしれない。このアプリ [rpsychologist.com/pvalue/] を数時間イジってればわかると思うが。
イリュージョン・オブ・エビデンス
同意。このようなアプリ [rpsychologist.com/pvalue/] を数時間はイジって挙動を理解しないとまず無理。ツイッター上で #医学論文 を10年以上も解説しておられる #薬剤師 集団でさえも #P値 の基本をまともに理解できないので。
2歳
2歳 2021-07-24
なにこれめちゃくちゃ面白い
fdsfdhafd
fdsfdhafd 2021-07-24
Good job 👍 By the way, can you make this a max of N=9999 ? rpsychologist.com/d3/nhst/
Ken Bodnar
Ken Bodnar 2021-07-07
Understanding p-values Through Simulations rpsychologist.com/pvalue/
Dr DD
Dr DD 2021-06-23
this is so neat, explaining p-values with a simulation built with R, is this a shiny app??? rpsychologist.com/pvalue/ via @krstoffr
Carolyn Wilshire
Carolyn Wilshire 2021-06-22
Just had a play with this. nice way of visualising what p values represent, cool teaching tool.
Dr. Jess Hartnett 📊
This guy.
Ricardo Lima Filho
Awesome tool!
Mauricio Parra, PsyD.
Great visualizations for understanding statistical concepts in psychology! #neuropsychology #psychology
Ricardo Lima Filho
Awesome tool!
Niklas Långström
Great stuff!
Mauricio Parra, PsyD.
Great visualizations for understanding statistical concepts in psychology! #neuropsychology #psychology
Niklas Långström
Great stuff!
David Speed, PhD
David Speed, PhD 2021-06-18
Hellllllloooo #PSYC6014 teaching tool...
David Speed, PhD
David Speed, PhD 2021-06-18
Hellllllloooo #PSYC6014 teaching tool...
Harald Groven
Harald Groven 2021-06-18
For verdens (trolig) aller beste interaktive simuleringer av statistiske sannsynlighetsdistribusjoner, sjekk ut nettstedet rpsychologist.com
Kristoffer Magnusson
Thanks, let me know if you think of any improvements!
Klajdi Puka
Klajdi Puka 2021-06-17
This is awesome! Thanks for creating and sharing. I'll be using it for class tomorrow, so nice to have a hands on demo
Kristoffer Magnusson
Thank you!
Francois van Heerden
This looks so good. It is seriously impressive.
Peter Steinbach
Peter Steinbach 2021-06-15
What a discovery with lovely visualisations!
Don
Don 2021-06-15
Nice!
Peter Steinbach
Peter Steinbach 2021-06-15
What a discovery with lovely visualisations!
♕Deborah Mayo♕
Thank you! I love having new severity apps--there are a few others out there (Jimenez, Morey are 2) linked to on my phil-stat-wars.com blog. I will study this one.
PhilosopherMD
PhilosopherMD 2021-06-14
Severity measurement on an app! Amazing and something I have been looking for.
Kristoffer Magnusson
That'd be cool! I will DM you my address
♕Deborah Mayo♕
Thank you! I love having new severity apps--there are a few others out there (Jimenez, Morey are 2) linked to on my phil-stat-wars.com blog. I will study this one.
♕Deborah Mayo♕
Your efforts deserve one of the remaining signed copies of Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars (CUP, 2019). If you'd like it email me your mailing address at error.
Cyril Forestier PhD
@FPommerell @LECLERCQAlicia1 un bon outil pour comprendre la valeur p
PhilosopherMD
PhilosopherMD 2021-06-14
Severity measurement on an app! Amazing and something I have been looking for.
Andrew Perfors
Andrew Perfors 2021-03-26
This is great!
Thom Baguley
Thom Baguley 2021-03-26
There's always stepwise regression ...
Schotz
Schotz 2021-03-26
Ive always wished my statistical software came with a “p-hack” button.
Kristoffer Magnusson
Yeah, they work great for calculations that can be truly run in parallel to the UI thread
Richard D. Morey
Richard D. Morey 2021-03-21
just figured out web workers for an app; the setup is annoying, but they're effective!
Tanner Delpier
Tanner Delpier 2021-03-20
This is so satisfying... rpsychologist.com/pvalue/
Nona
Nona 2021-03-20
Brilliant!
Kaleb Mathieu
Kaleb Mathieu 2021-03-13
rpsychologist.com/pvalue/ Fantastic visualization by @krstoffr to help understand what a p-value is. #AcademicTwitter #stats
Santiago Silvestrini
Muy buen recurso interactivo!
Daniel Wiczew
Daniel Wiczew 2021-03-09
Want to play with m̶e̶ some statistics ? rpsychologist.com/pvalue/ #visualization #Statistics
Per Damkier
Per Damkier 2021-03-08
Folks, this is rather awesome👇👏
Dominique Roche
Dominique Roche 2021-03-08
Amazing!
megan peters
megan peters 2021-03-08
the world can never have enough of these beautiful, simple, easily-understood visualizations of basic stats concepts ❤ stats is hard for many. i also used to find stats hard. so here's another tools to add to our repertoire, to help build our students' intuitive understanding.
megan peters
megan peters 2021-03-08
the world can never have enough of these beautiful, simple, easily-understood visualizations of basic stats concepts ❤ stats is hard for many. i also used to find stats hard. so here's another tools to add to our repertoire, to help build our students' intuitive understanding.
Dominique Roche
Dominique Roche 2021-03-08
Amazing!
Hacker News記事題日本語翻訳
シミュレーションによるP値の理解–インタラクティブな視覚化 rpsychologist.com/pvalue/
Hacker News 50
Hacker News 50 2021-03-08
Understanding P-Values Through Simulations – An Interactive Visualization rpsychologist.com/pvalue/ (news.ycombinator.com/item?id=263762…)
Doğu
Doğu 2021-03-08
This can truly come in handy rpsychologist.com/pvalue/
Zieloli
Zieloli 2021-03-07
P-values are often misinterpreted or misused. The goal of this page is to explain p-values through an interactive simulation. rpsychologist.com/pvalue/
AV Speech Processing
Do you ever have to explain p-values to students? (I know I do). Take a look at "Understanding p-values through simulations - an interactive visualization" by @krstoffr rpsychologist.com/pvalue/ 👏👏
Hacker News 20
Hacker News 20 2021-03-07
Understanding P-Values Through Simulations – An Interactive Visualization rpsychologist.com/pvalue/ (news.ycombinator.com/item?id=263762…)
HN Front Page
HN Front Page 2021-03-07
Understanding P-Values Through Simulations – An Interactive Visualization L: rpsychologist.com/pvalue/ C: news.ycombinator.com/item?id=263762…
Winson Tang
Winson Tang 2021-03-07
Understanding P-Values Through Simulations – An Interactive Visualization rpsychologist.com/pvalue/?utm_so…
Hacker News
Hacker News 2021-03-07
Understanding P-Values Through Simulations – An Interactive Visualization: rpsychologist.com/pvalue/ Comments: news.ycombinator.com/item?id=263762…
Ruben Dario Palacio
I'd say abandon p-values but this is how statistics should be taught!
Polina Beloborodova
The most straightforward demonstration of p-value issues that I’ve ever seen.
Mathieu M.J.E. Rebeaud
Oooh c'est un beau joujou pour les stateux.
Jorge Joo
Jorge Joo 2021-03-07
Understanding p-values through simulations - an interactive visualization by @krstoffr rpsychologist.com/pvalue/
Sarah H
Sarah H 2021-03-07
https://t.co/OXlkawMVOd
manulu
manulu 2021-03-07
Do you plan on implementing a simulation that shows the central limit theorem? :)
321
321 2021-03-06
Understanding p-values | R Psychologist rpsychologist.com/pvalue/
Sarah Myers
Sarah Myers 2021-03-05
THIS IS SO HELPFUL
Begüm Özkısaoğlu
This is cool!
Kristin
Kristin 2021-03-05
rpsychologist.com/pvalue/
Antoine 10km
Antoine 10km 2021-03-05
Voilà qui a autrement plus de tronche que mes petits graphiques tiens.
Duhyadi Oliva García
Puntitos
Dr. Jennifer Provencher, wears a mask
Ping @heathmacmillan
Oriane Armand
Oriane Armand 2021-03-05
Covering tests and plots in R will already be quite dense but I’ll see if I can touch upon it 😉
Kristoffer Magnusson
Thanks! I've opened an issue for this
Peter Dahlgren
Peter Dahlgren 2021-03-05
Jättebra som vanligt!
Nicolas
Nicolas 2021-03-05
😍
Nicolas
Nicolas 2021-03-05
@susyandrausdl
BOT RESERVA 😡 ( em ☁️ ) ( LEIA O FIXADO )
(¯ □ ¯ 」) baixaessaporra.com/video/13673862…
Constant Pieters
Constant Pieters 2021-03-05
Awesome, its strength is in the animations. Two suggestions: app crashes when n goes to 0; might be useful to update n when adding "p-hack" given its operationalization (add 1 observations to all samples that do not show a positive effect and reanalyze).
stasha.antonijevic
@Renagalway
Simon Dellicour
Simon Dellicour 2021-03-05
@FMassonnet
Dr Anastasia S Mihailidou FAHA FCSANZ
This is definitely #MedEd #CardioEd #scicomm💥& thank you @krstoffr
MammothReg l'Amish des 🐘
big fan! thanx
David Hajage #JeSuisVacciné
Poke @Nibor_Tolum @Clara_Locher @SagittariusHH C'est beau.
Sarah Myers
Sarah Myers 2021-03-05
THIS IS SO HELPFUL
louislongin
louislongin 2021-03-05
@oriane_armand maybe we can cover this next week..?
Valmir Matos
Valmir Matos 2021-03-05
Congrats from Brazil! It's awesome!!
David Colquhoun 💙
Another quotation from that. "Harold Jeffreys recommends the lump prior only to capture cases where a special value of a parameter is deemed plausible" Sadly in many experiments, zero (or near zero) effects are only too plausible. @dnunan79
Tiago André Marques
Cool visualization!
Homero San Juan
Homero San Juan 2021-03-04
Excelente visualización para entender qué significan los valores p en estadística, usando simulaciones
Luis Ortega Paz
Luis Ortega Paz 2021-03-04
#EAPCI
Begüm Özkısaoğlu
This is cool!
Polina Beloborodova
The most straightforward demonstration of p-value issues that I’ve ever seen.
David Colquhoun 💙
Steven Goodman pointed out years ago that p=0.05 corresponds to a likelihood ratio of about 3. Odds of 3:1 are not very impressive
David Colquhoun 💙
It isn't a *huge* prior -in the absence of good date, ie almost always, it's unreasonable to postulate a value >0.5
♕Deborah Mayo♕
Read: Who's exaggerating what? errorstatistics.files.wordpress.com/2020/05/ex4-ti…
David Colquhoun 💙
I have read it several times. You say "This might only mean it’s worth getting more data to probe for a real effect.". That is how I would interpret p=0.05. But it's far from the interpretation used in practice.
♕Deborah Mayo♕
posteriors using p= have the same deficiencies & are completely distinct from our goals in hypotheses tests
♕Deborah Mayo♕
giving the null a huge spiked prior is radically at odds with how we understand null hypotheses, not to mention, they aren't proper frequentist priors. but we've been through this too many times.
David Colquhoun 💙
PPV is different. It uses the p-less-than definition for likelihood ratio which is inappropriate for tests designed to tell you whether an effect is chance or real. For that you need p-equals calculation of likelihood ratio (or BF) royalsocietypublishing.org/doi/full/10.10…
David Colquhoun 💙
The screening test analogy is not necessary, nor is it an exact analogy, for the reasons I just gave.
David Colquhoun 💙
The crucial assumption in my approach and several others has nothing to do with screening tests. Rather it's that it's sensible to test a point (or near-point) null. I think that that is often the case (but not always).
♕Deborah Mayo♕
this is according to YOU, but not according to people who know how to use p-values correctly & are not trying to fit hypothesis testing into the mold for PPV screening measures.
Marce Cvallos
Marce Cvallos 2021-03-04
I love this! 😍
David Colquhoun 💙
The problem, surely, is that the simulation doesn't distinguish between type 1 error and false positive risk. So it doesn't really help people to understand what's bad about p values. tandfonline.com/doi/pdf/10.108… and fpr-calc.ucl.ac.uk
Dominik Liebl
Dominik Liebl 2021-03-04
P p pppp p-values. Explained through simulations. (Simulations: the only way to understand frequentist statistics ... since simulations allow frequent repetitions.)
Gavin Buckingham
Gavin Buckingham 2021-03-04
This is gorgeous
Luis Benites
Luis Benites 2021-03-04
Very nice! Congrats.
Kai-Markus Mueller
Super awesome, my students will love this!!!
dolores frias-navarro
Thank you
Gareth J. Harvey
Gareth J. Harvey 2021-03-04
So many people (students, lecturers & practitioners) really don't understand P values. This interactive simulation from @krstoffr should help make it a little clearer. I love how it features both p hacking & publication bias. rpsychologist.com/pvalue/
cbergenholtz
cbergenholtz 2021-03-04
Great illustration! Very timely, I can use it in my philosophy of science class today, where we discuss Carney's 'retraction' of her power pose paper (with Cuddy), and more generally how and why false positives appear in the literature.
Dominik Liebl
Dominik Liebl 2021-03-04
P p pppp p-values. Explained through simulations. (Simulations: the only way to understand frequentist statistics ... since simulations allow frequent repetitions.)
Julia Jerke
Julia Jerke 2021-03-04
Awesome! I really like that it integrates p-hacking (here optional stopping: sampling until significance is reached) to illustrate how it contributes to inflated results and, eventually, publication bias. I may borrow this for my teaching 😃
Gavin Buckingham
Gavin Buckingham 2021-03-04
This is gorgeous
Max Berggren 🐍🐼
A++ for under construction gif
Julia Jerke
Julia Jerke 2021-03-04
Very nice! I really like how it features p-hacking (here in the form of sampling until significance is reached) to illustrate how it contributes to inflated results and, eventually, publication bias.
♕Deborah Mayo♕
It would be great to give corresponding severity assessments.
Raimondo Bruno
Raimondo Bruno 2021-03-04
OMG this is GREAT!!!!

(Webmentions sent before 2021 will unfortunately not show up here.)

More Visualizations

Understanding p-values Through Simulations

An interactive simulation to help explain p-values

Maximum Likelihood

An interactive post covering various aspects of maximum likelihood estimation.

Cohen's d

An interactive app to visualize and understand standardized effect sizes.

Statistical Power and Significance Testing

An interactive version of the traditional Type I and II error illustration.

Confidence Intervals

An interactive simulation of confidence intervals

Bayesian Inference

An interactive illustration of prior, likelihood, and posterior.

Correlations

Interactive scatterplot that let's you visualize correlations of various magnitudes.

Equivalence and Non-Inferiority Testing

Explore how superiority, non-inferiority, and equivalence testing relates to a confidence interval

P-value distribution

Explore the expected distribution of p-values under varying alternative hypothesises.

t-distribution

Interactively compare the t- and normal distribution.