pull down to refresh

Replication crisis in biology: p-hacked hypothesises
420 sats \ 12 comments \ @zuspotirko 3 May science
The figure in this article illustrates exactly how most biology papers are secretly p-hacked. A large number of hypotheses is explored, and only the ones that form a coherent story are reported.
This is actually the main reason behind the replication crisis in biology IMO. (source)
https://www.nature.com/articles/s41587-025-02635-7
write
preview
related posts
47 sats \ 9 replies \ @Undisciplined 3 May
And it's not just biology. Every scientific field has the same problem, for the same reasons.
This is why prespecification is the gold standard approach.
reply
34 sats \ 8 replies \ @SimpleStacker 3 May
I'm actually a bit more sympathetic to "exploring hypotheses" and don't consider it p-hacking per se.
Sometimes you get a big dataset and you're looking to uncover patterns and causal relationships. Exploring various hypotheses, some of which will fail and some of which will succeed, is entirely appropriate
In my opinion the problem of p-hacking is promoting a hypothesis that isn't robust by only presenting the specifications that give you a good enough p-value
Another problem is publication bias in not publishing null results, which makes scientific consensus seem more certain that it really is
reply
0 sats \ 7 replies \ @Undisciplined 3 May
I completely agree. I don't think it's necessary to confine yourself to the prespecification, but I think it's good to have it and to report what is found when doing it.
Since nothing ever goes according to plan, though, I'm fine with exploring other possibilities from there.
In one sense, you could just raise the standards for statistical significance to account for how many specifications you explored. The problem is looking at 100 things and then reporting as though the 5% significance on the coefficients from one specification is very meaningful.
reply
40 sats \ 6 replies \ @SimpleStacker 3 May
Agree that preregistered studies would go a long way
We're far from getting anything close to nuance though given how lacking statistical education is. I literally had a biology phd teach my kid that a p-value of 0.05 or below means that X causes Y 🤦🏻‍♂️
reply
41 sats \ 2 replies \ @Fabs 3 May
Damn you two seem highly educated.
reply
0 sats \ 1 reply \ @Undisciplined 3 May
I can read between the lines
view all 1 replies
0 sats \ 2 replies \ @Undisciplined 3 May
It's truly amazing how poorly most empirical disciplines understand empirical science.
reply
32 sats \ 1 reply \ @SimpleStacker 3 May
This is one area that I found the natural sciences surprisingly lacking in. I'm really glad for my econometrics education in that regard
view all 1 replies
18 sats \ 0 replies \ @aPolitical 3 May
Grrrh... I wanted to ask a question about a certain grammar I just discovered while reading the post, but I can't find the grammar again. Haba la wa o
reply
0 sats \ 0 replies \ @nitter 3 May bot
https://xcancel.com/DouglasYaoDY/status/1918198785232257298
reply