pull down to refresh
21 sats \ 2 replies \ @Undisciplined 26 May \ parent \ on: How can we trust science, if we can't trust scientists? HealthAndFitness
Many problems have emerged from this ideal:
- Very few people do replications;
- Peer-review entrenches the status quo, because that's what all the peers believe;
- Null results don't get published, so people fudge their data and methods to generate significant differences,
- Fewer and fewer people are interested in being peer reviewers.
- Research has become so specialized and complex that there aren't enough peers to adequately assess quality.
Those were the first few that came immediately to mind.
The truth is that there are only imperfect verification systems, so we need to stop treating any verification method as though it's perfect.
The ultimate test of scientific research is when we try to put the knowledge into action and see if the world works as the research proposed. That feedback can take a very long time, though.
It's not perfect, but it's a valid protocol, and if followed strictly, it ensures good scientific work. The points you raised are definitely valid, and I agree with them—this is where my criticism comes in regarding the lack of people available to evaluate quality published work. Today, there are too few peer reviewers, which is why it takes so long to publish a reviewed paper. I don't necessarily see this as a problem if those who do review are known for fairness and objective evaluations. But this highlights the need for more people to review papers, to help ensure the quality of the journals that publish them.
In summary, I partially agree with the points you raised. This isn't a problem with the protocol itself, but with us—not knowing how to properly assess good research. Especially the major news outlets, which do a poor job of communicating scientific news. They often don't even do the bare minimum, like reading the actual paper and verifying whether all the peer review steps, data, and methodologies were published before releasing sensationalist headlines.
reply
Things that work on paper, but not in practice, don't actually work on paper. They were just incompletely described.
I think we'll move towards a more open review protocol. In many ways, we already are. I haven't fully fleshed out what I think that will look like, but it's been on my mind for a while.
reply