pull down to refresh

... what will that mean for public trust in science? (rest of the title). I'm not sure I have a strong opinion yet about this second part.
What caught my eye were all the interesting ways that people seem to already have implemented to use AI to fight known weaknesses in the scientific system.
Until recently, technological assistance in self-correction was mostly limited to plagiarism detectors. But things are changing. Machine-learning services such as ImageTwin and Proofig now scan millions of figures for signs of duplication, manipulation and AI generation.
Natural language processing tools flag “tortured phrases” – the telltale word salads of paper mills. Bibliometric dashboards such as one by Semantic Scholar trace whether papers are cited in support or contradiction.
Yes, this! How many papers I've seen cited just to remind the reader about their huge methodological flaw. It's almost like making mistakes pays off in the end, in terms of citation metrics. There is no such thing as bad publicity?
AI – especially agentic, reasoning-capable models increasingly proficient in mathematics and logic – will soon uncover more subtle flaws.
For example, the Black Spatula Project explores the ability of the latest AI models to check published mathematical proofs at scale, automatically identifying algebraic inconsistencies that eluded human reviewers. Our own work mentioned above also substantially relies on large language models to process large volumes of text.
Do you know of any other potential promising uses of AI in science?
Considering the severity of the Replication Crisis, I imagine a full audit will be disastrous for trust in science (really it should just be disastrous for trust in those scientists, but people aren't very nuanced).
reply
Yeah, i did not comment on that part as that fear of losing trust in science was the main concern of OP and how to respond to it (kinda like how PR teams gear up when their politician got caught cheating) rather than thinking about how to address the underlying issue. Maybe having those AI tools is a good thing...
To be fair, reading it again now, the author is more nuanced than the impression I got from my first reading.
reply
41 sats \ 1 reply \ @gmd 27 Jul
I would love an AI tool that takes a study and a claim and asks if the study actually supports what the person is claiming, and to what degree.
There are so many BS fitness, nutrition and health grifters on twitter who careless throw around study titles and abstracts and bend the implications without reading the actual studies.
This tool could be used to read papers themselves to detect if they are erroneously citing other studies that don't support their claims.
reply
Yeah, that would be a good usecase I think.
I haven't really done it yet, but I know of some colleagues who use AI for a first reading of a paper before digging in deeper themselves. Due to recent bad experiences, I'm refraining from doing it, at least for now.
I'm sure there must be some AI wrapper somewhere, funded by millions in VC money, which basically does that.
reply
I fear AI will not be able to appreciate genuine papers with new knowledge since that would be missing in its training data. It might end up with several false negatives.
reply
Yeah, but I think the ways too use AI here is more into check basic methodology flaws, negative paper citations, etc.
I would never rely on it to assess the novelty of state-of-the-art research.
reply
Trusting in this will be shameful for the scientific community. They will exist, no doubt, but papers reviewed this way should be disregarded until they are reviewed by human peers. Just because there are bad journals and unethical reviewers doesn’t mean the process is flawed — quite the opposite, the fact that we’re aware such cases exist is what makes the system trustworthy.
reply
I still think human peer review is necessary. But there are tasks that can be realistically only be done by automated tools, at least, to the point of flagging them. A few of them were highlighted in the snippet I pasted. Humans should still assess how trustworthy the results are. Like anything spawned by current AI models, that, by definition, are stupid.
reply
Even so, every filter is a bad filter. You can’t trust these AIs, so letting them review papers and prioritize some based on whatever criteria they use is bad. I know that today the ones that stand out the most might not be the most important, but academia is still largely guided by good science and successful experiments. Nature isn’t a respected journal for nothing.
reply
Yeah, but none of the examples that I cited have anything to do with actually reviewing papers on their scientific merits.
It's about flagging tortured sentences, negative citations, figure duplication, data manipulation, algebraic inconsistencies, etc.
reply
I saw the post. Even though that’s the goal, the applications go beyond just being a compliance reviewer. Even these things should be reviewed during the journal’s editorial process and by human peers.
reply
For sure, human reviewers are still a crucial part of the process.
Autonomous AI agents are doing more bad than good, from what I've seen.
And the final communism will fall over the whole planet...
reply
You do not use any AI for any of your daily tasks? Just curious where you draw the line.
reply
none
reply
What I find interesting is the phrasing
This “science is broken” narrative undermines public trust.
I think that this is awesome. Question everything. Not just the X-tard but also the guy with the credentials to do much larger damage.
Perhaps, as the author suggests, AI will make this easier. However, thus far all I have really noticed was Qwen3 in its reasoning step finding a real error in some python code and then "deciding" to not tell me about it.
If you want something as dumb as an LLM to help you, you need to train it well. The good news is that you can train it fast.
reply
If you want something as dumb as an LLM to help you
You should not become the spokesperson of OpenAI. You are way too honest in your assessment of current LLM iterations. And probably also of future ones, unless, maybe, something drastically changes in their design.
reply
I'd do an awesome job for them if I'd agree to taking such a position. I'd also likely be forced to betray everything I believe in so there are probably better suitable candidates for that. Like anyone without a soul.
reply
stackers have outlawed this. turn on wild west mode in your /settings to see outlawed content.