I'm refereeing a paper for a pretty good journal. It's a joint theory-experimental paper. I looked at the paper yesterday and realized the authors overlooked a possible theoretical interpretation for their experimentally observed features. To support my report, and out of interest, I've checked this interpretation myself through simulation. Without access to the frontier LLMs, I'd never have bothered, but here, in less than 24 hours, I've confirmed that this alternative interpretation is the most likely one.
If I were acting in bad faith[1], I could stall sending in the report, prepare a simulation-only paper in a few days, put it on the arXiv, which would essentially scoop their work.
We've read reports of scientists getting scooped by the companies behind the proprietary LLMs, this shows another situation where the current system is dealing with unexpected cracks...
I'm not, so I will just ask them to consider this specific scenario, so they have a chance of saving their paper. And potentially also very much increase its impact, as it would be the first experimental observation of that specific scenario. They could even consider retracting the submission to this journal and going for a Nature family journal. ↩
How many human systems work by relying on the difficulty, slowness of reasoning? I have a feeling it is many more than we think because we take it for granted.
Interesting situation just occurred.
I'm refereeing a paper for a pretty good journal. It's a joint theory-experimental paper. I looked at the paper yesterday and realized the authors overlooked a possible theoretical interpretation for their experimentally observed features. To support my report, and out of interest, I've checked this interpretation myself through simulation. Without access to the frontier LLMs, I'd never have bothered, but here, in less than 24 hours, I've confirmed that this alternative interpretation is the most likely one.
If I were acting in bad faith[1], I could stall sending in the report, prepare a simulation-only paper in a few days, put it on the arXiv, which would essentially scoop their work.
We've read reports of scientists getting scooped by the companies behind the proprietary LLMs, this shows another situation where the current system is dealing with unexpected cracks...
I'm not, so I will just ask them to consider this specific scenario, so they have a chance of saving their paper. And potentially also very much increase its impact, as it would be the first experimental observation of that specific scenario. They could even consider retracting the submission to this journal and going for a Nature family journal. ↩