They can also use APIs such as search engines. The system described in the paper uses NNs on Huggingface that most certainly can be optimised much more. Much more. I think it is over for us humans in our capacity as being intellectually superior. But we can dance ballet and the LLMs can't.