Idk how to phrase this question because in a way humanity achieved an internet with more information than every single human ever could know a long time ago. Especially the width of topics is astonishing.
But will AI chatbots like chatGPT or Bard ever know stuff that no human does? Will it become smarter in a supernatural way? Beginning with extrapolating findings from academic papers before humans have or making conclusions from scraping huge amounts of data before humans came to the idea.

First Arguments (pro & contra)

  • On one hand, yes of course, AI has become smarter and smart basically in a straight (or even exponential?) line up. Why should it stop an an arbitrary line?
  • We also created machines that move faster than humans, machines that are better at flying than humans, machines that are better at remembering than humans, machines that are more precise at small things than humans... why should smarts be the exception
  • On the other hand we are ACTIVELY training AI exactly against that all the time. Extrapolate an idea from two existing papers - boom punished. Make up a fact - boom punished. Adverserial training and human labeling might be in the way (or helpful instaed of "in the way" depending on how you see it)
  • AI just reflects our biases back at us. Famous urban ledgends that are untrue that get parroted back at us. AI art just reflect our beauty standards back at us. It might just reflect our intelligence back at us too

Why is this interesting

I think that is a super interesting topic since many tech-illiterate people and jounalists assume the answer to be yes without ever questioning this premise or having it said out loud.
What would the world look like if the answer is yes? Like literally talking to the all knowing machine. Does life even have a meaning after you met an oracle?
Is there even a thing like objective intelligence if we can't judge it by measuring it against ourselves? Would the world like go bananas if the first math problem is solved by AI - especially something humanity has thought about for a long time and not something nobody ever was creative enough to ask.
I could have fever dreams from that. On the day I learned about chatGPT I went through my "small python scripts I should do but am to lazy to spend an afternoon on" list and generated them instantly and they worked almost all first try. That they I had a fever dream being lost in Ether and it took me half an hour the next morning to have a coherent thought again. (Caveat: to be fair I am subsceptible to that...)

Finally

So: Can chatbots become smarter than humans? If yes, why/how? And when (guess)? If no, why/where is the ceiling?
Smart in what capacity? In the capacity that they can take over basic customer service functions yes. But in the capacity that they need to make nuanced decisions sometimes based on a deep understanding of emotions that will not be possible with any current technology. Brains are not binary machines, so you cannot build an AI to compete with the human brain on a binary system.
reply
it doesn't "know" anything. It's a system that predicts the next word in a text it generates; that's literally all it does. If it gives an answer you don't like, you tell it no and it re-generates the opposite answer based on next word prediction with a different prompt. It's not intelligent, or smart, or knowledgeable. We really have to stop falling for that "I" propaganda in the name.
reply
That's irrelevant imo. It could generate a word after a scentence it wrote before that contains brilliant information. It already does so to e.g. write sourcecode that nobody has written before.
By your same argument you could also argue humans were incapable of new thoughts. Human brains also only remix thoughts that it heard before and extrapolate from there. That's not that different
reply