"LLMs learn in the opposite direction from humans. LLMs start out learning language and attempt to abstract concepts. Human babies learn concepts first, and only later acquire the language to describe them.

So LLMs are doing it backward. In other words, perhaps reading the internet might not be the correct strategy for acquiring intelligence, artificial or otherwise."

Saw this from an article recently and it's really helped me understand what to expect (and not expect) from LLMs. Will probably make a post about it at some point.