pull down to refresh
The narrative I hear out there - maybe this isn't what you personally mean - and maybe I am interpreting this wrong, is:
- It's all distillation (which is true for some but not all Chinese-made models, after all Kimi K2 was from scratch, and iirc GLM 5.0 was too)
- Therefore the source of all LLMs is the US labs
- Therefore if the US innovates on the scratch-to-base layer, the Chinese labs are helpless
All that I'm saying is that if the new tech is just some proprietary Google software then there is no moat, because software can be reverse engineered in a few days now, thanks to LLMs. See for example this repo that reverse engineered the unpublished Kimi K3 training pipeline
If there is some Nvidia or Goog hardware needed as well, then the moat will last for a few months at most; the Chinese hardware manufacturers can go from lab prototype to production at scale much faster than anyone else.
Therefore, I reject any claim of "China cannot do this". Not because I am pro-China, but because I am extremely skeptical of location/geopolitical based superiority of intellect and attitude. I think I know this to be a lie, as I've had the honor of working with people in almost every culture thinkable and the one thing we all excel at is overcoming barriers, if only we want to overcome it.
You seem to be under the impression that distillation was a bad thing?
This is not the case. This is not a "hopium" narrative. Quite the opposite: it's a pro-china narrative that makes the US situation look bleak because of how much better this strategy is