pull down to refresh
204 sats \ 2 replies \ @SimpleStacker OP 29 Jan \ parent \ on: Is DeepSeek a game changer for AI? - Computerphile tech
Damn, I still have a lot to learn. But that blog also confirms what I've been learning, that it's the Reinforcement Learning part that's the real big innovation.
As to the cost, the blog says this:
So, the cost figure is definitely a lot more believable if you consider that it was built on top of previously trained models and weights. I think the way some people were talking about it, they made it seem like they just did it from scratch.
I think the ChatGPT scraping is still unverified afaik. They might've used it to build the RL training data I guess.
In this blog that gets a lot more technical they note that the optimizations they made were obviously good targets for optimization in retrospect:
None of these improvements seem like they were found as a result of some brute-force search through possible ideas. Instead, they look like they were carefully devised by researchers who understood how a Transformer works and how its various architectural deficiencies can be addressed.
reply
Fascinating
reply