I suspect, regardless of the training technique, AI makers will always benefit from more chips and more energy. If you can iterate on a model 2x, 4x, 10x faster or make it 2x, 4x, 10x smarter with 2x, 4x, 10x chips, you will desire more chips. Until there are diminishing returns with more chips and there's a bottleneck in AI elsewhere.
I suspect, regardless of the training technique, AI makers will always benefit from more chips and more energy. If you can iterate on a model 2x, 4x, 10x faster or make it 2x, 4x, 10x smarter with 2x, 4x, 10x chips, you will desire more chips. Until there are diminishing returns with more chips and there's a bottleneck in AI elsewhere.
rumor mill says China has many more nvidia chips than they claim, but are downplaying it to not breach export controls