pull down to refresh
51 sats \ 6 replies \ @0xbitcoiner OP 15 Oct \ parent \ on: The AI Industry's Scaling Obsession Is Headed for a Cliff AI
I don't think we'll have to wait that long.
NVIDIA DGX Spark Arrives for World’s AI Developers #1256239
reply
I'm waiting for more comparisons between AMD AI 395 and the DGX.
Main benefits of DGX is that NVidia tooling and ecosystem is so much better....however raw CPU of AMD is probably faster.
Preliminary testing on https://www.reddit.com/r/LocalLLaMA/comments/1o6izz2/dgx_spark_vs_ai_max_395/ seems to indicate that AMD AI 395 wins (also ~33% cheaper)
reply
llama.cpp was in need of some impactful Cuda tuning until yesterday: https://github.com/ggml-org/llama.cpp/pull/16585 and probably still is.This has been over the decade(s) the persistent issue with Cuda (complexity, cross-version instability/regressions) and why I'd principally love to get some AMD tech. But what I hear from friends working with Vulkan, it has its own problems.
My biggest issue is that Metal works almost out of the box, but the cost is steep...
reply
I didn’t even know what Vulkan was until I installed the new Civilization 7. When you launch the game, it lets you pick between DirectX12 and Vulkan. Fun fact: the game ran like crap on my old graphics card, and it wouldn’t even start with DirectX12. But with Vulkan? It actually worked! Hahaha
sold out? Noooooo, sales kicked off today.
reply