pull down to refresh

244 sats \ 7 replies \ @freetx 15 Oct \ on: The AI Industry's Scaling Obsession Is Headed for a Cliff AI
In other words, meek models (those with limited computation budget) shall inherit the earth, approaching the performance level of the best models overall.
I actually have a similar take. The diminishing returns combined with retail hardware advances are going to make it that in 2030 your average home computer will run a model that more or less performs similar to todays frontier models.
We've seen this same thing constantly. There was a time when an iPhone 4 could do a tremendous amount more than an iPhone 1....but that hasn't been true for a long time. Does the average phone user really need anything more advanced than say a iPhone 9? Text, Photos, Emails, Maps, etc....basically its all become a "solved problem".
I think the same is going to happen in AI. At a certain point, the day-to-day use cases are going to be solved by open source models running on commodity hardware.
write
preview
51 sats \ 6 replies \ @0xbitcoiner OP 15 Oct
I don't think we'll have to wait that long.
NVIDIA DGX Spark Arrives for World’s AI Developers #1256239
reply
33 sats \ 5 replies \ @optimism 15 Oct
I want one but its sold out. Oh and its probably expensive haha
reply
144 sats \ 3 replies \ @freetx 15 Oct
I'm waiting for more comparisons between AMD AI 395 and the DGX.
Main benefits of DGX is that NVidia tooling and ecosystem is so much better....however raw CPU of AMD is probably faster.
Preliminary testing on https://www.reddit.com/r/LocalLLaMA/comments/1o6izz2/dgx_spark_vs_ai_max_395/ seems to indicate that AMD AI 395 wins (also ~33% cheaper)
reply
83 sats \ 2 replies \ @optimism 16 Oct
llama.cpp was in need of some impactful Cuda tuning until yesterday: https://github.com/ggml-org/llama.cpp/pull/16585 and probably still is.
This has been over the decade(s) the persistent issue with Cuda (complexity, cross-version instability/regressions) and why I'd principally love to get some AMD tech. But what I hear from friends working with Vulkan, it has its own problems.
My biggest issue is that Metal works almost out of the box, but the cost is steep...
reply
100 sats \ 1 reply \ @0xbitcoiner OP 16 Oct
I didn’t even know what Vulkan was until I installed the new Civilization 7. When you launch the game, it lets you pick between DirectX12 and Vulkan. Fun fact: the game ran like crap on my old graphics card, and it wouldn’t even start with DirectX12. But with Vulkan? It actually worked! Hahaha
reply
33 sats \ 0 replies \ @optimism 16 Oct
TIL DirectX still exists lol
reply
100 sats \ 0 replies \ @0xbitcoiner OP 15 Oct
sold out? Noooooo, sales kicked off today.
https://marketplace.nvidia.com/en-us/developer/dgx-spark/
https://www.microcenter.com/product/699008/nvidia-dgx-spark
reply