pull down to refresh
0 sats \ 0 replies \ @random_ 30 Jul 2024 \ parent \ on: How to Run Llama 3.1 405B on Home Devices? Build AI Cluster! alter_native
https://medium.com/@isaiah_bjork/deploying-llama-3-1-405b-a-step-by-step-guide-9b1b852f3dc9
"with some optimization, we can run it on 192 gigabytes using 8x4090 GPUs"
Still not a home solution but closer.