pull down to refresh

21 sats \ 1 reply \ @SimpleStacker 29 Jul 2024 \ on: How to Run Llama 3.1 405B on Home Devices? Build AI Cluster! alter_native
Eh, not really a home solution. To run Llama 3.1 405B which requires >800GB of RAM you still need devices at home totalling more than this (over 10 devices with 64 GB for example). Not exactly something people are gonna have "at home"
write
preview
0 sats \ 0 replies \ @random_ 30 Jul 2024
https://medium.com/@isaiah_bjork/deploying-llama-3-1-405b-a-step-by-step-guide-9b1b852f3dc9
"with some optimization, we can run it on 192 gigabytes using 8x4090 GPUs"
Still not a home solution but closer.
reply