10 sats \ 3 replies \ @m00ninite 16 Aug 2023 \ parent \ on: Setup your own private chatGPT tech
I've been considering an M1 Mac mini as an AI workhorse. The minis in particular are unusually cost-efficient for an apple machine
write
preview
0 sats \ 2 replies \ @nullama OP 16 Aug 2023
Apple silicon is a game changer.
reply
0 sats \ 1 reply \ @m00ninite 16 Aug 2023
How quick are stable diffusion and gpt with M1, in your experience? I tried running a gpt-like on some regular x86 desktop hardware and it was horrible, basically unusable.
reply
20 sats \ 0 replies \ @nullama OP 16 Aug 2023
SD, particularly from here takes a few seconds, maybe a minute or two depending on other load (I'm usually doing other stuff there) to generate a full image on an M1 mac mini. GPT responses following this particular setup should generate a response in a few seconds.
There's a lot of tweaking that can be made, in particular, if you have a GPU, you can configure a llama model to be run in parallel on the GPU.
reply