15 sats \ 4 replies \ @nullama OP 15 Aug 2023 \ parent \ on: Setup your own private chatGPT tech
It really depends on what you want to achieve.
A reasonable laptop can run this GPT4ALL and StableDiffusion quite well.
Probably the best deal would be a mac mini with at least an M1 chip, those are extremely powerful machines, and their are silent. They do run both GPT and SD without any issues.
write
preview
10 sats \ 3 replies \ @m00ninite 16 Aug 2023
I've been considering an M1 Mac mini as an AI workhorse. The minis in particular are unusually cost-efficient for an apple machine
reply
0 sats \ 2 replies \ @nullama OP 16 Aug 2023
Apple silicon is a game changer.
reply
0 sats \ 1 reply \ @m00ninite 16 Aug 2023
How quick are stable diffusion and gpt with M1, in your experience? I tried running a gpt-like on some regular x86 desktop hardware and it was horrible, basically unusable.
reply
20 sats \ 0 replies \ @nullama OP 16 Aug 2023
SD, particularly from here takes a few seconds, maybe a minute or two depending on other load (I'm usually doing other stuff there) to generate a full image on an M1 mac mini. GPT responses following this particular setup should generate a response in a few seconds.
There's a lot of tweaking that can be made, in particular, if you have a GPU, you can configure a llama model to be run in parallel on the GPU.
reply