0 sats \ 0 replies \ @nullama OP 16 Aug 2023 \ parent \ on: Setup your own private chatGPT tech
Interesting.
I think there's no GPU acceleration at all by default at least on this setup, so basically the entire speed is based on the CPU and the instructions it supports. The more modern the CPU, the better.
I've read that you can configure llama to use the GPU to get much better results