0 sats \ 4 replies \ @OT 15 Jun \ on: Making my local LLM voice assistant faster and more scalable with RAG tech
Pretty crazy.
I guess us non-nerds are going to get something like this in 5-10 years.
reply
reply
Yes. Its getting much easier. You can easily spin up a back end now without much tech knowledge. You don't even need a GPU (although that speeds things up significantly).
If you're interested, look into Ollama or LM Studio. They provide APIs to interface with the LLMs. Then there are a bunch of clients you can install and point to this endpoint.