pull down to refresh
qwen2.5:14b is a valid answer! I wonder one thing tho: why Qwen 2.5? It's from September 2024.
Wanted something really lightweight to play around with for a 16gb machine.
Anything heavier you recommend?
Turns out it was much easier to set up than I had anticipated, so eager to try a few other ones.
And yes, I can feel a world of difference using such an old model as I got used to the proprietary flagship models. Feels almost retro, even though it's only been 2 years.
Ollama has these in their library:
gemma4:e4b(gemma4:12bcould work but e4b works better on my 16GB macbook than any 12b ones)qwen3.5:9b
Reporting back, using qwen3.5-9b locally, this has been a pretty useful experience. I've used it for all these little questions you want to ask an LLM rather than googling them, but don't want to have a record of them online. E.g., medical questions, etc.
There is a sense of relief that starting last week, ChatGPT does not know as much as it used to about me. Now I need to convince/train my wife, who routinely uses it for personal advice. And I need to get to a larger model/better hardware so I can also start switching away from the proprietary models for my work-related stuff. Also, deleting old records, but that's just to make me feel good. They probably already recorded it in several locations.
Small steps.
That's awesome progress!
If you want to share the interface with your family, you could opt to get a Mac Studio rather than a macbook, and serve the local model through a web interface with either llama.cpp serve or vLLM Playground over LAN/tailscale if you wanna use it outside?
Hardware is crazy expensive right now though.
Thanks.
Will keep that in mind. But budget will be the deciding factor.
The good news is that Apple uniformly prices their stuff
The bad news is that both macbook and mac studio are expensive right now haha
Neat. Will report back after trying.
If you had asked me last week, I'd have said only proprietary models...
But this weekend, ended up running Ollama with qwen2.5:14b on my entry level Macbook Pro, just to get familiar with what it is to run things locally and privately. Budget for new hardware will free up in coming months, so I hope to choose the right specs based on this early experience, so that I can run something more powerful.
So probably can't give you very useful feedback or input for now.