I'm still super happy with Jan-v3-4B-base-instruct for structured output tasks (w/ pydantic), works better for me in that particular setup than gemma or qwen at that size (or even one bigger).
I don't like qwen3.5-3.8 - it lacks precision for me, even the large models. I instead run gemma 4 on mlx for private stuff.
(remote but on my hardware) I currently also have gemma-4-E4B-it running on CPU for (very low priority) summarization on a Xeon server that would otherwise mostly be idle. Since I am not happy with the pricing on new hardware right now, or the insane 3x I'm seeing on rentals, I'm not changing anything I don't have to and just do dumb things like dedicate 32GB of unused CPU-only ram for llama.cpp. Actually works.
I have two subscriptions: one for Maple which honestly was a waste of sats due to not being able to even run a single code mapping task through it on the monthly quota, and it is especially shitty because I had to pay for a year or be a KYC'd fiat slave to find that out. And the cheap one for Claude, which I don't like much anymore. Currently I prefer Kimi K3 through ppq for large, complex tasks (of which I run 30-100 a day, depending on what happens out there)
Forgot: my locally most ran model right now is probably MinerU2.5-Pro-2605-1.2B-GGUF in my newly updated mineru pdf->markdown->(embed and/or epub) pipeline. I have soooo much stuff to read haha
Jan-v3-4B-base-instructfor structured output tasks (w/ pydantic), works better for me in that particular setup than gemma or qwen at that size (or even one bigger).gemma-4-E4B-itrunning on CPU for (very low priority) summarization on a Xeon server that would otherwise mostly be idle. Since I am not happy with the pricing on new hardware right now, or the insane 3x I'm seeing on rentals, I'm not changing anything I don't have to and just do dumb things like dedicate 32GB of unused CPU-only ram for llama.cpp. Actually works.I have two subscriptions: one for Maple which honestly was a waste of sats due to not being able to even run a single code mapping task through it on the monthly quota, and it is especially shitty because I had to pay for a year or be a KYC'd fiat slave to find that out. And the cheap one for Claude, which I don't like much anymore. Currently I prefer Kimi K3 through ppq for large, complex tasks (of which I run 30-100 a day, depending on what happens out there)