pull down to refresh
I was leaning that way too but if the token costs keeps coming down such as they have I'm not so sure. Gork 4.6 is literally the cheapest, bigger issue is I can't just easily plug a Cursor API key in everywhere I would an OAI compatible key.
Also keep flip flopping on buying hardware, first the local models were too shit to justify it, now that they're good the hardware is still expensive but the closed models are getting cheap.
Would need a $7500 box to run GLM 5.3-Flash correctly, and can't reasonably expect to amortize that over years because who knows what the next gen will need.
Bigger argument for local is not leaking data, if hardware gets reasonable again that'll favor open-weights again.
Bigger argument for local is not leaking data, if hardware gets reasonable again that'll favor open-weights again.
Yep. You're right about cost. Hard to justify just on basis of token pricing.
I got my rig (AMD AI 390) mid-2025 when prices were reasonable, so $1900 for 96GB RAM, don't think its worth it at double or triple the price.
But like you said, the big benefit is privacy. I've moved my accounting setup to be all "beancount" based (python text based accounting program).
So I can drop CSV files from my bank into my local LLM and have it transpose all the entries into beancount format.
Absolutely you can do it programmatically - and that is in fact what my LLM does for me: (a) when new CSV format is encountered it writes a beancount importer to handle that specific file format, and (b) intelligently understands the descriptions to categorize the entry and update that rule in the categorize.py script if it doesn't exist (so PHO/GRILL it understands is a restaurant, etc)
Those are generally one-time task. However the real use is being able to chat with the LLM about hypotheticals....ways to become more tax-efficient, etc.
What does SOTA mean?
I use Cursor, mostly connected to GPT5.6, to manage a lot of web development projects. Mostly because my client is OK with data going into GPT.
And now Open AI has announced they are pulling their service from Cursor I have to consider my next steps.
They probably have no idea about the difference between GPT and Grok, and it's just front end code and automatation tasks, there's no sensitive data or proprietary code, but I am considering my options.
the most important tool for an architect or engineer is CAD or visio or both
SOTA models will always have a place, but I think smaller models are going to wind up soaking up the majority of tokens in future.
I already use qwen-3.8 for most of my work tasks (I'm not a developer, so my LLM usage is primarily scripting + ansible + sysadmin stuff).
As a development test of my opencode install I asked cloud hosted Kimi-K3 to write a detailed plan for developing a web frontend to an existing piece of software my company uses. I went back and forth with it thru a number of passes to make sure it understood all the requirements (auth, permissions, how to access the sql backend, what graphs to use, how to containerize the app, etc) Then I handed off that detailed plan to local qwen-3.8 and let it build it. Pretty much worked first try.
My point the gap between SOTA and local at this point is that SOTA can do the big picture thinking that is too much for a local LLM, however with a detailed enough plan, local LLMs are more than capable of building non-trivial apps.
I think the days of just monolithic-ally using SOTAs for every task is eventually going to wind down. The classic "architect vs worker" paradigm.