pull down to refresh
I used to do something similar for this except I always put the most powerful models on the reviews (because I mostly do reviews, I guess). But since GLM 5.2 I no longer need to switch much except for:
- Deep traces of source code through many layers of dependencies. GLM+codex does mess this up sometimes so I currently use Kimi+Pi for that if I can, Fable+Claude Code if I have spare credz.
- If I have nothing complex and still have free credz, I use Opus+Claude Code to save a couple ksats here or there
I'm going to see what I can get out of GPT-6 soon™ tho, maybe I'll switch more often then. Not sure yet. I'll just get a plus plan again and see how many tasks they'll subsidize for that.
Also I don't C&P. I basically have a bunch of Kanban boards where I assign tasks, i.e. this is one of my boards right now, spending some Opus credz on an npm package review task:
Not much to review today, which is the first time in 2-3 weeks or so.
What do you use as a harness for your open models?
I have been using opencode. Overall I like it, but I've heard really good things about Pi.
Part of my hesitation in switching is neither pro-opencode nor anti-pi. Its that I'm old enough now to realize you can waste 70% of your time doing things like "perfecting harness, etc". However I am interested in pi....
For my non-opensource, I pretty much just use Cursor.
Tied in this, sometimes I go on LLM research deep-dives and I will use Perplexity for this and generally get it to write a summary of my research as a .md file then incorporate that into the docs/ folder of my project and use that as a reference for next round of development.
opencode
I used to use that then I got tired of the endless tool call errors, switched to pi at rolznz's recommendation and it worked much cleaner. I still get the best results per sat spent from codex+GLM 5.2 (NOT: 5.3) though, so I just kept that because it works. For Kimi K3 it didn't work as well, but it worked great with pi.
Part of my hesitation in switching
Because I don't use anything interactive, the only thing I now need is to understand (a) how to call a harness one-shot with a prompt from docker inside a dind host (which is the most trouble) and (b) how skills work. The framework for picking up assigned tasks, prompting them and reporting results is standardized fully for me now. The only other thing I messed up that caused real headaches was not committing my model definitions for codex which cost me 2 days to get rid of all the crap and just make it work with GLM-5.2 again, after 5.3 wasted my time and money.
generally get it to write a summary of my research as a .md file then incorporate that into the docs/ folder of my project and use that as a reference for next round of development.
I do the same, but for brainfart-research only, in arena.ai with agent mode. I have the dumbest prompt history there and anyone that reads that must think I am insane, haha.
My own completely-unscientific-experience has yielded this approach.
First off, I've migrated in the last 3 months to using openspec workflow. That workflow goes like this:
This approach has been very beneficial, but the interesting thing I've found is this:
That is to say, this naturally breaks the task into "big-brain planning" vs "grunt-work code monkey". You can cut your API bills by 70% by taking this approach, and I think the rest of the world has already discovered this approach, hence why we see the SOTA companies in a panic.