pull down to refresh
I kinda do that by getting one LLM to generate such structured task templates and then feeding them into Claude. And then copy-paste Claude's output back into the first LLM. By separating analysis from coding, it helps me avoid polluting Claude's context window and catch reasoning mistakes from Claude.
But I could do a better job with my templates, based on what you shared here.
I have that pattern for larger source code diff analysis. Like right this second I'm running a source code mapping on yesterday's Phoenix release, that went through a first pass to basically identify the areas needed for review, and that I then read and made sure was good (nowadays it doesn't need too much editing but that is probably a function of both evolving models and me evolving my templates) and if needed I could have split some things up, for the context window guard you mention (not needed on this diff though.)
When it's done I get a massive report. The most work is going through that. I have specifically spec'd the desired output format to not become overly brain damaged from reading that.
I never use bots for coding anything serious anymore though. Just tools.
PS: For small jobs I often just delete the decomposition section. It's not always needed.
I never use bots for coding anything serious anymore though
What do you mean? For coding new features, you use your pre-LLM skills? No LLM assistance? Or you're talking specifically about bots with less supervision?
Everything I have coded for production in the past month has been hand coded. It's easier to maintain, makes more sense for reviewers, and I get the bugs out by running review rounds with bots, who get to write comments, not code. Then I fix the code if I agree with the review, also manually.
Examples that I haven't hand coded: template automation, my personal mostro monitor (that still isn't finished), some scripts to find out what the fuck is going on with people trying to brute force through my mTLS pki... internal stuff.
A dying breed.
I wonder how many of the new generation of coders will even be able to do that.
Probably depends a lot on how current CS classes are being taught. Or how people self-teach themselves coding with sufficient discipline not to use bots while doing so.
I'm 100x slower. This is what is prohibitive. And I still cannot guarantee that there aren't any bugs. In fact, last week I caused a bug in SN despite my own review, tests and multiple rounds of bot reviews. And it was ugly (even though low impact), I should have caught it from the beginning, and I still don't understand why the bots didn't catch it. Especially since I made a note about the issue but got the scope wrong.
Sometimes I find my bots all queued up and as I hate deserializing worker queues because it complicates my review process, I just throw something at
arena.ai/agent. But even for that I use structured task templates, i.e.### Objective <!-- One sentence: what should be true when this is done? --> ### Context <!-- Links/pointers to related issues, files, or docs. The agent needs these to avoid hallucinating context. --> - url 1 - url 2 - attached file ### Constraints <!-- Scope the task by describing/constraining the steps to perform --> - step 1 - step 2 ### Acceptance Criteria <!-- What can the agent measure output against to make it match expectations --> - Criterion 1 - Criterion 2 ### Task decomposition <!-- Specify how the agent should use subtasks to get to the desired level of detail --> - Use the main task to check the overall implementation and discover all items that must be checked, then - Use subtasks to analyze each discovered item in a detailed manner