pull down to refresh
by that I mean letting the ai unilaterally decide the methodological framework which will be used to establish a result
also, I don't trust the ai to interpret the degree to which a result does or doesn't support a hypothesis
letting the ai unilaterally decide the methodological framework which will be used to establish a result
100% agree. I never let an LLM decide anything. If it made a decision then it means my instruction was incomplete.
I don't trust the ai to interpret the degree to which a result does or doesn't support a hypothesis
After Opus 4.6 release, and experimenting a bit, I added instructions to my intel gathering processes on 3rd party code to summarize the findings to me. I don't use this as definitive but it cleanly warns me in case there is a finding that is important to look at. It's not so much that I will only read that finding, but that I want to make sure that I won't miss it. This works very well with GPT 5.x and GLM 5.2 too.
For example, when I woke up this morning there were 56 findings waiting for my inspection regarding the new signal release, 2 of which were highlighted in summary. Both were in webrtc which is written mostly by Google, and were important to validate, and understand context on. Without the nudge I could have read over at least one of them in the long list of things to do, especially with my first coffee.
This makes me think that you can have it flag things up to you, but don't trust an "all good" verdict (so don't ask for it - it took me quite some effort earlier this year to make Claude to stop guessing the severity of an issue it spotted, because every bs item it finds was HIGH if it was the most important thing it found.)
Yeah, AI seems pretty bad at calibrating the severity of a problem.
I've tried using it for feedback on my work. I find that the feedback it gives you actually depends a lot on whether it thinks this is your work or someone else's. If it thinks it's reading your work, then all the issues are minor and fixable. If it thinks it's someone else's work and it's helping you write a referee report, then the same issues are paper killers.
I've even had it tell me to upsell a particular claim more in one session, and then in another session tell me that the claim is undeserved and should be taken out.
Well... which is it?
If it thinks it's reading your work, then all the issues are minor and fixable. If it thinks it's someone else's work and it's helping you write a referee report, then the same issues are paper killers.
Oh yes. Everything I do is through a forge so it always reasons about everything in third person. "Maestro (the handle I use) requested that...." Or more fun: "The user golden-apples (another bot account) said: ...". Benefit of most of the training data being done from personal access tokens and the bot expects that its own user is the operator's account. I like this form of automatic confusion - it helps.
Well... which is it?
Haven't seen that in a while. I don't like that behavior but I never ask for styling advice. Only facts.
There was a time where I would give Claude 2 separate accounts and critique itself but I found that it works better to let another model do the review. Different training, different results. Currently Claude does mostly review for me. GLM does most of the work. I cancelled my GPT subscription so now that I pay top dollar for it over API, I rarely use it. Not worth the cost.
There wouldn't be much point to us if we could trust AI to do those things. Of course, there might not be much point to us either way.
For game development, I'd be pretty comfortable using AI for all of the above. That's because I like the idea of developing games (I'm working on one now) but I don't know how to do just about any of those things well.
I'm curious what you mean by "letting AI do the core analysis".