pull down to refresh

Not to put words in Rearden's mouth, but it seems like he has some belief that AI-written projects will shortly be more secure than projects with predominantly human written code.

Currently, my only way to evaluate any software I use is vibe-review or some sort of web-of-trust evaluation (do people I trust also use this software). If I think that vibe-review contributes to the security, don't I have to extend the same belief to vibe-code?

I don't buy such a viewpoint, and I stubbornly insist on relying on my web of trust + vibe review approach to software, but I'm trying to figure out how I should evaluate all the vibecoded stuff that comes across my radar.

I'd vibe my own

Would you use a vibe-coded node even if you were the vibe-coder? In my ignorance, I assume that vibing something as complicated as a node will result in lots of weirdly-shaped corners around which I cannot necessarily see -> edge cases where my node will not implement the exact same block validation rules as other nodes.

1133 sats \ 0 replies \ @optimism 20 Oct

It really is a thing about perspective and ownership, I think.

belief

Why would I trust someone else's beliefs, especially if they go contra to my own experience? Yes, bots can do infinitely better code than a year ago, or half a year ago. But 99% of it is still bad, for me. Perhaps even worse because it is harder to understand.

vibe-review

How many times have you decided to not run a particular software, a particular update or have you patched vulns the bot found or at the very least decided to stop using a particular function of a software because it is a shitshow you don't wanna touch with a stick? Because that's the metric of how effective your review is. If you have low standards, you'll probably abandon or patch half of what you use. If you have high standards... welcome to my world, I guess. I run exactly 1/5 of my high requirement apks unpatched at the moment, and that is only because they patched a bug and released it quicker than I could patch it up myself.

trust

I also operate on trust. Devs I trust to take things seriously get a vuln report or a patch. I've given up on everyone I don't trust - which is nearly everyone - as most devs have gone completely retarded now. They don't get a report from me anymore and if tomorrow they die then the only difference is that there is a very insignificant drop in demand for GPU ticks. I most often just stop using their software or library, or otherwise I patch it[1].

I do use LLMs to fight the declining quality of their output against ever-rising gates - the bar goes up on the daily. I have to because there aren't enough hours in the day for me to read all the slop that gets released. But I created a process based on a couple of decades reviewing other people's code and I'm continuously monitoring and optimizing it, and the bots aren't doing the judgement. Reviewing traces of what the bot did (I have a process for that too now) and tuning instructions, templates, sporadic skill files. Developing a new process takes me months, but when it's there and I maintain it, it goes well. Every process is hybrid, nothing runs without my request (it's literally me assigning a task by dragging it on my screen though.)

Would you use a vibe-coded node even if you were the vibe-coder?

Yoloboi vibecoded? Nah. I don't do that really. This morning I had Opus code me some stuff for a tool I use. I then went into my editor and implemented it for real. Threw away the bot code. Now I own it. If I merge that Claude code I won't know what the fuck is going on in my own systems. Might as well run closed source software then. If it were an inconsequential tool I wouldn't care, because then I can just replace the entire thing for peanuts[2], but this is for one of the tools that I use for tracking defects across an n-dimensional reality so it is important that I can rely on this even when there is no bot to help me figure out my own system - the same goes for my Bitcoin node, but that is something you can ultimately turn off and be okay (as long as you don't need it to power your LN node.)

But then, I do own the prompt of any self-vibecoded software. So even if I wouldn't do any actual reimplementation or even review, my slop > someone else's slop. And there is nothing anyone or their bot can do about that. My slop is sovereign and it stays with me, it empowers me, until it doesn't then I change it. Their slop is the enemy, it transfers my power to them, I have no control over it until I replace it with something of my own.

I'd suggest that from Rearden's perspective, this is absolutely grand, I am very happy for him. For me, the exact same software is meaningless and I'll stay away from it.

edge cases where my node will not implement the exact same block validation rules as other nodes.

Or just bugs. Because they're rather hard to spot, simple mistakes, scope misinterpretations or tiny hallucinations, perhaps a context window compression at exactly the wrong time removing something important from the conversation... are causing bugs that may cause real issues when still no one has read the code 2 years in. Same like with humans; some pretty insane linux kernel bugs are being found lately. There is no difference between bot and human output other than that people actually believe that bots are infallible as if it were a calculator, whereas for humans it is widely understood that they make mistakes.

  1. For example, I've written a 3-liner patch a few weeks ago that neutralized not only an upstream vuln itself, but 3 other subsequent vulns that happened upstream in subsequent patch releases afterwards. All that was Fable 5.1 vibe coded by them (because FOSS devs get free LLM credz.) So I don't really believe that bots fix anything. You still need a great engineer to get great code. The only difference that can be is how much of it is typed out. ↩

  2. Anything vibed is a commodity, because LLM generated code is a commodity, just like the sloppy text, pictures and videos - literally anyone can produce it, especially if it is sloppyvibed (i.e. (near-)zero effort) - which most stuff out there is. ↩

reply