Last week I posted an article about Github Workspace, and gave my broad (and long) thoughts about LLM-assisted software development in general. To restate briefly, a theoretically simple win for LLMs would be to arm them with common security vulnerabilities, and let them loose on code bases, opening issues and submiting patches. I am claiming this "easy win" is not easy, and probably less of a win than we might expect.

While writing my article, I had an incident written about by Daniel Stenberg, who is the maintainer of curl. In his blog post, titled "The I in LLM stands for Intelligence", he shares a number of trends coming to form what I am terming "AI Spam".

He details two examples, one where it appears a human operating an LLM submits an issue, and another where the LLM appears to be operating autonomously, appearing to impersonate a person. They are both CVE security vulnerability claims, and need to be taken seriously if valid. However, the issues are totally bogus, and end up wasting the time of the maintainers. Verbose hallucinations take the most time to filter out with human eyes.

LLMs are finding use among developers who don't natively speak English, so it's important not to dismiss issues just because they have the charactaristic language we've all come to expect. One of the common responses to this issue is to have "LLM generated" text be labeled, or to set your own LLM to detect and tag non-human content. I don't think this is a great solution as there are legitimate reasons to use LLMs to generate some or all of the submitted text of a code issue.

Whether it's for the bug bounties, or for the social clout of getting accepted contributions to open source, the incentives to continue trying this spam will continue, and I don't see a great mitigation strategy for the code maintainers. Thus, I like to think of it as "spam" as in unwanted email. Long term, I would like to see some cost incurred to post an issue (sats) which could be returned upon a valid issue being confirmed and/or improved identity / social graphs used (nostr maybe?).