Substack is rolling out an in-app tool to use Pangram to check if a post or comment is AI generated.
Going forward, you’ll be able to scan text to see how much of it was likely written by hand or with AI assistance.Going forward, you’ll be able to scan text to see how much of it was likely written by hand or with AI assistance.
I suspect every platform that allows user-generated content is struggling with this right now. The slop is everywhere. Just checking isn't going to cut it, so it looks like this is just the beginning of what Substack has planned:
We’re also introducing tools for creators, including the ability to add a “How I make this” statement where you can explain your process and set expectations for readers. You can run Pangram on your drafts prior to publication, as well as report and remove any scans on your own work that you believe are mistaken.
Here is their lost of potential new tools to combat slop (not yet introduced):
Depending on feedback and interest, we are considering tools that will:
- Let you set preferences around AI content for your own community in your Reply Rules
- Give creators more ways to express the individual value they offer, whether through human writing or thoughtful AI use
- Give readers the option to set preferences about what gets recommended to them
- Continue to improve our systems that fight spam, bots, and scams of all kinds as AI accelerates these problems
I wonder for how long the AI detection tools will actually work. If they are just pattern matchers, it seems like it will be fairly trivial for other AI to outsmart them.
At the end of the day, it is something more than a tone against which people who don't like slop are reacting:
The core problem is not people using AI, or the quality of its output. Not everything made with AI is slop, and not all slop is made with AI. The problem is when there is a mismatch between a reader’s expectation and reality, especially when they unwittingly invest their attention in something with no human thought on the other end. That’s Claudefishing.
Best includes a Freddie deBoer quote:
I access human-made art because I know there’s a human behind it and that’s what I’m looking for, other humans, showing me in art what they hide in their selves. Fooling me in that process is just a con.
I don't know if I quite agree that it's just a con. It doesn't feel great to be duped: if we think we are reading a post by a favored personality and it turns out it was a not that produced it, we feel different than if we discover that it was ghostwritten by another human.
But it is true that it is difficult to put words to the actual problem here. Perhaps I go back to the void, feeling that there isn't any soul at the core of an AI and that this somehow explains things. But I'm not sure I have any grounds on which to believe any of us have a soul, so that's hardly a good argument.
Strangely enough it feels like the strongest support I have for disliking slop is a labor theory of value for writing: it's just too easy for the not to wrote for it to matter. This isn't good.
I'll try, with a relatively simple point:
Do you know why LLM output (no matter the form) often feels incredibly
mid? Because the model is literally the mid. Yes, you can influence it with RL and throw in somerandom(), but the base is always the exact middle point of the bell curve. Over billions of inputs. Books, articles, people on reddit, captures of chatbot interactions from the past, slop produced by earlier bots, and so on.For most humans, our median is different from person to person, because we all have slightly different chemical balances which compound heavily over time. This is why if you ask 2 people about a crisis situation they will likely not tell you the same thing. We literally experience things differently because of our chemical markup and for adults, we've been compounding the delta for years, and actively do it with each new experience.
LLMs don't have intrinsic learning capability right now - at best some external "memory" cache and a context window. That doesn't change the LLM though, it changes the question you ask the LLM. Weights are static.
And thus, most people are experienced enough to tell when something is up. If you get confronted with different slop from different sources, you can probably tell which bot wrote it, when you're not even deep into this experience yet. You can tell because you recognize the pattern. And you know: this nym isn't interesting. They are
mid. Because they're just a dumb af bot that produces the statistically most appealing outcome; not anything based on experience, nothing surprising.I agree with this. But it gets messy because I am still offput by a human with a surprising or possibly interesting idea using a wad of slop to present it. I don't think that I need to marvel at every turn of phrase (although there are some writers who almost achieve this), yet most slop leaves me feeling frustrated.
There's no doubt that
midis boring, but AI is still somewhat boring if prompted into outrageous statements. (I suppose you might say that even their outrageous statements aremidand I would probably agree). I am willing to go out on the limb to say that should the geniuses develop AI that surprises us, I still think it may be slop.It took me more time than I would like to admit to write this response. Many minutes were spent with my fingers hovering half way through a sentence because I had to think about what I was writing. It could almost be described as grinding it out -- writing is work. There's something about slop that feels like fake work. Like a badly put-together building where human writing can be a well-crafted structure that stands for a thousand years.
I keep using the phrase "labor theory of value for writing" even though I know it's wrong. But there is something to this idea that you can't get to a thoughtful conclusion without actually spending time thinking -- and slop is too fast.
Right! I'd suggest that to be caused by that everything still goes through some form of dampening by
median() over (everything_ever_written)- or at least within the "expert" in a MoE model. That takes the edge off.That's my world, also because I guess I suck as a writer. Every post I write goes like that, including this one.
I fully agree with that, which is why I have been emphasizing LLM use for half-fabricates or intermediate work product, never end product, ever since the output became better (about a year and a bit ago.) This takes away the "consciousness" dream/nightmare (or what I believe to be a
fable, pun intended) and replaces it with the assigned role oftool. After all, if I wanted to read bot output, I don't need anyone to middleman it, I'd just ask the bot directly - even if you have more skills in prompting then me: the interesting part then becomes how you build your prompts, not the resulting output from the bot.This goes for articles or books, but also for code, for images, for business process documentation... for everything basically. The tool is there, it's largely a commodity and anyone can use it. So you don't need to publish it and expect to get something for nothing.
Personal example: Since this morning I did 30+ analyses on software. 2 android apps, their dependencies and also a bunch of libraries SN (or mostly 3p components of SN) depends on, and each of them got sent through an LLM for pre-analysis. I read (almost) all the slop coming out. It's very detailed because I let the LLMs do what I would do: go over every changed line of code, trace it end-to-end and form an opinion on correctness and risks, check for oddities like opaque blobs or postinstall scripts, network and filesystem calls, and so on. I then read what the LLM says and flag up what needs to be further analyzed, which I do manually. I then document my findings next to that of the LLM (also when it was wrong - happens sometimes, but less often now that I've spent almost a year tuning this process and while newer model versions got finetuned a lot more) and I can make decisions.
No one ever sees any of this slop. I don't need to read it a second time either, unless maybe I made a mistake somewhere and I need to go over my steps. I guess my usage of LLMs in this is 100% as a research tool; not as an assistant, "agent", replacement or extension of myself, or fake simulated entity that doesn't exist, because it is a computer program, not real.
I personally like it that way. Also I hate reading slop and I do it non-stop. Which is why, when I'm away from the bots for a while, I really hate slop.
I do anticipate a potential revival of human-oriented, embodied spaces. The wide availability of AI is only going to increase exclusivity. The people with power and influence will create spaces where they can trust that they are interacting with other human beings, but because the bar of verification will be high, it will be difficult for the peasants to break into those spaces.
We'll make our own and I'll throw you an invite <3
I wouldn't want to join any club that would have me as a member
Groucho Marx~~
Why? Do you think that pre-qualification/elitism is a good selection mechanism and not arbitrary?
I wonder though if the things that make a space exclusive (aside from money) are actually more generally available to everyone. Reputation can be built in a group chat just as easily as an exclusive pay gated member community.
True, but I think if communities become more gated, it'll naturally gravitate towards more separation between the elites and the non-elites
I mean, I kinda see it on SN already, where new users don't often get much benefit of the doubt.
And recently, arXiv changed their policies from "anyone with a .edu email address can post a paper", to "you must be endorsed by an active member before you can post"
This is true. I still try very hard to check whether a user who sounds like a bot actually is. Unfortunately, most of the time they are.
A world of invite only is looking better and better. But it is also the saddest version of the internet ever.
After rereading this sentence a few times I figured out what it was that bothered me about it, and that is that I don't consider whatever it is that bots do writing, for that would intimate the process of putting thoughts, of which a bot has none, into words.
I enjoy good writing. I like reading the work of writers.
I do not enjoy the whatever trimmings - ground, emulsified, packaged and then steamed to form snappy bite-sized capsules - that bots produce.
sadly, that sentence was mangled by my fat fingers and autocorrect. it was meant to be:
I think that the bots intentionally sabotaged me.
I made out the meaning!
I wasn't trying to point out your mistake, I was just saying that I think what bots do isn't in fact writing.
So no need for labour theory, we just need stricter definitions.
Lol
It's funny: I also enjoy good writing, I practice writing every day -- but I don't think I agree that we can define our way out of this.
I'd love to say that what the bots do is excluded from writing by definition. But the problem is that I keep reading it and then realizing a little way in that of is probably slop. Or sometimes only after someone else points it out. I'm ashamed to admit that my slopometer is not as tuned as some.
I can dismiss it as not writing, but this move sounds a little like the reason my mother refused to let me read cool SciFi and fantasy books or comics -- they weren't literature.
I want a stronger defense against the slop for when it gets better -- because it will get better.
Currently, my problem with slop is that it is thoughtless, like a person who doesn't quite care what they say or even if it makes sense. It might just as well make an argument for living without any technology or tools at all (Thor Heyerdal style) as embracing all the post humanist longevity stuff of Bryan Johnson. It just doesn't care what it says.
This lack of loyalty to any particular thought is very unnerving. A human who behaved like that would quickly be friendless.
The AI detectors don’t work and will never work. AI-isms have crept into everyday speech, and statistically improbable phrases no longer have the same distribution in training sets vs everyday speech for detection. More advanced models are indistinguishable.
this is what I fear
I respect the effort here, but I wonder if this won't feel a bit pedantic?
Also,
I lol'd at this. Introduce AI detection and then glorify AI maxxing. Brilliant.
You are spot on here.
People deserve to know who's really writing..