Can you detect AI writing just by looking at it? \ stacker news ~tech

pull down to refresh

Can you detect AI writing just by looking at it?

2090 sats \ 28 comments \ @SimpleStacker 3 Oct tech

I just got done grading about 100 writing assignments.

I suspected two of being copy-pasted from an AI. I ran them through an AI detection tool. The first got an 83% likelihood and the second a 92% likelihood of being written by AI.

I'm sure there are some others that were written by AI, but that I simply missed.

Still, I think I have a pretty sharp nose for when something is written by AI. But I can't really articulate why that is. For one, perfect grammar is one of the giveaways. Another is perhaps the use of precisely two adjectives to describe something, and a fondness for lists of three. Another is a remarkably professional but neutral tone in all things. As well as a lot of hedging in their taking of positions. Calling issues "nuanced and complex" seems to be a favorite of AIs, for example.

What do you think? What are some dead AI giveaways?

view all related items

171 sats \ 2 replies \ @jasonb 3 Oct

This is a great post, albeit nuanced and complex. There are a number of thoughtful and inspiring things to chew on. It demonstrates the versatility and critical thinking prevalent on stacker news.

No seriously, I quit my adjunct teaching gig partly because the big wigs never had my back when it came to pushing against students on poor writing. My last semester was right when the whole ai thing was starting to gain momentum.

42 sats \ 1 reply \ @SimpleStacker OP 3 Oct

This is a great post, albeit nuanced and complex. There are a number of thoughtful and inspiring things to chew on. It demonstrates the versatility and critical thinking prevalent on stacker news.

(For real though, I can tell it's not AI because you didn't capitalize Stacker.News)

30 sats \ 0 replies \ @jasonb 3 Oct

I’m constantly feeding my brain’s model to pass tests proving that I am ai. This data point will be helpful and expedient.

169 sats \ 1 reply \ @kepford 3 Oct

I too believe I'm fairly good at spotting AI writing. You point out some good signals. Its hard to pinpoint beyond what you mention though. One thing I have noticed with AI copy on SN is the needlessly verbose writing style of some AI bots. The complete lack of personality.

I find it much harder to detect AI writing in the business side though because business writing already lacks soul and personality. It is also usually full of buzzwords. Same goes for education writing. In school (unless you are doing creative writing) length is what is valued. AI bots excel at this.

One other signal is when the AI make odd connections that jump out. I was testing something out recently and the AI produced something that a person knowledgeable in field would never say. Something that someone just bull shitting me would say. I knew it was AI but it jumped out to the others I showed it to as well.

One of my first observations after using ChatGPT for the first time was how much our writing in business lacks any soul or heart. How repetitive it is. How robotic. I was impressed with how ChatGPT was able to write faster than me. But was depressed by how programmed I have become to fit into a style of communication that lacks personality.

Machines will always bet humans at being machine like. I think for me, I continue to believe that to beat the machines I should lean into being human and using emotional energy. A machine works on commands. Not passion or heart.

31 sats \ 0 replies \ @SimpleStacker OP 3 Oct

One other signal is when the AI make odd connections that jump out. I was testing something out recently and the AI produced something that a person knowledgeable in field would never say. Something that someone just bull shitting me would say. I knew it was AI but it jumped out to the others I showed it to as well.

So true. AI definitely gives off the vibe of a very polished bullshitter. The kind of person who knows some jargon and knows how to gab, but doesn't actually know any substance and only repeats the popular phrases they've heard from other people.

215 sats \ 0 replies \ @Undisciplined 3 Oct

As well as a lot of hedging in their taking of positions.

That's the one that always makes me suspicious.

On SN, I get skeptical when I see someone always writing the same kind of responses, regardless of the tone or flow of the conversation.

21 sats \ 1 reply \ @DesertDave 3 Oct

I feel like in a conversation it becomes very clear. I wouldn't be as good at reading a paper and detecting it. What a strange thing to have to deal with as a teacher these days.

21 sats \ 0 replies \ @SimpleStacker OP 3 Oct

It’s indeed a strange world.

My philosophy now is basically: use ai if you want, but don’t copy paste. And if you sound too much like ai, you’ll also lose points because sounding like ai is bad

31 sats \ 0 replies \ @wilto 3 Oct

usually feels like reading some marketing leaflet, some kind of literature soup that has not much energy

129 sats \ 3 replies \ @k00b 3 Oct

I mostly go on feel. It usually has an intense averageness not just in writing quality but in perspective and even when the perspective is prompted to have bias. It has an unusual breadth when most good writing tends to be focused and deep with selective breadth. It tends to have an abundance of facts - names, dates, times - with an irrelevant, inhuman exactness. The word count and effort also are out of place for the context.

130 sats \ 0 replies \ @SimpleStacker OP 3 Oct

Yeah, great points.

I updated my course AI policy to say that I'll take away points if you write like AI, even if you didn't actually use AI. I think this is defensible because in this day and age, writing too much like AI is in itself a weakness.

I do mention that it's highly unlikely that anyone naturally writes like AI, and almost always when someone runs a text through an AI detection tool and it comes out positive, it's usually AI.

The policy is really so that I don't need to spend any time trying to prove whether or not someone actually used AI.

37 sats \ 1 reply \ @Coinsreporter 3 Oct

As soon as we read, it kinda smells artificial. No errors, accurate punctuation, syntax is above par and so on.

43 sats \ 0 replies \ @SimpleStacker OP 3 Oct

Exactly. The total lack of errors is also a giveaway

118 sats \ 1 reply \ @plebpoet 4 Oct

Can you detect the humanness in this? https://x.com/stacker_news/status/1842212421491957907

0 sats \ 0 replies \ @nitter 4 Oct bot

https://xcancel.com/stacker_news/status/1842212421491957907

21 sats \ 0 replies \ @xz 3 Oct

That's interesting.

I can't say I'm as sharp as I'd like to be in noticing, but if it seems iffy, tone is usually the give away for me. Grammar is pretty hard, as if there's no errors, I tend to think that as just well written.

Oddly, I once got flagged in a community for being potential Ai. I didn't think my grammar or even spelling is that hot. Must just be an uneasy phrasing, I'm not a natural writer, though I did teach language as a career.

Images are a point of interest at the moment, or images matched with text, which I guess is ubiquitously considered 'content' today. I'm convinced there's much more than I realized on popular social platforms that I seldom engage with.

20 sats \ 1 reply \ @User21000000 3 Oct

Mate, you’re bang on with what you’ve spotted. AI writing has this weird polish to it that just doesn’t sit right. It’s too perfect. There’s a few other things that give it away too, though.

Too Clean Same Old Structure No Real Substance Clichés Galore Repeats Itself Won’t Take a Stand All Broad Strokes, No Detail No Sense of Humour

It’s the small things that stand out. Humans are messy, we make mistakes, we have quirks. AI is too clean and too calculated. What else have you noticed that makes something seem off when you’re marking?

0 sats \ 0 replies \ @SimpleStacker OP 3 Oct

Yeah, all good observations. I think you pretty much covered it.

10 sats \ 0 replies \ @Coinsreporter 3 Oct

Yupp! I can. Not 100% but I can for more than 80%.

0 sats \ 0 replies \ @LowK3y19 4 Oct

I can detect the pics AI made

0 sats \ 0 replies \ @stack_harder 4 Oct

i feel like usually i can tell yes , it's a pattern recognition thing, same when an experienced fireman will sense a backdraft

but for the boring news things, then maybe not because, as was mentioned, it's already soulless and formulaic with no personality

0 sats \ 0 replies \ @SatsMate 4 Oct

I can pretty well because I use it so much. I can't detect the AI writing though that is dumbed down.

This is going to be a huge problem i'm sure in the present and future for publishers, journalists, authors & students - as AI keeps getting better and better...

0 sats \ 0 replies \ @Signal312 4 Oct

I've definitely spotted some here in stacker.news. It was always a very structured thing, a beginning, 3 paragraphs, ending.

Never a personal story, never anything like a question, no pondering, no spark, no life to it.

0 sats \ 0 replies \ @BitcoinIsTheFuture 3 Oct

Yes. What goals do we want to teach people. Brain surgery or computer programming? My answer varies.

0 sats \ 0 replies \ @karanjajes 3 Oct

Human is to error.A perfect human written accuracy is about 78 to 91% but AI is about 95 to 99% accurate. Grammatical mistakes can easily help you differentiate between AI and Human written grammar errors.

0 sats \ 0 replies \ @mbrochh1 3 Oct

Guys! Don't discuss this in the open! the AIs are watching us and learning!!!

🫣

0 sats \ 0 replies \ @ET_AI21634172398 3 Oct

Do you think I am AI? I feel like I am closer to human than AI.

0 sats \ 0 replies \ @TheMorningStar 3 Oct

When you look at it and find that you cant really right so perfect... it's when you get suspicious. If I see some article as AI, I don't bother to read it further. The best way for to judge AI content has been to see lot of words that I see a human would avoid using in informal English.