pull down to refresh

Why AI is so bad at sports information (?)
808 sats \ 11 comments \ @SimpleStacker 9 Sep Stacker_Sports
As a joke, I once asked ChatGPT to give me a 2025 power ranking for all 32 NFL teams (#1206256). It turned out that the Power Ranking was absolute trash, getting key facts wrong and even listing the same team twice and forgetting about a team!
Today, I asked ChatGPT how long Kawhi Leonard, Paul George, and James Harden played together on the Clippers, and somehow it told me that James Harden signed with the Suns after the '23-'24 season, which I'm sure is wrong!
That got me wondering, why is AI sooo bad with sports information? I understand mixing up power rankings, keeping track of 30+ teams is hard! But a simple fact like what team Harden's on?
So, I asked ChatGPT itself why it got this wrong, and it confessed that it gets confused by the massive amount of fan speculation and internet rumors surrounding sports moves.
I know that ChatGPT regularly consults Reddit for certain topics, and Reddit is (sometimes) reliable, but for sports it's almost certainly gonna get a massive amount of speculation, rumor, and even straight shitposting.
Just food for thought that some people might've been interested in. Crosspost ~AI
write
preview
related posts
72 sats \ 3 replies \ @grayruby 10 Sep
That's really interesting. They can't decipher between legitimate facts and shitposting.
reply
40 sats \ 1 reply \ @SimpleStacker OP 10 Sep
Yeah to me this is interesting insight into why AI is good at certain stuff and not at other stuff.
It'd good at code because for the most part no one is shitposting fake code. Most of the code that exists in public repos online are for real projects.
It might be good at some arcane academic subjects that get relatively little discussion outside of the experts, because most of the stuff posted online is by said experts.
But when it comes to something like sports which is massively popular among everyone, laypeople and experts included, and in which many people deliberately don't take too seriously (a lot of joking, fake rumors and speculation just for fun), it has a lot of trouble sorting fact from fiction.
I wonder if it will learn to approach online sources differently for different topics...
reply
30 sats \ 0 replies \ @grayruby 10 Sep
I am sure it will eventually learn to decipher which data to weigh as more relevant.
reply
40 sats \ 0 replies \ @Undisciplined 10 Sep
Just like my dad!
reply
30 sats \ 1 reply \ @nout 10 Sep
Did you try different AI, like Gemini?
reply
0 sats \ 0 replies \ @SimpleStacker OP 10 Sep
No, I didn't
reply
130 sats \ 3 replies \ @HardMoney 10 Sep
Ai companies are building verifiable finance data apis like perplexity finance or a range of other competitors. I’m sure something similar will occur for verifiable sports data
reply
100 sats \ 2 replies \ @SimpleStacker OP 10 Sep
But then that doesn't feel like artificial intelligence anymore if they have to plug into specific APIs. It's just a natural language interface around a more traditional data service.
@optimism
reply
100 sats \ 0 replies \ @HardMoney 10 Sep
Like this but a sports statistical record search
reply
0 sats \ 0 replies \ @optimism 10 Sep
I'm not an authority on this, I just read and experiment a lot.
It's definitely a deviation from the AGI doom scenario that we've been spoon-fed by megalomanic CEOs for a few years now, but this is precisely what high end integrations do today, even ChatGPT pivoted to this model: "web search", or more generic: using RAG, which is what Perplexity has been doing for a longer time.
Perhaps it's not that strange that the role of the LLM plateaus at that of explaining, translation, transposition 1 and summarization and being an interface. That's the role of language in our day-to-day lives too. It's just not that all-knowing, all-influencing overlord that we've been led to believe was imminent.

Footnotes

  1. https://m.stacker.news/107522
    ↩
reply
0 sats \ 0 replies \ @SimpleStacker OP 10 Sep
Also, sorry ~Stacker_Sports, this is SN -- you can't avoid nerdification even in your jock territory!
reply