Did a Chatbot Solve a Problem for You? \ stacker news ~AskSN

pull down to refresh

726 sats \ 10k boost \ 36 comments \ @optimism 11 Oct AskSN

Since the word plateau is being used more often now when chatbot capabilities are mentioned, I figured that it could be a good time to start doing qualitative analysis of the tech, among stackers. I currently plan to measure this quarterly.

In the past 3 months, have you used a chatbot to solve a problem for you, in work or private life?

If you did this more than once, answer for the problem that was most important to you. ¹

Thank you for your assistance, stackers.

Bonus sats if you leave a comment with a description of which bot you tried and what it could and/or couldn't solve for you. ↩
@remindme in 3 months ↩

Yes, to satisfaction, all by itself37.1%

Yes, but I had to push/correct it37.1%

I tried, but it failed5.7%

No, I haven't tried11.4%

No, I don't use these on principle5.7%

Don't know / other2.9%

35 votes \ 2 days left

view all related items

110 sats \ 0 replies \ @keele 11 Oct

I don't have any chatbot service subscription so I chose "No, I haven't tried". But nearly all of my google or duckduckgo searches now display an AI overview. It's basically always wrong.

I try to keep an open mind towards new technology, but the current AI craze I don't understand. I'm a programmer and I've been doing "AI" since the 90's. I've also used ML in things like image recognition, translation and speech synthesis.

I've read multiple books on artificial intelligence such as "Behind Deep Blue" and of course "Gödel, Escher, Bach". I think Hofstadter is spot on that the paradox of self-reference is present everywhere.

I hear a lot of people say that you need to learn how to use a chatbot. It's a skill and the answers improve as your prompts get better. Well, yeah, this is self-reference in play if I've ever seen it. My problem is that the effort spent on improving the input to the chatbot might as well be spent on proper research into the topic.

For asking simple questions where I can easily spot the wrong answer it's fine. However, when I'm doing coding, I need to understand the code and code written by an LLM can't be trusted. I've also seen some of the answers Gemini returns when I've asked some medical questions and it's horrifying to be quite honest.

200 sats \ 0 replies \ @TheBTCManual 11 Oct

It doesn't happen often so I tend to remember it, recently asked a chatbot how do I capture URL parameters in an internal search query and then push the selections as custom variables into a search report for a hotel so I could give them all searches in a clean report based on their rooms, amount of guests, date range, etc and it was able to spit out a solid solution that worked first time of asking.

Was pretty stoked

200 sats \ 2 replies \ @unboiled 11 Oct

In two (ie. all) cases, it was a resounding no. The bots just regurgitated what the help section already said and in both cases didn't solve my issue.

One bot eventually offered to let me speak to a human. I gave up waiting for a real person to show up.

In the other case, I was able to get an actually working answer by emailing the support address. Interestingly, I had to dig it out of some 3rd/4th level page on their site. I couldn't get the bot to give me that address, either.

21 sats \ 1 reply \ @optimism OP 11 Oct

The on-site chatbots are truly awful and seem to still run on inferior and outdated models. I twice had one of these send me to support only to in one case be ignored, and in the other, go through what it said then in the instructions be redirected to the same bot again!

100 sats \ 0 replies \ @unboiled 11 Oct

The on-site chatbots are truly awful and seem to still run on inferior and outdated models

Yep, I have the same impression. But many places do have humans answering if you poke the bot to get one, so I've had some success before.

For someone like me, who tries the help section first, these types of bots hardly ever tell me an option I haven't tried before.

200 sats \ 2 replies \ @Machiavelli23 11 Oct

I got into a car accident a few weeks ago, no one got seriously injured just scared, I asked ChatGPT if my car was a total loss by uploading a picture of the damage and it said it was after asking a few questions. The insurance company just told me it was a total loss yesterday. I only had to upload one picture to ChatGPT and answer a 2 or 3 follow up questions.Compared that to long point less conversation with insurance agents have to upload multiple pictures and waisted time. I think that in this regard AI can be really helpful and could have people a lot of time and money. Sadly ChatGPT doesn’t work the insurance company so it hasn’t completely solved my problem but at least it gave the answers I needed to begin making predictions.

0 sats \ 1 reply \ @optimism OP 11 Oct

Glad no one got seriously injured!

I'm personally a bit on the fence about binary / judgement questions to LLMs simply because whenever I bench a true cognitive skill "is this A or B", or "express as a % certainty" then the results are still all over the place, also with expensive models. But it's cool that it worked for you.

100 sats \ 0 replies \ @adlai 12 Oct

binary / judgement questions to LLMs

Well they're not classifiers, so they shouldn't be used as classifiers.

How do you explain this without your audience reaching for ChatGPT to ask, "what's a classifier?"?!?

200 sats \ 2 replies \ @cryotosensei 11 Oct

I read your question several times, for you made the use of a chatbot sound rather momentous. On the other hand, I use ChatGPT so reflexively that I have downloaded the app for it haha

I have leaned on ChatGPT in various areas of my life, but you got me thinking about its most significant impact on me. I think ChatGPT has made me a more reflective writer. I typically paste my writing and ask it for a critique that covers good bad and ugly. Ngl, I kinda thrive on the glowing compliments ChatGPT give me - it’s like a morale booster haha. Anyway, having done this enough times, I have since come to realise that I have a tendency to 1) switch tone in my writing and 2) end with a weak conclusion.

I accept and acknowledge the areas of growth. Sometimes I ask for suggestions to see what ChatGPT can come up with. Other times, I brush it aside, embracing the quirks and idiosyncrasies that are uniquely mine. I think my love for writing has deepened

174 sats \ 1 reply \ @optimism OP 11 Oct

I think that it kind of is momentous because 2 years ago (and probably even 6-9 months ago) you would have lol'd at what you're using it for today - or at least the confidence you have in it. And if it has somewhat plateau'd now - in terms of model capabilities - it means there is unresolved (and maybe unresolvable) friction still and that's worth exploring. It's cool that it's there, but does it meet up with expectations? And what does and doesn't work?

A lot of stackers are using chatbots (the usage among us seems to be really high thus far, especially if you realize that the alleged global penetration rate is about 1.5B/5.5B) but the problems we feed it are all different, and results do vary, probably through a mix of prompting effort put in, expectations and problem-space alignment. So I'd like to get a better feel for what experiences people have outside of the common ~AI commenters too.

I typically paste my writing and ask it for a critique that covers good bad and ugly.

Interesting. I generally ask "explain x" when I write about something, to make sure I that I am not completely off the mark. Like a double check. But I never say "I wrote this" - to avoid the PC. The other day I was challenging Qwen3 because it was being generic to a specific question and it actually started calling me names, lol. The training must be absurd at times.

100 sats \ 0 replies \ @cryotosensei 12 Oct

Thanks for the thought-provoking response!

While I’m confident about my writing, I turn to AI with the mentality that there is something to fix about it. Probably because I’m writing for audiences in strictly delineated contexts (eg an email to the entire school, an introduction written by sixth graders for national examiners). So, while I’m aware that accepting some of AI’s recommendations means pandering to formulaic/perfunctory writing, I’m quite eager to give up part of my individuality in these contexts haha.

I’m looking forward to reading your analysis in regard to the attitudes various Stackers bear towards AI and how that subsequently shapes the way we use it!

200 sats \ 3 replies \ @adlai 11 Oct

Bonus sats if you leave a comment with a description of which bot you tried and what it could and/or couldn't solve for you.

I've been making heaviest use of the GPT4Telegrambot that allows selection between several different models and customisation of the SYSTEM prompt. I've been honing it for a while, and needed to make major adjustments across the switch to GPT-5.

I read lots of news channels and geopolitics discussion groups, and strongly dislike the oversimplifying false confidence given by "translate" buttons that don't give you any indication of their level of confidence, and only show you the best guess. I also believe that use of these tools should result in some IA, and not only increased dependence on AI... thus I have developed the following prompt for coping with multilingual chat groups:

Highlight evidence identifying dialect from the following sample;
and then gloss each semantic unit, including phonetic spelling,
English definition, and grammatical interpretation.

REGARDLESS OF DIALECT IDENTIFIED IN THE SAMPLE,
YOUR GENERATED RESPONSE MUST BE UNDERSTANDABLE
BY SOMEONE WHO ONLY KNOWS ENGLISH

I also fed that to GPT5 one time as the ending of a regular prompt, with the first half describing what I've been doing, no customisation of the SYSTEM prompt, to request help improving the prompt. The response overflowed the Telegram relay's message length limits, and also seemed to lean towards the "overdetermination" that I've warned against in another comment. Part of my concern is using fewer tokens, so I've currently remained with this minimal prompt.

100 sats \ 0 replies \ @adlai 12 Oct

major adjustments across the switch to GPT-5

In case anyone wonders about this:

GPT-5 required the addition of the all-caps bit. Earlier models would still respond in English even if I gave them an entire paragraph of foreign language after just the first half of that SYSTEM prompt, however when I used that unmodified with GPT-5, it would basically switch to whichever language comprised the bulk of the USER prompt. I consider this a classical example of how a human foreign-language interpreter would obviously understand that the client needs the interpretation into English, while a GPT model happily barfs whichever language was most dominant towards the end of the context window.

0 sats \ 1 reply \ @optimism OP 11 Oct

Do they have any subscription plans or is it still fully free to use?

100 sats \ 0 replies \ @adlai 12 Oct

I don't think any of these services is "fully" free to use, by ideological standards; and I haven't reviewed any of their policies in detail to figure out which of the freely-given data and metadata is the cost of the freemium tier. Logs are definitely kept while the "remember context" setting is enabled, and I wouldn't be surprised if part of the agreement between services like these and the inference providers allows retention of partially blinded logs by the providers for improvement of their models.

Specifically regarding GPT4Telegrambot, all the heavier models are available only to subscribers; the free tier only gives "lite" models. The ones I used most in the way described above are GPT-4o mini and GPT-5 mini; there are also Gemini and DeepSeek models, although I've not played around with them much just because the OpenAI models were the default and honestly I was more interested with getting good enough results than surveying the entire market.

200 sats \ 0 replies \ @itsrealfake 11 Oct

I use copilot at work. It's great for the sorts of tasks I give it.. it breaks on complicated tasks tho

I've had relatively poorer results with the open models

ive not used ChatGPT much, because I'm ethically put off by the company.

100 sats \ 0 replies \ @astro_penguin 11 Oct

I do not use them at all. I have a strong dislike of how they are created, what happens to people that use them, and what is happening to the internet as a result.

100 sats \ 2 replies \ @SimpleStacker 11 Oct

How big of a problem does it have to be?

30 sats \ 1 reply \ @optimism OP 11 Oct

Whatever was most important to you that you have tried it for.

200 sats \ 0 replies \ @SimpleStacker 11 Oct

In that case, I'd put it this way:

Chatbots solve problems for me every day. The most common things are:
- Asking about some topic I don't know much about.
- Reminding me how to accomplish something using large libraries like matplotlib, pandas, sklearn, etc, including finding built-in functions that save me 5-10 minutes and 30-50 lines of code
- Helping me find a better word to convey a certain meaning
- Reminding me about some mathematical or econometric theorems so I don't have to go look them up
I mostly use ChatGPT and Claude. Claude is a bit better at spinning up code from scratch, I think. Chat is better at reasoning about things, including code.
The biggest problem AI solved for me was when Claude Code built a single-page applet for me from scratch that was meant to illustrate an educational point. (Basically, it runs a simulation to illustrate a point, and the user can set the parameters of the simulation.) I've never vibed anything more complicated than that.
- Overall, i'd say that for this single page site, it did a pretty good job. I had to re-prompt it to add a few features, and had to correct it once or twice, and I can't remember anymore but I think I had to go in and modify the code directly once or twice as well.
- I remember that to get it to work well, I often had to get it to reset its context memory.

100 sats \ 0 replies \ @OT 11 Oct

I haven't spent much time with it. When I have tried it hasn't been able to help me much.

I'll probably continue to try as it's still pretty early and I see there's potential too.

100 sats \ 1 reply \ @adlai 11 Oct

I think you need to narrow down the sort of problem-solving that interests you in the next survey; I've had all four of the main options on wildly different sorts of prompts, and arguably also "I don't use these on principle" because I've not been feeding them things like private chat histories, to respect my interlocutor's privacy.

5 sats \ 0 replies \ @optimism OP 11 Oct

Good point, and noted. Thank you!

100 sats \ 0 replies \ @jakoyoh629 11 Oct

I run with the free ChatGPT and Gemini. They sort out almost everything I need, but honestly, Google could probably do it too. It's just way quicker in the bot chats.

100 sats \ 3 replies \ @NovaRift 11 Oct

I've used it a lot of times and as a student, AI sometimes really help me. I only use Claude, ChatGPT, and Perplexity Pro. These three work more than enough for me and my needs. ChatGPT explains better, Claude does better in technical aspects, and Perplexity is like a very smart search engine that uses GPT, Claude, and many models. I use research mode in perplexity a lot.

But the thing is, no single AI is perfect for you. You need to keep pushing them to correct it, doublecheck the response, check the sources, and reprompt them when they forget previous chats. It's a bit frustrating for me, so I just rely on them for very basic stuff and casual things like searching now. I think we're still a long way from when AI can actually help you with the things the whole hype is about. But it does help you plan things better to be honest, like groceries. It reads what a product contains, at what amount, and what it means for your health. I feel like if you use it in a way that doesn’t cross the red line (I mean, overuse), it's just like a cool tool.

42 sats \ 2 replies \ @optimism OP 11 Oct

But the thing is, no single AI is perfect for you. You need to keep pushing them to correct it, doublecheck the response, check the sources, and reprompt them when they forget previous chats.

It really helps to switch sometimes when there's been a "model influx", like we had the past month-and-half. Try something else. The commercial products are constantly copying each other's successes to try and gain the upper hand.

I use research mode in perplexity a lot.

Example: In terms of commercial chatbot products I found that plain free Grok 4 fast currently (the last few weeks) blows Perplexity Pro out of the water on basic facts accumulation, interpretation and explanation. Like for example, Perplexity could give me some link leads in the footnotes, but Grok incorporated these much better into the answer it gave me, for the same question, and seemed to be much more aware of non-frontpage information (In this case I was asking about the Colombian debt swaps of this year)

But it does help you plan things better to be honest, like groceries.

Interesting. I haven't used it for this. How do you do that practically? Do you ask about products you were looking to buy or do you let it recommend you? Something else?

100 sats \ 1 reply \ @NovaRift 11 Oct

Thanks, I'll try Grok. It was good, but I deleted my X account, so it went with that. I think we can now create a Grok account without having X account? I can say that Perplexity is like a hype too, and I made a wrong purchase. Still, it is what it is for general things, it's fine.

Interesting. I haven't used it for this. How do you do that practically? Do you ask about products you were looking to buy, or do you let it recommend you? Something else?

Like for example, I love stargazing, but what if I'm not sure how will be the weather of the area where I'll be there to watch the sky? Cloudy? Clear? It matters a lot because to escape light pollution, I try to go different areas, and I can ask ChatGPT or any other AI to look for weather in those areas and what time is best to view? (I prefer ChatGPT as its search function is good enough, and explanation is better and straight forward.) It can even help you plan your day if you tell your situations (like if you have an exam and need to cover topics, it will help you from where to start and how I can finish a lot of stuff in time). When I'm purchasing groceries and if I'm confused whether this food item will be healthy for me, I just give the AI picture of its labels, and it gives me an idea of what I can expect. It helps in cooking also, like I tell what stuff is in the fridge and what I can make from that stuff. There is so much you can do from AI, just see it as a tool.

25 sats \ 0 replies \ @optimism OP 11 Oct

I think we can now create a Grok account without having X account?

It is available at grok.com where you can sign-in with email.

100 sats \ 2 replies \ @zuspotirko 11 Oct

In my experience people who complain about having to correct chatbots often (not always) don't prompt long enough. For GPT-5 or GPT-OSS or equivalent models you don't write a Google query into a sentence. You can write 3 - 4 sentences and be surprised.

100 sats \ 0 replies \ @adlai 11 Oct

I've actually been experimenting with minimal prompts; ideally, I'm familiar enough with the model's interpretation of the prompting structure that I can get good results without overdetermining the prompt to the point where there is no information beyond the narrow range allowed.

I think part of the defensible value proposition of these generic models is discovering your "unknown unknowns", and providing some long detailed prompt will eliminate that possibility; however, they're not as deterministic as programs that you write yourself for directly solving tasks, so they'll often fall short of the highest expectations, if you only use them as workers for solving completely-specified tasks, and even when they do perform some task perfectly, the total costs are ridiculous.

5 sats \ 0 replies \ @optimism OP 11 Oct

This is a good point. You need to elaborate your context, so that you can get the answer you seek automatically, because you only get one answer, rather than having to click through a bunch of top search results to find the answer you need.

100 sats \ 3 replies \ @satdesigner 11 Oct

This one yes https://chat.bitcoinsearch.xyz/

0 sats \ 2 replies \ @adlai 12 Oct

Could you please elaborate a little about that service, and how you use it?

100 sats \ 1 reply \ @satdesigner 13 Oct

ChatBTC is designed to help you learn about bitcoin technology and the history of how it was built. All the data is sourced from the bitcoin-dev mailing list, Lightning dev mailing list, Bitcoin StackExchange, Bitcoin Optech, and BTC Transcripts.

0 sats \ 0 replies \ @adlai 14 Oct

Interesting.

Three additional sources that seem notably absent from your list are the Bitcoin Wiki, the old forums (bitcointalk), and IRC logs. It's beyond trivial to log IRC channels¹, and many channels populated by strong personalities end up having official logs to help arguments converge rapidly.

some clients do this by default and it actually needs to be disabled, otherwise you're technically violating the social contract of some communities, although it is impossible to enforce ↩

5 sats \ 0 replies \ @035736735e 11 Oct outlawed

stackers have outlawed this. turn on wild west mode in your /settings to see outlawed content.

In the past 3 months, have you used a chatbot to solve a problem for you, in work or private life?

Footnotes

Footnotes