First of all, using ChatGPT (paid version, but not the most expensive one) has been a net positive for my work. I use it a lot to code simple scripts that I know how to code, but would take me 10 times longer to code myself. Also, it has introduced me to CS algorithms that I didn't know of, but that allow me to do some stuff in much cleaner ways. Also, to write research proposals, it's of great help, if only to save time.
But, whereas I used to really check every detail it provided me with, I slowly became complacent and started to trust it blindly, more and more. Then, its limitations hit me, several times, in more or less painful ways.
- I am working on a paper for Science, which, by definition, goes beyond the state-of-the-art. In it, some quite technical stuff is being discussed. I then used ChatGPT to ask it a straightforward question: is method A or method B considered a stricter version of the other one (mathematically). It very confidently told me, method A is stricter than B. However, I know it is the opposite. I worked with some of the people who developed these methods. I told ChatGPT it's wrong. For another 15 minutes, it kept arguing that it may look like that, but that I am wrong and it is right. I did not want to admit its mistake. Because of this, I started questioning many of its previous statements...
- Another time, I asked it a pretty basic question about a widely used software in my field. I was too lazy to Google it. I trusted it at its word, and built 10s of simulations on top of the answer it gave me. Only today, I realized it completely hallucinated the answer to my question. I confidently reported on my simulations to colleagues, but now have to backtrack on several of those statements. And it does not look good when I have to admit it's because I relied on ChatGPT instead of doing the basic checks myself.
- A pretty similar occurrence as the previous bullet point happened a week ago or so. I spent a whole weekend trying to implement some new stuff with ChatGPT, helping me along the way. But as it was new stuff that I did not yet know how to code myself, it took me a while to catch the mistakes and hallucinated assumptions by ChatGPT. Also, this led me to report some shamefully wrong results to colleagues.
- A recent report by a Nature referee (supposedly, the best in the field) clearly showed they had used AI for their report. The em dash, for one, but also because of probably the stupidest question about a feature in my data that no one who works in this field would even dare to ask, as it is so obvious... but ChatGPT would suggest to ask such a question, for sure.
I have no proof for this last one, but it is quite a likely conclusion based on my own experience using LLMs.
Many times, I realize I cannot trust junior colleagues anymore. They face the same AI limitations that I do, but are even less able to spot that ChatGPT can and will hallucinate when it hasn't been trained on the appropriate data for some questions. I often get annoyed by the sloppiness of their code, but can I even blame them if I end up making the same mistakes because of AI?
I am mostly writing this to reflect on my use of AI. It's useful, for sure, but at what cost... I really need to implement a better process so that I can benefit from it without quietly getting f'd without me realizing it at first.
My first instinct now would be to copy and paste this block of text into ChatGPT and ask it to review it for clarity and flow. It'd do a great job, I know, but not today... not today.
So, my question to you? Have you had bad experiences using AI?
code_fixing::correct_semicolons(files[])
that implements the syntactical logic in code, without needing the LLM to actually write correct code.standard --fix
to lint .js files?