Memory in ChatGPT looks convenient, but... \ stacker news ~AI

pull down to refresh

Memory in ChatGPT looks convenient, but...

150 sats \ 11 comments \ @tonyaldon 15 Dec AI

Memory in ChatGPT looks convenient, you don't have to repeat who you are, what you want, and how you like to chat. But I keep thinking about overfitting: great on familiar context, weaker on new directions. If ChatGPT adapts too much, it risks becoming a comfort mirror, not a sharp sparring partner.

Have you thought about that before turning it on?

view all related items

30 sats \ 10 replies \ @optimism 22h

With several LLMs, I reset context before I re-ask my question with a better prompt. I don't think I'd use memory if I were using chatgpt.

130 sats \ 9 replies \ @SimpleStacker 22h

Indeed! I highly suspect that the people who have killed themselves or gone crazy by chatting with ChatGPT were working with memory on. I strongly believe that if memory were off, and assuming they started new chats each time rather than continue an old chat, that they wouldn't have gone crazy.

I let my kids use AI, but I force them to use it with memory off.

100 sats \ 2 replies \ @tonyaldon OP 16h

How do you explain it to them?

130 sats \ 1 reply \ @SimpleStacker 16h

I tell them straight up that memory on is dangerous. It will reinforce your own wrong beliefs and trap you in self reinforcing loop

100 sats \ 0 replies \ @tonyaldon OP 16h

It will reinforce your own wrong beliefs and trap you in self reinforcing loop

I like the way you put it. It says what I meant from the start, but I could not find the words.

30 sats \ 4 replies \ @optimism 22h

Yes. And then it all depends on the context compaction algo too. I've found in Claude Code (especially with 4.5) that if I need context window compaction, it means I asked too much at once. Although I could benefit from cached context discounted tokens, the result is in my opinion much worse with it, and ultimately expensive, because i need to have stuff redone.

So if memory4coding is anything like memory4unlicensed-shrink, then this is a big engineering error (or, unfortunately, more likely a yolo error)

100 sats \ 3 replies \ @tonyaldon OP 16h

I agree that when I hit the context window limit, the task is often too big. It is also probably not well defined or scoped. It means I did not spend enough time thinking about what I want to get done.

What are memory4coding and memory4unlicensed-shrink? I searched a bit, but I could not find anything.

30 sats \ 2 replies \ @optimism 12h

Ah, I invented these terms as a sarcastic joke, because in both cases (code and chatbot) it's just context manipulation to the LLM chat template, and it's a very similar process on each LLM stack (judging from what can be seen about it from the outside.)

I mean it like this: if the memory function in ChatGPT is as broken as it is in Claude, and people are literally dying as a result, then that is an error on the engineering side (and in this case, I am highly suspicious that it is made because of the lack of diligence, due to never ending pressure cooking.)

Yes, you can sometimes pressure cook your way to results as a startup. But you need time to consolidate and as the org matures, you institutionalize your processes. What's the real difference between OpenAI and Google? Maturity. And I'd suggest that this is why Google does a much better job on the safety front for Gemini than OpenAI does with ChatGPT (or, to be fair, better than any of their competitors.) See for example #1040555 re: OpenAI org.