pull down to refresh
100 sats \ 1 reply \ @SimpleStacker 20h \ parent \ on: Claude runs a vending machine: decides to stock tungsten cubes, loses $250 AI
Aren't you just describing the decoder part of the encoder-decoder model? The part where you decode embedding states into tokens?
Just playing devil's advocate here. I don't have a strong position on how similarly AI and humans think. But I want to push the limit of the argument.
Nice hypothesis.
I think it's different, because embedding-to-token mapping is a lookup (at least it is in
transformers
), simply because integers are cheaper to store than strings whereas the translation thought-to-language is more of another inference process than a lookup (though I'm not sure if that is a mechanically correct assessment because I'm not a neurologist, so grain of salt plz.)Now the fun thing is that how I understand the last and current generation of chat bots is that they actually do the second step as inference too. But the difference is that it's two iterations over the same linguistic base data (with different weights applied) vs our brains - to my understanding - have different source data for each step, eg extremely simplified (grain of salt again) first fight-or-flight, then creative-or-mundane, then narration?
reply