Compressing Information for Other Humans and Self \ stacker news ~mostly

Compressing Information for Other Humans and Self

3156 sats \ 15 comments \ @k00b 10 Mar mostly_harmless

zaps forwarded to @elvismercury (50%)

In #457261, @elvismercury and others were discussing the intrigue of price and Elvis said:

But wrt what makes prices special: imo, it's that they roll up such gigantic and diffuse kinds of value and instantiate that massive vector into a single number that people can think with. Which means that everything pertaining to price carries that complex bouquet with it -- it's inherently compelling, the perfect psychological racoon trap.

It reminded me of Lex's interview with Meta's head of AI who argues LLMs will not produce AGI because language is a lossy compression of thought and experience. It reminded me of Saylor's, and many others', criticisms of measuring inflation with a single number too. Also, of my founder friends taking the vector of all the things a company must do and turning them into a scalar of "it's all about this one singular manageable thing."

We overwhelmingly collapse vectors into scalars, compressing complex information like thought and experience into something manageable like language, to coordinate with others and our future selfs. I think these compressions are the media of human coordination, even money could be viewed as such a compression, and we compress things for practical reasons. Data compression allows us to transmit whatever is left after compression farther and store it longer than the original.

I often remind myself how hard it is to communicate with people. We're are only ever seeing what any person has managed to compress of themselves.¹ Like prices, every artifact of coordination creates by its nature the mystery of the original. We might not be able to know the uncompressed original vector of anything, but to Elvis's point, sensing the magnitude of the original probably alters our feelings about it quite a bit.

Have you noticed compressed representations of the things you care about as they get shared widely? What is the impact of us compressing them?

Even their compressed self is hardly seen unless we're studying closely. We probably further compress their compressed self to process it too. ↩

view all related items

2224 sats \ 2 replies \ @elvismercury fwd 10 Mar

A thing that is not understood, and then widely misunderstood, is the role of compression in intelligence.

Many people nowadays are familiar with this idea as pertains to machine learning: a common critique in model evaluation is that the model has simply memorized the training set. This does in fact happen, most often when the model has a huge number of free parameters. It has so much capacity that it can simply encode what you give to it.

The most useful behavior comes when the model has a lot of capacity, but way less than what it would take the memorize the relevant training corpus. So for it to do anything interesting, it has to basically compress reality -- to find patterns in it that it detects with reasonable fidelity. If the system can do this, we say that it generalizes.

This is what the brain does, too. Compression and generalization are at the heart of intelligence. (Early work on this is filled w/ attribution controversy.) The limitation -- the collapse of the vector to a scalar -- is what makes it so powerful. It's what lets you recognize diverse situations as, fundamentally, instances of the same kind of thing.

It's also interesting to note that different kinds of brain damage ruin this ability in a surprising way: by giving the person nearly total recall. So the person can remember everything, but, like the model with too many parameters, can't really think. The most famous example of such a case is memorialized in this Borges story.

Anyway. It's intriguing to consider this wrt money, which is a great compressor, as you and @siggy47 are mentioning. It makes money hugely useful, but dangerous, insofar as what is left out of that invisible hand's grasp may be vitally important.

We talk about leaving out bad consequences as externalities, and there are different attempts to mitigate that, but what about leaving out good things? What happens to a value computation that doesn't include the most important thing about being alive, or systematically leaves out the concerns of millions or even billions of beings, whether they're slaves in 1850, quasi-slaves harvesting rare Earths in Congo, women a century ago, or all the animal life on the planet, in perpetuity?

What do you get, then? This amazing system that computes value, what are the consequences of the value that it computes? And I wonder if there's such a thing as a money that captures too much, like an overfit model, or like poor Funes?

228 sats \ 1 reply \ @Undisciplined 10 Mar

or systematically leaves out the concerns of millions or even billions of beings

I think about this sometimes, but not enough to have anything super interesting to add. It's a reminder that prices solve a problem of overwhelming complexity, but the statement of the problem being solved by prices has certain assumptions built into it.

Prices are the mechanism that allows the problem to be solved, but other institutions and incentives are important to make sure it's the right problem to be solving.

25 sats \ 0 replies \ @elvismercury fwd 11 Mar

It's a reminder that prices solve a problem of overwhelming complexity, but the statement of the problem being solved by prices has certain assumptions built into it.

Yes. Related: prices solve one problem of overwhelming complexity, but just because a problem is overwhelmingly complex does not mean it is solved by the naive application or prices. Or perhaps, even their non-naive application.

322 sats \ 3 replies \ @siggy47 10 Mar

The paperback version of Adam Smith's The Wealth Of Nations is 692 pages, yet I have freely reduced it to the phrase "the invisible hand", which I don't think he even used in his book.

Speaking of which, I wonder if the idea of loss compression applied to money and price could be a genuine criticism of the free market- is it possible for all the points of information to be compressed to price?

151 sats \ 1 reply \ @k00b OP 10 Mar

I wonder if the idea of loss compression applied to money and price could be a genuine criticism of the free market- is it possible for all the points of information to be compressed to price?

I think that's a compelling criticism of the free market. Prices often don't reflect negative externalities. Strangely, I can think of cases where prices do reflect purported positive externalities like in ESG products.

154 sats \ 0 replies \ @Undisciplined 10 Mar

The side of the free market that people don't pay enough attention to is property rights. The externalities people talk about as problems for the free market are usually a result of poor property rights assignment or enforcement.

152 sats \ 0 replies \ @elvismercury fwd 11 Mar

is it possible for all the points of information to be compressed to price?

I think it would be possible, even though it isn't happening now. But even if it were to happen, we'd have to keep in mind that it would be a lossy compression.

Relevant to btc, now that I think about it, as a kind of one-way function. In the theoretical limit case, which we could only asymptote toward, you could "hash" all possible information into the price; but, given the price, you couldn't reverse that process and understand the thing in anything like its full complexity.

380 sats \ 1 reply \ @sorukumar 10 Mar

intriguing and thought-provoking writeup.

Even though it is about one idea of compression, it is getting applied in so many diverse contexts that my mind is numb in a good way. So I may be off track here.

First, it is not that the only benefit of compression is coordination, it does bring incremental insight. If we agree with 'I write to know what I am thinking' we have to agree that we don't have a good sense of our thoughts and experiences if we don't compress them, and in some cases, condense them into metrics.

In general, by compressing an idea or experience, we gain wisdom, and when we uncompress a number or a thing we may get incremental wisdom that we didn't get while compressing it. So both are necessary and both are great.

It is a cycle.

On KPIs for a company: I agree with both perspectives and don't view it as a dichotomy. One person suggests condensing company goals into meaningful metrics, while another, after careful consideration, chooses not to. In many cases, the process of debating and creating metrics provides insights and wisdom, It is not about following it dogmatically

Similar thoughts on Inflation. I don't take it as measuring inflation with a single number is 'wrong', I take it as it is not the best way to track it now, and it is due for a change.

30 sats \ 0 replies \ @elvismercury fwd 11 Mar

If we agree with 'I write to know what I am thinking' we have to agree that we don't have a good sense of our thoughts and experiences if we don't compress them, and in some cases, condense them into metrics.

This is a Deep Truth: the act of forcing something into a compressed form is a useful kind of training. I do this all the time, explicitly, but have never considered it this way. Thanks.

208 sats \ 0 replies \ @nullama 11 Mar

This reminded me of compressed sensing, in which you sample a much smaller portion of whatever you want to sense, and then reconstruct the entire signal with that limited data.

But it also reminded me of the crucial aspect of attention. Attention is a form of compression of the current reality, and is critical for sensing the world.

Because you cannot process everything that is happening you need to pay attention to specific things, you choose what you pay attention to, and ignore everything else.

This attention then leads to thoughts in your brain, and those thoughts lead to ideas, then to plans, and finally to actions.

This means that your life quite literally becomes what you pay attention to.

Big tech and the advertising industry know this very well, so they try to get your attention no matter what. Once they get your attention they eventually get your life.

Attention is a crucial thing but many people give it away for free.

Control your attention, control your life.

127 sats \ 0 replies \ @0xIlmari 11 Mar

Planning in software development by simplifying problems to patterns and comparing them to "similar" things we remember doing.

"You solved a problem like this last time using pattern X and it took 4 hours, therefore it will also take 4 hours this time."

And then the engineer descends into the weeds of the problem and it turns out to be more like 4 days.

100 sats \ 0 replies \ @WeAreAllSatoshi 11 Mar

This is deep yet also surprisingly simple. Very intriguing discussion point! It almost feels like we are basically playing a game of telephone with both ourselves and each other, with literally every piece of communication we partake in. No wonder there’s so much confusion!

37 sats \ 0 replies \ @random_ 10 Mar

The Hutter Prize for data compression.

Interestingly enough, also interviewed by Lex Fridman on the topic of AI.

Turn the hutter prize into a block subsidy a la COCP lol.

110 sats \ 0 replies \ @cascdr 10 Mar

The Virgin Single Number Scalar Midwit vs The Chad Matrix Algebra Enjoyooor

0 sats \ 0 replies \ @Lanter 11 Mar

I found this as a perks/attitude manipulator should have. They hide their true intention and strike in the shadow stealthily.

Footnotes