This is indeed the question anyone should ask themselves when they get interested in bitcoin as an investment (you can consider its properties as a means of exchange separately, though of course it all intertwines).
Arguments based on "it's about an agreement between people that it is .. (insert your concept" are superficial. Hence, your:
In other words, the thing that's really scarce is that collective commitment
is to me just fundamentally wrong. That is not what is different about Bitcoin compared to a million other things.
The real answer is proof of work. Not "work == value" (literally the 'labour theory of value', more or less), it's more subtle than that. The proof of work is almost like a binding agent; without it, you could have just as many people committing to some other "history of ownership", but it would be an unstable equilibrium.
Bitcoin is the "money of enemies" because of the proof of work.
This is why I don't believe proof of stake systems will ultimately work in being a "better bitcoin", and also why it doesn't bother me that there are other proof of work coins, because the availability of that "binding agent" is scarce, and it's unsurprising that it overwhelmingly stays with the biggest and most pure instantiation of the idea.
The proof of work that accrues to a project is a function of its commitment to a particular instantiation of an idea. (I'm using PoW in its most literal instantiation here -- hashpower. If you mean "proof of work, in aggregate, across all system activities" then I'd say that's isomorphic to the idea I'm proposing.)
Proof of work is being directed to something. What is that something, and why? That's the distinction here. Here's a thought experiment:
Imagine that a supervillain devoted to the enslavement of mankind gets control of 75% of btc hashpower, and proceeds to censor transactions, double-spend strategically, decry carnivory, etc. What happens?
If the community -- defined in the hazy way that's necessary when talking about something as nebulous as the people who have some kind of stake in btc -- believes, collectively, that this state of affairs violates the ideas that it's committed to, they'll fork away from that chain, despite its preponderance of PoW, and direct their various efforts toward the new enterprise. The fact that there's exahash sitting on the sidelines is not the issue.
In the same way that hashpower follows price, and not the reverse, hashpower is a function of commitment, which itself is mediated by price.