If all the bitcoin was held in custodians, on other chains, or on other Bitcoin layers all those entities would still need to settle on chain. In this case it would be like fedwire where only large entities are moving large amounts where paying 130k sats in fees for a transaction isn't a huge deal.

Also by the time this becomes an issue we have no clue what the usage of block space is. There may be some other demand for block space that we aren't even considering.