Agreed. The crucial thing is that you are tying yourself to the mast, as it were, and introducing actual difficulties that prevent you yourself from accessing your stash in a manner that is so friction-full that it makes villainy too expensive / too difficult to bother with, given the other factors.
If this is a credible system -- meaning, it's the normal state of affairs -- then people don't bother trying to casually compromise it, and only really high-value targets get targeted. If it's not normative, then normal people become attractive honeypots. That's why the norms around this stuff matter.