You are right about this.
But TOR Browser is the system that 99% of people use to connect to the TOR network.
Therefore, if the browser were compromised, network access would be difficult for many people.