I just hadn't thought about each territory being able to choose which of these to do.

It would be interesting to grant them some of those other powers. I don't see why they shouldn't have banishment powers or maybe something like bail, where you have to pay a fine to regain access.