Yeah. Privilege escalation could turn off secure boot - at least all I needed was admin to turn it off last time I tested that. So that risk remains, but it is pretty obvious that this is happening when you boot, because it tells you and iirc it doesn't auto boot then.

optimism

Yea the only obvious hole I can think of without thinking too much about it would be something infecting the boot loader, a physical switch would prevent that, but there's probably other ways to mitigate.