Correct, your seed phrase is what actually identifies your ability to spend the Bitcoin associated with it. Wallet software just helps you manage what you have.

It is impossible to access any of your funds without the seed phrase, so make sure you keep a copy of it offline.