Bitcoin wallets can be sorted into two categories; custodial wallets and non-custodial wallets. Custodial wallets are like unsecured bank accounts; kind of like banks in the old west. When you put your money in a custodial wallet, the company that runs that wallet has your actual money, but you have access to it. If a “bank robber” (aka hacker) robs the old west bank (the custodian) your money is gone and you’re out of luck.

Banks will usually have better security than you might have, so it might be ok to let them hold your bitcoin for you, but there is nobody to complain to if the custodian gets hacked. Your money is gone. Most exchanges' wallet (Coinbase, Kraken, Binance, etc.) is this kind of wallet.

A non-custodial (or self-custody)wallet puts you in full control. You’re responsible for security, and if you lose the hardware wallet itself, it is not a problem as long as you have your secure keys saved somewhere. The keys to a bitcoin wallet are represented by an ordered series of English words that you can write down or memorize.

As long as you never digitize these words (no photos, no email, just in your head or on paper), you cannot lose your money, and nobody can take it from you by hacking. There is a saying in bitcoin circles: “Not your keys? Not your coins.”

For small amounts of bitcoin, a custodial wallet is no problem. For larger amounts of bitcoin, I would highly recommend using a non-custodial wallet.

Here’s a good rule of thumb. Only keep an amount of bitcoin in a custodial wallet that you would feel comfortable with in your real wallet. It might be normal for you to carry up to $100 of cash in your wallet. If you lose your wallet, it’s gonna suck, but it’s not the end of the world. Now if you had $5k , you probably wouldn’t carry that in your wallet.

If you ever get to the point that the bitcoin you have is worth more than you would carry in your wallet, you should move it to a non-custodial wallet where you control the keys. See https://www.lopp.net/bitcoin-information/recommended-wallets.html for some good options.

If this is a "medium" amount of money comparable to a month or two of salary, a non-custodial wallet on your phone is fine.

If it starts climbing into a 6-12 month salary or more, I'd recommend a hardware wallet. I have used the original Trezor One, and it is easy to use.

Once you get a non-custodial wallet, be sure to protect the seed words they give you for backup. That is your bitcoin, not the hardware device or your phone. If you have those seed words safely stored somewhere, you can recover from any hardware loss.

If you want to go really secure, stamp those words in metal. I like these cause they aren't expensive: https://crypto-keys.com/pages/jameson-lopp-test

Be sure to never digitize those words. Don't take a pic, don't put them on a computer, etc.. Write them on paper at a minimum (The trezor comes with cards for this), and put them somewhere secure like a safe. Safety deposit boxes at banks are not safe since they can be seized. If you do use paper, make sure they are in waterproof and fireproof enclosures. If you decide to go with a hardware wallet, buy it directly from the manufacturer. If its tamper evident seals are broken, don't use it.