Yes we should be worried about Binance hence it is one of the biggest custodians and since the recent problems with FTX. No your keys, know your coins!