I had quite a fun night last night because I needed to find out some mystery missing sats! Sericiouly, nothing motivates you to learn more when your precious sats went missing.
The Background Story
I received a new payment in BTCpay yesterday - someone paid for my course, onchain and already has six confirmations, but it's not showing up in my external wallet ( I'm using Blue Wallet as watch-only ), which is so strange; I thought maybe something is wrong with my internet or VPN at first, but checked both are fine; Then I thought maybe some bug in BW because it's quite slow updating, so I created a watch-only wallet in Green wallet to have a check, still, no new tx, how is this possible? And what happened to run my own bank if I can't even locate my own money. 🤔
Then I started looking around in BTCpay, got the tx, and it says 6 confirmations, so someone did pay and I located the address that received the payment, and I also found a series of strange details as I kept digging. 🕵🏼
- BTCpay confirmed the payment and showed the correct balance, yet no new tx in my watch-only wallet. 1
- The newly created addresses from BTCpay don't even match the addresses in my external wallet. 2
- The master key in the BTCpay shows xpub, while my external wallet is showing zpub. 3
Did my BTCpay get hacked, and someone changed the master key?
If you want to exercise your troubleshooting skills, here is how I set up my BTCpay - I'm using Voltage to host it, with a master key from an external wallet to configure the onchain wallet, and the problem is someone paid over onchain and confirmed, the BTCpay dashboard also showing the correct balance, but no new tx in my external wallet, so need to find out the missing tx and the sats.
Curious to see what you would do if you were me. Feel free to pause here for a little creativity and Bitcoin knowledge exercise! 👀
...
Here is What I did
I contacted @Darthcoin to see what he thinks, and he told me that it could be caused by the derivation paths, and he patiently explained to me that xpub, zpub, and ypub are the same, only the derivation path and address format are different - BIP32/44 use xpub, BIP49 use ypub, BIP84 use zpub.
Oh, so basically it can create many addresses under different derivation paths within the same seeds! It's so interesting, but at the same time, it gets quite complicated, as there are so many paths; He also shared this link for me to play around: https://iancoleman.io/bip39/ and told me to try importing that xpub/zpub under different derivations paths in Electrum or Sparrow. So, the new direction to solve this problem is to locate the addresses under other derivations paths. I then used the master key to import the watch only wallet to both Electrum and Sparrow.
Results:
- I couldn't see the path choice in Electum when importing the wallet using Xpub or Zpub.
- I've managed to verify that both the Xpub showing in BTCpay and Zpub showing in my wallet are actually the same in Sparrow, and that's relieving!
And it makes sense why the balance in BTCpay is showing correctly, so no one hacked it, but the new tx went missing in my wallet! One more detail to share, when choosing which script type to restore, you can simply check the address: if it starts with bc1p, then it is taproot; if it starts with bc1q, then it is native.
I even tested with different derivation paths that I found and tried to locate the new tx, but all failed. Then I kept checking and reading because I knew that I wouldn't get a good sleep if I didn't solve this problem. It was about 9pm, so I decided to brew a pot of Turkish coffee to relax and rethink.
Nothing can't be solved with a good cup of tea or coffee In Turkey:)
Calm Down and Rethink
After a cup of coffee, I could think with a much clearer mind and started searching if anyone had a similar issue, but I didn't find anything; then I went back to look at both docs from Voltage and BTCpay, and guess what I found!
Wow, I've never heard of this problem, but it sounds like it! I then also contacted BTCpay to see if there was any bug or update, and someone sent me the same link that I found, which gave me more confidence that this should be the solution for this tx missing problem.
It's not easy to solve the gap limit problem. You have two options:
I would like to keep using my own external wallet, so onto the NEXT - increase the gap limit! and currently, you can do this only in two wallets: Electrum or Wasabi, and I chose Electrum.
🛠️ Process
Step 1.
The first thing I did was move the old sats into a new wallet, because I needed to input the seed words online, and it's always good to minimize the risk in case I messed something up. Sent and done.
Step 2.
I then followed here
a. Enter the following commands in the Electrum console and press enter on your keyboard.
wallet.change_gap_limit(100)
b. Restart and check.
wallet.gap_limit
>>> 20
c. Strange, so I tried again; patience you MUST have!
wallet.change_gap_limit(100)
d. Restart and check again.
wallet.gap_limit
>>> 100
Vola! and then the wallet magically showed the same balance in Electrum as in the BTCpay, it was showing the old balance before ( without the new tx ) , but now it's synced!
Step 3.
I sent all the sats to a new wallet because after inputting the seeds online, the old wallet had served its purpose.
Step 4.
I configured a new master key into BTCpay and wrote down this fun experience to share!
⚡️ Takeaways
- Addresses created within the same seeds are unlimited.
Honestly, I don't think people need to have many wallets. Keeping things simple and learning how to master them is the way - the fact that only 12 seed words can create unlimited addresses blows my mind!
- Freedom is indeed not free.
You need to troubleshoot your own problems and have no one to call or cry on in the Bitcoinlandia! But it's really rewarding and satisfying, you just need to stay calm and enjoy the learning, and be patient.
- Problems can be good learning opportunities, embrace them!
Given that we are still at the early stage of Bitcoin and the majority of people would rather buy ETFs than self-custody Bitcoin themselves, that gives real Bitcoiners many good advantages to test and learn more.
Solving problems is like exercise - the more you solve or have been through, the stronger you are, and the more you learn:)
Footnotes
-
It turned out that is something called the gap limit problem. ↩
-
BTCPay Server uses a derivation scheme to represent the destination of the funds received by your invoices. The destination of those funds will be your wallet, located by the extended public key that you provide. ↩
-
BTCPay Server supports yPub and zPub formats. Please note that these will be converted to xPub once the wallet setup is completed. This has no effect on how you receive or send funds. ↩
showing redis with ``Footnotes