First of all, sorry about loosing that much. I would also be devastated and angry if it happened to me.
That said I think we need to add some context to all of this. All the software you use is open source and free of charge. Developers want to build tools to help others to make their lifes easier and in the case of the Bitcoin ecosystem to even change the world and make everyone more free and independent.
You seem to be angry because d11n did not apologise to you on nostr. Reading the reply you wrote, you claim to have lost 4 BTC, you lost your life savings and ask for any help. Anyone could make that claim, there was no context how credible your reply was. Could have been any random troll to react on the alert to bash an already extremely stressed developer, no?
I believe you that you lost the BTC but the blame game here seems a bit off.
Some questions and strange datapoints randomly thrown together, your slogan is don't trust, verify - so here we go.
Sounds like you did not yet have any merchants onboarded, even if you had - putting 4 BTC on the line seems a bit much to make first steps in onboarding merchants?
Software versions: You say 3 months ago you setup btcpay and lnbits:
  • btcpay 1.11.1 was released in July (5 months ago), latest version is 1.11.7 released 18th October
  • lnbank 1.6.2 was released 20th July, latest version (before the vuln fix) 1.8.8 from 20th Nov.
  • lnbits of you site is running v 10.9 from 4th July, latest version 11.2 from 27th Nov.
The reality is even if the vulnerability was disclosed responsibly and a fix was in place for some versions already, you very likely would not have it installed when you were hacked. In comments you and darthcoin say you are technical and very security aware. Sorry, but first thing is to not run outdated software.
Timeline of events: You say 20 mins after you woke up December 6th, noticed 998 outgoing transactions, contacted darthcoin and 20 minutes later you shut down the server when 4 btc were gone.
  • Timestamp in UTC of your last transaction in the sheet is "2023-12-06T21:02:02.150Z", El Salvador is UTC-6, so about 15:00 afternoon.
  • The sum of BTC drained the last 20 minutes is 0,01645517 BTC
  • Draining of funds started 15:00 UTC which is 09:00 El Salvador time, so ongoing for 6 hours (you said you shut down server 20 mins after waking up at 15:00 local time?)
  • in your nginx log there is lnbank call even on Dec 7: [07/Dec/2023:02:05:35 +0000] "POST /plugins/lnbank/wallets/7c996caf-e08a-4b7b-b570-7e5a53eb7aea/receive HTTP/2.0" 200 9532 "https://btcpay.maximacitadel.org/plugins/lnbank/wallets/7c996caf-e08a-4b7b-b570-7e5a53eb7aea/receive" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
The story that you noticed it after you woke up does not quite match to the transaction table nginx logs you provided.
Also the table shows only draining of funds so either you cleaned up 1700 transactions manually from that table or it is a sign that there really where 0 merchants on your node. The Nginx logs show 15k entries for lnbank /send/confirm endpoint so I would expect many more entries in the transactions table.
Don't want to beat an already down man but like your podcasts slogan we are in bitcoin and we don't trust but verify. Certainly the vulnerability in lnbank did not help but running outdated software also is a receipe for trouble and even a fix would not have saved you if you don't update your software stack. Wish you all the best and keep the head up.
Brother, thank you verifying all the data I posted. I can assure you all is true and I was completely drained. This was already confirmed by Bitlifi, BTCPay Server and @C_Otto.
Yes some timings may not match exactly but who's looking at the watch while getting completely fucked? I can't remember exactly what time I got to the computer but I can assure you it was very late because I work on my projects all night long.
Regarding the NGINX log crossing over to Dec 7th: I restarted the node and BTCPay server after disabling the hacker accounts on BTCPay so that people using the server could move their sats to other wallets. I personally told them to do so. During that period there were a few other attempts to continue the attack but the accounts were already disabled.
reply
The reality is even if the vulnerability was disclosed responsibly and a fix was in place for some versions already, you very likely would not have it installed when you were hacked.
This is highly speculative.
One thing is to update to the newest shining version. Another is to do that after a critical security flaw has been found.
How many security vulnerabilities have been found on btcpayserver/lnbank on these three months?
The story that you noticed it after you woke up does not quite match to the transaction table nginx logs you provided.
Everything can be explained if he woke up afternoon. He didn't say he woke up in the morning.
in your nginx log there is lnbank call even on Dec 7
Maybe some testing during the investigation period?
But anyway it's good that you took the time to double check everything that has been said. We indeed should not blindly trust random people on Internet.
reply
I know Hugo personally and he's pretty much a night owl and goes to bed at the very early hours of the morning, so the waking up in the afternoon part is totally normal for him.
reply
Thanks for vouching for that. It's absolutely true!
reply