pull down to refresh

The other option is WAL corruption, I think? Did you have any errors like running out of disk space?

It didn't run out of disk space. Of the channels that were opened today, I can see at least some of the coin onchain. It looks like only the anchors were swept. There is def an attack going on here.

Ill have to run a deeper chantools to see if these outs are unrecoverable.

reply
reply

This matches what I am seeing re: CHANNELD_AWAITING_SPLICE All the attacker channels looked like this -- this is from SOMBERPLOW

{
"peer_id": "02870e583af5f9ca5f5cb3473a279b9e0428b7e0ea5bec36744d4093c9b6cfcf70",
"peer_connected": false,
"channel_type": {
"bits": [
12,
22
],
"names": [
"static_remotekey/even",
"anchors/even"
]
},
"updates": {
"local": {
"htlc_minimum_msat": 0,
"htlc_maximum_msat": 1558593000,
"cltv_expiry_delta": 34,
"fee_base_msat": 1000,
"fee_proportional_millionths": 10
},
"remote": {
"htlc_minimum_msat": 0,
"htlc_maximum_msat": 1558593000,
"cltv_expiry_delta": 34,
"fee_base_msat": 1000,
"fee_proportional_millionths": 10
}
},
"last_stable_connection": 1790796837,
"state": "ONCHAIN",
"scratch_txid": "4a3a4246fa021b49960ef3e7e3cfb2a1278edd7f38d9e9b0646a99bff28a0878",
"last_tx_fee_msat": 1411000,
"lost_state": false,
"feerate": {
"perkw": 1255,
"perkb": 5020
},
"owner": "onchaind",
"short_channel_id": "969346x79x1",
"direction": 1,
"channel_id": "d15dcb8ccc94c74cf0c46f7719f1555413902f434ffa8034e56da74297f11b9a",
"funding_txid": "871b7f86abd319c75d67897b59e90f10f24e9789e0efa64d71272ec1bbfdbeb5",
"funding_outnum": 1,
"close_to_addr": "bc1pkscvdr7pvp63rl07vl9wnrmg28ec5aaky2u0yhqt7ahk6v5ryxas7rjq9k",
"close_to": "5120b430c68fc1607511fdfe67cae98f6851f38a77b622b8f25c0bf76f6d328321bb",
"private": false,
"opener": "remote",
"closer": "remote",
"alias": {
"local": "15388406x15269626x57877",
"remote": "11942166x3538766x39305"
},
"features": [
"option_static_remotekey",
"option_anchors"
],
"funding": {
"local_funds_msat": 1513899000,
"remote_funds_msat": 69026000,
"pushed_msat": 0,
"withheld": false
},
"to_us_msat": 1514899124,
"min_to_us_msat": 0,
"max_to_us_msat": 1514899124,
"total_msat": 1582925000,
"fee_base_msat": 1000,
"fee_proportional_millionths": 10,
"dust_limit_msat": 546000,
"their_max_htlc_value_in_flight_msat": 18446744073709551615,
"our_max_htlc_value_in_flight_msat": 18446744073709551615,
"their_reserve_msat": 15829000,
"our_reserve_msat": 15829000,
"spendable_msat": 1499070124,
"receivable_msat": 47890876,
"minimum_htlc_in_msat": 0,
"minimum_htlc_out_msat": 0,
"maximum_htlc_out_msat": 1558593000,
"their_to_self_delay": 144,
"our_to_self_delay": 144,
"max_accepted_htlcs": 30,
"state_changes": [
{
"timestamp": "2026-09-30T20:04:53.590Z",
"old_state": "CHANNELD_AWAITING_LOCKIN",
"new_state": "CHANNELD_NORMAL",
"cause": "remote",
"message": "Lockin complete"
},
{
"timestamp": "2026-09-30T20:16:18.983Z",
"old_state": "CHANNELD_NORMAL",
"new_state": "CHANNELD_AWAITING_SPLICE",
"cause": "remote",
"message": "Splice signatures sent"
},
{
"timestamp": "2026-09-30T20:54:35.357Z",
"old_state": "CHANNELD_AWAITING_SPLICE",
"new_state": "CHANNELD_NORMAL",
"cause": "remote",
"message": "Lockin complete"
},
{
"timestamp": "2026-09-30T20:54:50.254Z",
"old_state": "CHANNELD_NORMAL",
"new_state": "AWAITING_UNILATERAL",
"cause": "protocol",
"message": "channeld: received ERROR channel d15dcb8ccc94c74cf0c46f7719f1555413902f434ffa8034e56da74297f11b9a: failing channel"
},
{
"timestamp": "2026-09-30T20:59:15.138Z",
"old_state": "AWAITING_UNILATERAL",
"new_state": "FUNDING_SPEND_SEEN",
"cause": "onchain",
"message": "Onchain funding spend"
},
{
"timestamp": "2026-09-30T20:59:15.258Z",
"old_state": "FUNDING_SPEND_SEEN",
"new_state": "ONCHAIN",
"cause": "onchain",
"message": "Onchain init reply"
},
{
"timestamp": "2026-09-30T23:19:20.243Z",
"old_state": "ONCHAIN",
"new_state": "FUNDING_SPEND_SEEN",
"cause": "onchain",
"message": "Onchain funding spend"
},
{
"timestamp": "2026-09-30T23:19:20.468Z",
"old_state": "FUNDING_SPEND_SEEN",
"new_state": "ONCHAIN",
"cause": "onchain",
"message": "Onchain init reply"
},
{
"timestamp": "2026-09-30T23:22:09.273Z",
"old_state": "ONCHAIN",
"new_state": "FUNDING_SPEND_SEEN",
"cause": "onchain",
"message": "Onchain funding spend"
},
{
"timestamp": "2026-09-30T23:22:09.460Z",
"old_state": "FUNDING_SPEND_SEEN",
"new_state": "ONCHAIN",
"cause": "onchain",
"message": "Onchain init reply"
}
],
"status": [
"ONCHAIN:Tracking their illegal close: taking all funds",
"ONCHAIN:All outputs resolved: waiting 84 more blocks before forgetting channel"
],
"in_payments_offered": 16,
"in_offered_msat": 12171712785,
"in_payments_fulfilled": 4,
"in_fulfilled_msat": 1514899124,
"out_payments_offered": 0,
"out_offered_msat": 0,
"out_payments_fulfilled": 0,
"out_fulfilled_msat": 0,
"htlcs": []
},

Eventually these all drop of listpeerchannels, ive taken a copy of what I can.

reply

Maybe you can recover 0f023d4bc23f4d643291c5d5586aac34d0553e761a9548de8994943b0c08b943:3? (with chantools)

reply

The remainder of the channel :-(

The issue is the node isn't seeing these outputs, i think this was a bug in .7?
Ill have to wait the 144 blocks and see if chantools can recover it

reply

Looks like they took that one too, in 46fa610fba9444f9161d9cf022ca86d5d6ba2e7ecb3627df986b62b84f93d653.

reply

The 'remainder' does not seem to be claimed by my node either, because the node has abandoned it. I'm waiting for the last of the legitimate (node shutdown) force closes to settle and then ill chantools scan it.

This attack has fucked up this node and I will recover what sats I can. I have lost confidence in this technology now and will no longer be offering any public services. The security model of lightning with justice transactions is insufficient when an attacker can just fuck your node so it cannot issue the Justice tx, and it forgets to handle the case.
The dream years ago was 1 million lightning nodes routing payments -- this cannot and will not happen, and we will see only professional operations like exchanges running custom lightning servers moving forward.

If you run any public facing internet service today in 2026, consider if the extreme risk is worth it. Software is too difficult to secure now with the capabilities of the attacker.

I'm kicking myself because I got that feeling two weeks ago telling me to pull everything out and shut it down, then i was offline for a week.

reply
548 sats \ 1 reply \ @optimism 1 Oct

I review all software I run and sometimes it takes me a couple of days to get through it. That's clearly too long now. I don't really know how to help because you can't ever say that something is safe; only that you didn't find anything. But that doesn't mean its safe. Nothing that is online and reachable is safe.

Having always-online liquidity is hard right now. I was rebuilding my (private channel only) LND and I simply shut it down a few weeks ago because of all the crap going on. No point in risking it right now.


I did spend some time to build up a framework to quickly simulate CLN and LND error scenarios, but it was no use because they took it before the timelock enabled you. I did get to do the sweep with CLN key inputs:

chantools sweepremoteclosed \
  --hsm_secret <hexdump from cln> \
  --peers <nodeid1>,<..nodeidN> \
  --sweepaddr <from bitcoin core getnewaddress> \
  --apiurl <some esplora capable electrs http endpoint>

it'll give you txs that you can then use with bitcoin-cli sendrawtransaction or add --publish and it'll use the esplora pushtx.


I'm sorry this happened to you. I've not much to offer but sympathy, which makes me even more sorry.

The remainder of the channel

It's the .015 one though, because I think that this is your funding amount? No justice tx (yet) it looks like?

144 blocks

969493 is the trigger number. So 95 more blocks now.

chantools

It ought to work. Otherwise, manually craft it.

reply

That output should be mine (my node forwarded that many sats to peers), but I did not fund this channel, this was the attacker. I'm not quite sure how to to determine if i can claim that back. The node is not tracking that output.

reply

I'll try to simulate this scenario on regtest, see what works