Recover force closed channels \ stacker news ~lightning

1212 sats \ 21 comments \ @npub1q67p49masrcjf__d7qaq6mwh0q 14 May lightning

nearly two months ago now, had to shut down my LND node, most of the channels were closed successfully within a week or so. Other three channels refuse to close so I triggered the force close option. Not sure I made a mistake, but it look's weird that after so long the channels are still in this pending state.

I'm keeping the node on only to recover these channels, ideally recover the UTXOs and move the sats to cold storage. Is that the right thing to do? What's the worse case scenario in this situation?

Shall I just turn off the node and forget about it? Or alternatively, there's anything else I can do to finalise the the closure, maybe check the status via cli to obtain more info?

Any suggestion will be highly appreciated

view all related items

64 sats \ 7 replies \ @Wumbo 14 May

What is the fee rate on the pending transactions?

Is it high enough to get mined?

21 sats \ 6 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

how can I check it? the only things I can see is the channel capacity and remote public key

33 sats \ 5 replies \ @Wumbo 14 May

If you are using a GUI interface there is often a gear icon which gives you more detail to see the close transaction id.

What UI are you using to interact with the LND node?

ThunderHub RideTheLighting terminal etc...

0 sats \ 4 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

have all of them installed, let me check:

on RTL the closing transaction does not appear

https://m.stacker.news/30864

on TH same same, closing transactions ID are not there

https://m.stacker.news/30865

Funn fact, when I check the opening transaction on one of the nodes, I get the following from the mempool website:

Transaction not found. Waiting for it to appear in the mempool...

Other two nodes I can see the opening transaction on the mempool website, both with XXXXX confirmations

33 sats \ 1 reply \ @Wumbo 14 May

In ThunderHub, do you have anything in the pending channel section (top right in channel page)?

21 sats \ 0 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

yes the screenshot I shared above is from the Pending Channels section

21 sats \ 1 reply \ @Wumbo 14 May

We are getting past my level of expertise.

Few other questions and thoughts:

I assume you node is in Sync?
Is the mempool you are using, is it your local mempool or a online website?
In thunderHub can you look in your transactions section and see if you can find the opening transactions manually?
What does amboss space show for these channels for your node?

21 sats \ 0 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

I assume you node is in Sync?

yes indeed, 100% sync

Is the mempool you are using, is it your local mempool or a online website?

both, local mempool and the mempool.space website provide same output. That's is actually really weird because the channel was open since over a year... not sure it was routing but it was definitely open.

In thunderHub can you look in your transactions section and see if you can find the opening transactions manually?

Look's like there's no transaction history older than a month, since when the force closures!

What does amboss space show for these channels for your node?

Two of the peers node are still having the channel as open when I check 1ml, same on amboss. The other node (the one that give me error on the opening transaction ID) does not have the channel active anymore, nor is appearing in its list of closed channels

45 sats \ 7 replies \ @justin_shocknet 14 May

Looking at comments at this point you should only be looking at the CLI to ensure you're correctly getting any errors that might be present

How did you close them? Any errors or odd behavior at the time you closed/force closed?

Did you maybe clean out the node before closing not leave enough in the wallet for anchor fees?

Use ./lncli listchannel to be sure LND doesn't they're just open

./lncli pendinchannels to get the closing tx id, if the closing tx id doesn't exist in the mempool then it might not have gotten broadcast or was pruned already due to low fee

./lncli listunspent to check if you have any utxo's that came back from closures

If some closing tx's are confirmed and you're just waiting on the timelock, you can power off the node and turn it on later to spend those UTXO's, it doesn't need to be online if you just want to keep your state for later use.

21 sats \ 6 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

Thanks for your suggestion:

How did you close them? Any errors or odd behavior at the time you closed/force closed?

I do not remember if and what error I got when trying to close-mutually, that's why I ended up force-closing the remaining three channels

Did you maybe clean out the node before closing not leave enough in the wallet for anchor fees?

No I did not clean out the node before closing these last 3 channels, leaving all the sats from the previous closed channels in there on the onchain wallet

Now, using:

lncli listchannels does not provide any output. Probably because have any active channels atm
lncli pendingchannels provide the following

{
    "total_limbo_balance": "0",
    "pending_open_channels": [
    ],
    "pending_closing_channels": [
    ],
    "pending_force_closing_channels": [
    ],
    "waiting_close_channels": [
        {
            "channel": {
                "remote_node_pub": "hidden",
                "channel_point": "hidden:0",
                "capacity": "43095473",
                "local_balance": "0",
                "remote_balance": "0",
                "local_chan_reserve_sat": "0",
                "remote_chan_reserve_sat": "0",
                "initiator": "INITIATOR_REMOTE",
                "commitment_type": "ANCHORS",
                "num_forwarding_packages": "0",
                "chan_status_flags": "ChanStatusRestored"
            },
            "limbo_balance": "0",
            "commitments": {
                "local_txid": "",
                "remote_txid": "",
                "remote_pending_txid": "",
                "local_commit_fee_sat": "0",
                "remote_commit_fee_sat": "0",
                "remote_pending_commit_fee_sat": "0"
            },
            "closing_txid": ""
        },
        {
            "channel": {
                "remote_node_pub": "hidden",
                "channel_point": "hidden:1",
                "capacity": "30000000",
                "local_balance": "0",
                "remote_balance": "0",
                "local_chan_reserve_sat": "0",
                "remote_chan_reserve_sat": "0",
                "initiator": "INITIATOR_REMOTE",
                "commitment_type": "ANCHORS",
                "num_forwarding_packages": "0",
                "chan_status_flags": "ChanStatusRestored"
            },
            "limbo_balance": "0",
            "commitments": {
                "local_txid": "",
                "remote_txid": "",
                "remote_pending_txid": "",
                "local_commit_fee_sat": "0",
                "remote_commit_fee_sat": "0",
                "remote_pending_commit_fee_sat": "0"
            },
            "closing_txid": ""
        },
        {
            "channel": {
                "remote_node_pub": "hidden",
                "channel_point": "hidden:0",
                "capacity": "10000000",
                "local_balance": "0",
                "remote_balance": "0",
                "local_chan_reserve_sat": "0",
                "remote_chan_reserve_sat": "0",
                "initiator": "INITIATOR_REMOTE",
                "commitment_type": "ANCHORS",
                "num_forwarding_packages": "0",
                "chan_status_flags": "ChanStatusRestored"
            },
            "limbo_balance": "0",
            "commitments": {
                "local_txid": "",
                "remote_txid": "",
                "remote_pending_txid": "",
                "local_commit_fee_sat": "0",
                "remote_commit_fee_sat": "0",
                "remote_pending_commit_fee_sat": "0"
            },
            "closing_txid": ""
        }
    ]
}

and lncli listunspent provide many UTXO

If some closing tx's are confirmed and you're just waiting on the timelock, you can power off the node and turn it on later to spend those UTXO's, it doesn't need to be online if you just want to keep your state for later use.

Ok it make sense, thanks. I'm just worried about the channels staying n that state forever! There's a way to check the status of the timelock?

45 sats \ 5 replies \ @justin_shocknet 14 May

The fact that they're waiting_close and not pending_force_close implies they weren't actually force closed (ui you used to do this may not have handled it correctly)

Try force closing one of them again with lncli closechannel txid vout --force and see if that changes things

There's a potential scan bug that could be at play too, if the force close doesn't change things the LL folks will be best to help https://github.com/lightningnetwork/lnd/issues/8560

Your funds don't appear to be in jeopoardy though, just don't panic and do anything dumb and you'll get your coin back eventually

21 sats \ 4 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

THat's great to hear. Thank you so much! Yeah for now I was just waiting to see if anything unfolded within the node.. but after a while I start questioning things and want to know why this is happening.

It's really weird that aren't force closing! how did you get to this conclusion?

wouldn't be better then to bump the fees as @TNStacker suggested?

In any case, the closing_txid is empty on all 3 channels (I did not delete anything else apart replacing the values on remote_node_pub and channel_point with hidden) and also the Uis (TH and RTL) does not provide me that info, so how can I (re)try to force close?

21 sats \ 3 replies \ @justin_shocknet 14 May

Because it's "waiting close", not "pending force close"

Since there's no closing txid there's nothing to bump

Use the command in my previous comment, the txid is the channel point and the vout is the :0 digit at the end of it (usually 0 but might be higher)

0 sats \ 2 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

tried the one that has vout = 1 and...

[lncli] rpc error: code = Unknown desc = cannot close channel with state: ChanStatusRestored

42 sats \ 1 reply \ @justin_shocknet 14 May

Gives you something to go on at least...

This is odd though since you didn't say it was a data loss / SCB closure, so its either the bug I mentioned before or something else altogether. You should take this to the discussion boards on the LND github to be safe.

My guess is they'll direct you here after further diagnosis: https://github.com/lightninglabs/chantools/blob/master/doc/zombierecovery.md

0 sats \ 0 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

Ok, this start to make sense now...

Also found this https://github.com/lightningnetwork/lnd/issues/7313 with the difference that it also has ChansStatusDataLoss so I assume the data on my side is not lost then 🤞

I'll update when I've any news

Thanks for now

reply on another page

21 sats \ 1 reply \ @Hakuna 17 May

In case your channel close are still pending, you may want to check out my guide at The Guide where your Lightning close-transaction can't get the channel closed

Let me know where you get stuck and I can help out further

0 sats \ 0 replies \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 18 May

This is really useful. Thanks! I'll take the time to learn and play with it

21 sats \ 2 replies \ @TNStacker 14 May

bump the fees. My forced closure went through at 1 Sat/vb

21 sats \ 1 reply \ @npub1q67p49masrcjf__d7qaq6mwh0q OP 14 May

I wish I could do so, but if I cannot see the closing transaction ID, how can I bump the fees? any alternative way?

0 sats \ 0 replies \ @TNStacker 14 May

That's above my skill level. Paging @DarthCoin