Cool i'll be following, its been too long with LND as the only implementation thats somewhat production ready and not having HA or even an squeel backend... would also like to know more about the cluster awareness so a passive node doesn't broadcast something
LND has actually had support for leader election for at least 3 years already. Some documentation on it can be found here: https://docs.lightning.engineering/lightning-network-tools/lnd/leader_election
But during my testing I did manage to get two nodes to become active at the same time, which is bad. I described it in this issue: https://github.com/lightningnetwork/lnd/issues/8913
This was an LND bug, where it would not resign from its leader role. etcd was working as it should.
Two weeks later the bug got fixed with this pull request: https://github.com/lightningnetwork/lnd/pull/8938
With the patch applied, healthcheck.leader.interval set to 60 seconds and cluster.leader-session-ttl set to 100 seconds, I could no longer produce a situation where multiple nodes were active at the same time.
With this configuration, each lnd node creates an etcd lease with a time-to-live of 100 seconds. This lease is kept alive at intervals of one third of the initial time-to-live. So in this case it is kept alive every 33 seconds. When a node loses its connection to the rest of the cluster, it takes 27-60 seconds to initiate a shutdown. And it takes 66-100 seconds for another node to take over. So in this configuration there is no room for overlap, so no chance of two nodes being active at the same time.
reply
Great drop ty
reply