Skip to content

bcli: retry a failed gettxout instead of answering that the output is spent - #9718

Open
nepet wants to merge 1 commit into
ElementsProject:masterfrom
nepet:bcli-getutxout-retry
Open

nepet wants to merge 1 commit into
ElementsProject:masterfrom
nepet:bcli-getutxout-retry

Conversation

@nepet

@nepet nepet commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Checklist

Before submitting the PR, ensure the following tasks are completed. If an item is not applicable to your PR, please mark it as checked:

  • The changelog has been updated in the relevant commit(s) according to the guidelines.
  • Tests have been added or modified to reflect the changes.
  • Documentation has been reviewed and updated as needed.
  • Related issues have been listed and linked, including any that this PR closes.
  • Important All PRs must consider how to reverse any persistent changes for tools/lightning-downgrade

Fixes #9714.

getutxout answered {"amount": null, "script": null} both when bitcoind says the output is spent or unknown (gettxout exits 0 with no output) and when the bitcoin-cli gettxout call fails, e.g. because bitcoind is unreachable or still starting up. lightningd takes a null answer as "spent", and some callers act on it: gossip leaves the output out of the filtered block it stores, so the channel announcement stays rejected, and fundchannel_cancel forgets a channel whose funding tx another wallet has broadcast.

This calls gettxout with -rpcwait, so bitcoin-cli retries while bitcoind is unreachable or warming up, until bitcoin-retry-timeout, and then returns an error. Errors that retrying can't fix, like a malformed txid, come back at once. Only an exit 0 with no output still answers null.

After the timeout lightningd still aborts on that error, as it does on any bcli error (#9573). #9574 would let lightningd ride out transient backend errors, starting with getchaininfo. getutxout's callers could follow. Until then this never turns a failed call into a wrong "spent". Like the getblock retry, the wait holds up bcli's other calls while it runs, which only happens while bitcoind is failing anyway.

The tests make the bitcoind proxy fail gettxout twice and then answer: getutxout returns the real output, and fundchannel_cancel refuses to forget a channel whose funding tx bitcoind broadcast. With gettxout failing for good, getutxout fails after bitcoin-retry-timeout and a malformed txid fails at once. Both tests fail on master.

Comment thread plugins/bcli.c Resolved
Comment thread plugins/bcli.c Resolved
Comment thread tests/test_plugin.py Resolved
Comment thread plugins/bcli.c Resolved
@cdecker

cdecker commented Nov 9, 2026

Copy link
Copy Markdown
Member

bitcoin-cli has long had the -rpcwait option to do exactly this type of retrials, shouldn't we use that instead? Or do we want to allow running other calls in parallel? The way it is done now blocks anything concurrent as well.

@cdecker cdecker left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using bitcoin-cli -rpcwait=60 or something similar.

@nepet

nepet commented Nov 10, 2026

Copy link
Copy Markdown
Member Author

Switched to -rpcwait -rpcwaittimeout=<bitcoin-retry-timeout>. It's a better fit than the loop: it only retries while bitcoind is unreachable or warming up (-28), and any other error comes straight back. On parallel calls: bcli runs bitcoin-cli synchronously, so -rpcwait holds up other calls just as the loop did. Avoiding that would mean making bcli async again, which I'd keep out of this PR. One difference from the loop: an HTTP 503 (work queue full) isn't retried. It now ends in a bcli error instead of a wrong "spent".

@nepet
nepet force-pushed the bcli-getutxout-retry branch from 498f658 to 7077a40 Compare November 10, 2026 15:51
@nepet
nepet requested review from Andezion and cdecker November 10, 2026 15:55
cdecker
cdecker previously approved these changes Nov 10, 2026
@daywalker90

Copy link
Copy Markdown
Collaborator

CI is not passing the new test: test_bcli_getutxout_failure_fundchannel_cancel:

        prep = l1.rpc.txprepare([{start["funding_address"]: amount}])
        l1.rpc.fundchannel_complete(l2.info["id"], prep["psbt"])
        signed = l1.rpc.signpsbt(prep["psbt"])["signed_psbt"]
>       bitcoind.rpc.sendrawtransaction(bitcoind.rpc.finalizepsbt(signed)["hex"])
>               raise JSONRPCError(
                    {'code': err.get('code', -345),
                     'message': err.get('message', 'error message not specified')})
E               bitcoin.rpc.JSONRPCError: {'code': -22, 'message': 'TX decode failed Unsupported version number: iostream error'}

.venv/lib/python3.10/site-packages/bitcoin/rpc.py:252: JSONRPCError

… spent

getutxout answered {"amount": null, "script": null} both when bitcoind
said the output is spent or unknown (exit 0, no output) and when the
bitcoin-cli gettxout call failed, e.g. because bitcoind was unreachable
or still starting up. lightningd can't tell the two apart, and acts on a
"spent": gossip leaves the output out of the filtered block it stores,
so the channel announcement stays rejected, and fundchannel_cancel
forgets a channel whose funding tx another wallet has broadcast.

Call gettxout with -rpcwait, so that bitcoin-cli retries while bitcoind
is unreachable or warming up, until bitcoin-retry-timeout, and fail on
any non-zero exit. Errors that retrying can't fix, like a malformed
txid, come back at once. lightningd still aborts on that error, as it
does for every bcli error, but it no longer acts on a wrong "spent".

Fixes: ElementsProject#9714
Changelog-Fixed: bcli: `getutxout` waits for bitcoind (e.g. while it starts up) until `bitcoin-retry-timeout`, instead of reporting the output as spent when `gettxout` fails.
@nepet

nepet commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

Fixed: the test passed the v2 PSBT from signpsbt to bitcoind's finalizepsbt, which older Core can't read (it passed locally on Core v32). It now finalizes with dev_finalizepsbt and broadcasts the raw tx, so bitcoind never parses a PSBT.

@nepet
nepet requested a review from cdecker November 16, 2026 16:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bcli: a failed gettxout reads as "spent", and gossip and fundchannel_cancel act on it

4 participants