Sign Up
Log In
Log In
or
Sign Up
Places
All Projects
Status Monitor
Collapse sidebar
home:marec2000:cryptocoins:ports
c-lightning
c-lightning.changes
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
File c-lightning.changes of Package c-lightning
------------------------------------------------------------------- Wed Sep 18 13:04:59 UTC 2024 - marec@detebe.org - Update to version 24.8.1: * meta: update python versions * meta: Add changelog for hotfix release v24.08.1 * topology: fix overzealous deprecation check in listpeerchannels. * pytest: add test for bogus deprecation BROKEN log. * schemas: "description" field in "wait(any)invoice" is optional since BOLT12 * subds: fix case where we keep retrying on EOF. * cln-plugin: Change default log level filter back to INFO * plugins/Makefile: fail if jq command fails. * configure: ensure that we have jq. * channeld: fix crash with pre-TLV peers. * lightningd: make sure channels always have a local alias. * lightningd: don't crash if plugin dies during initial plugins_init. * pytest: thoroughly test plugin death during startup. ------------------------------------------------------------------- Thu Aug 29 16:09:40 UTC 2024 - marec@detebe.org - Update to version 24.8.0: * meta: update changelog for v24.08 * docker: Cleaned up Dockerfile's Core Lightning installation * lightningd: fix up installs in subdirectories. * script: Updating SHASUMS for updated zlib1g-dev in Ubuntu Noble * meta: Update changelog for 24.08rc3 * pytest: suppress failing renepay test for Liquid. * askrene: hack in some padding so we don't overflow capacities. * askrene: fix up our handling of htlc_max. * askrene: round capacity *down* when converting to fp16. * pytest: make spendable test for askrene work properly, give a better name. * docs: Added info for NFS version * github: Update FreeBSD Test * github: Update Github actions for Pypi publishing * docker: Fix for cryptography v42 upgrade * docker: GitHub actions testing for docker release * lightningd: fix another trivial memleak when running installed version. * git: Update PR template * pytest: add spendable tests for askrene. * askrene: change finalcltv to final_cltv, and return it in response. * doc: note that getroutes return is different than getroute. * renepay: add daywalker90's test case * renepay: bugfix: apply channel filter globally * renepay: take it easy with local channels * renepay: bugfix refreshgossmap * renepay: bugfix reserved htlc in MCF * renepay: add precondition check + error msg * renepay: patch to assertion htlc_total<=known_max * Adding PR Template * meta: Update changelog for 24.08rc2 * doc: properly document sub-subobject fields. * plugins/sql: fix crash when subobject is missing which has subobject inside it. * bookkeeper: fix up out-of-order migrations in rc1 * docker: Docker file fixes for reproducible builds * renepay: add const probability cost * pyln-testing: fix catching of memleak/broken errors at exit. * pytest: handle more expected broken messages. * lightningd: fix trivial memleak. * meta: Update changelog for 24.08rc1 * devtools: Fixed changlog.py default commit range * gitignore: Add missing gitignore plugins an devtools * lightningd: handle duplicate watches on the same thing correctly. * renepay: add cli option exclude * renepay: add a channel filtering paymod * renepay: disabled channels in a map * cln-plugin: add multi options for String and i64 * reckless-rpc: catch failed reckless subprocess * reckless-rpc: catch failed json parsing of reckless output * reckless-rpc: add documentation * reckless-rpc: catch old installed version * reckless-rpc: accept and pass generic subcommands * reckless-rpc: auto-accept reckless config creation dialog * reckless-rpc: read lightningdir and network from listconfigs * reckless-rpc: initial boilerplate * askrene: memleak: scan reserved htable * askrene: change reserve_hash for reserve_htable * openingd: don't ask for scid_alias in channel type if remote doesn't tell us they support it. * askrene: inform-channel fixes argument order * renepay: fix strange gossmap_refresh log * renepay: fix double free of tal object * renepay: remove __PRETTY_FUNCTION__ * renepay: bugfix finalized routes * renepay: remove unnecessary payment list * renepay: rm HTLCs from uncertainty on route free * renepay: review routetracking * renepay: minor fix * renepay: rethink the computed routes storage * renepay: remove unnecessary rpc calls * renepay: listsendpay before computing routes * renepay: global record of computed routes * renepay: simpler routetracker * renepay: small tweak to decay time * askrene: reserve: fix assertion * bkpr: add payment_id parameter to listaccountevents to filter events. * bkpr: minor clarification in comment. * plugin/pay: fix crash if failcodename isn't set. * lightningd: always broadcast our own gossip when it changes. * lightningd: track whether we're supposed to be throttling gossip. * pytest: test that we inform all peers about gossip changes. * pytest: enhance test_gossip_pruning * lightningd: create new return code for enableoffer. * offer: allow re-enabling a previously disabled offer * doc: Instructions to use the script for generating examples * doc: Updated json examples in schema files * script: Auto generate rpc examples for documentation * pyln-testing: Increasing bitcoin rpc threads from default 4 to 20 * doc: Fixing schema definition * tests: Removed non-essential hsm copy step * lightningd: configvar style fixes * lightningd: trim whitespaces from end of config parameters * plugins: fix more uninitialized vars in bookkeeper. * bookkeeper: fix uninitialized variable. * renepay: remove unused declaration * renepay: bugfix reading pending sendpays * libplugin: Add parser for `channel_hint` from JSON * libplugin: Add primitive to serialize `channel_hint` to JSON * common: Add some more JSON primitives * libplugin: `pay` now emits `channel_hint` as we learn about the net * bkpr: add two custom notifications that we listen for * bugfix: node refuses to re-start after emergency recover * onchaind: Adjust the sweep target deadline for fee estimation * bkpr: properly account for fees and channel closures if splice * coin-moves: when a splice confirms, send a channel_closed event * pay: Use `paymod_log` when filtering routehints for changes * pay: Log the invoice we are about to pay * pay: Add a pre-flight check for the spendable balance * pay: Add an error code to the payments * reckless: correct logging levels * reckless: escape strings in log * reckless: handle failure to find entrypoint cleanly * reckless: handle other --json cases without crashing * reckless: handle unresolvable situations in a --json friendly way * reckless: accept json array arguments as input * reckless: don't polute stdout with python install * reckless: all command functions return objects to enable json out * reckless: add json output option * reckless: add logger class * reckless: make options flags position independent * reckless: add version information to reckless * reckless: add installer for rust plugins * reckless: place source code in source/plugin_name/ * reckless: add type hints for InstInfo * reckless: remove mimetype * Makefile: clean up install path handling. * askrene: add "auto.localchans" layer. * askrene: split json_getroutes into two parts. * devtools/gossmap-compress: allow setting the nodeid explicitly for generated nodes. * plugin/askrene: add "auto.sourcefree" layer. * pytest: simple getroutes tests. * plugins/askrene: attach getroutes call to MCF code. * plugins/askrene: remove local contexts. * askrene: include the mcf and flow routines. * askrene: make the flow.[ch] files compile. * askrene: simply fail if a flow amount exceeds 64 bits. * askrene: remove code which tries to handle tal failures. * askrene/flow: don't omit initial hop in flow_spend. * askrene: copy flow and dijkstra from renepay. * askrene: simplify minflow() * askrene: copy mcf.[ch] from renepay with minimal modifications. * pytest: test file for askrene. * askrene: fast lookup for capacities. * askrene: always set a dummy constraint when we add a local channel. * askrene: flesh out getroutes() a little. * askrene: reservation implementation. * askrene: add layers infrastructure. * askrene: skeleton which does JSON API. * common: promote useful routines from renepay. * bitcoin: add short_channel_id_dir_eq. * common: new parameter parsing routines. * libplugin: add data pointer for plugin convenience. * gossmap: don't process channel_announcement until amount is present. * doc: document askrene. * offers: fix crash when receiving response to offer without `offer_issuer_id` * pytest: make GenChannel arguments explicit. * devtools/gossmap-compress: print out node ids. * pytest: add routine to generate gossmap by topology. * devtools/gossmap-compress: use transparent zlib compression if available. * devtools/gossmap-compress: decompress code. * devtools: add gossmap-compress to give minimal representation of gossmap topology. * macos: fix ci and autoclean * pyln-client: Deprecate category, description and long description from method * gossipd: always ask first peer for all the gossip. * CI: don't fall over if cannot connect to upload server. * offers: fix flake in fetchinvoice. * lightningd: fix false positive in leak detection. * lightningd: don't create a second peer on stub recover if one already exists. * global: update BOLT12 quotes. * plugins/offers: handle invreq_paths in invoice_requests, set them if necessary. * lightningd: make the caller set invreq_metadata and invreq_payer_id for createinvoicerequest. * lightningd: use tweak on node_id of different key, for `createinvoicerequest`. * lightningd: make plugins set metadata for invoice requests. * lightningd: make invreq metdata deterministic, not random for recurring invoices. * common: fix up some minor typos in BOLT quotes. * hsmd: create an hsm variant to sign a bolt12 invoice using a tweak on our key. * common/bolt12_id: generate alias tweak. * common/bolt12_id: new common routine for creating path secrets. * common: update test to encode new test vectors now we handle them correctly. * common/bolt12: allow missing offer_issuer_id. * BOLT12: reject zero-length blinded paths. * wire: add new field `invreq_paths` from spec. * doc: mark schema offer_node_id and warning_missing_offer_node_id deprecated. * wire: rename "offer_node_id" to "offer_issuer_id" to match latest BOLT. * wire: renumber experimental recurring offer fields. * offers: fix invoice return comparison. * pyln-proto: update cryptography to v42 * grpc-plugin: Run cargo fmt for minor style fixes * Revert "ci: Pin `rustc` to version 1.78 due to a type inference regression" * grpc-plugin: Update rcgen * plugin: Removing `category`, `description` and `long_description` from plugin_command struct * rpc: Removing `description` from json_command struct * rpc: Removing `category` and `verbose` from json_command struct * docs: Adding `readonly` category for all get & list commands except listdatastore * onchaind: Remove key derivation involving the signer * tests: Adjust tests for the lack of unilateral attempt now * close: Do not publish unilateral when witnessing a close onchain * ci: Pin `rustc` to version 1.78 due to a type inference regression * common: fix endless loop in gossmap iteration. * common/bolt12, offers plugin: handle experimental ranges in bolt12 correctly. * devtools/bolt12-cli: fix decoding of unknown fields. * common/wire: update, don't replace fields array in TLV structures. * plugins/offers: fix JSON output of decode when there are unknown fields. * common: maintain unknown fields when unmarshalling/marshalling TLVs * doc: allow "offer_description" to be missing in decode. * msggen: fix up previous generated "deprecated": false. * msggen: fix deprecated from "False" to "None". * wire: remove unused obsolete file. * lightningd: store our id as a struct pubkey as well as struct node_id. * common/utils: macros to help get copy/compare across different types right. * wally: update libwally to 1.3.0 * plugins/sql: allow datetime functions Changelog-Changed: Plugins: now allows date and time sqlite functions. * doc: Tiny corrections in documentation * lightningd: remove expermential "sendonionmessage" function. * pay: handle case where we are head of blinded path, and next hop is scid. * plugins/pay: pay to invoices where first hop is a short_channel_id_dir. * common: hoist scidd->pubkey conversion function into gossmap. * offers: add self-fetchinvoices for offers, and self-pay for the resulting invoice. * plugins/offers: create blinded path if necessary. * offers: generalize "best peer" logic into core. * plugin/pay: Handle paying to blinded path starting with us. * lightningd: new internal JSONRPC "decryptencrypteddata" * plugins/offers: allow reversal of a single-element path for reply. * plugins/pay: separate route destination and pay destination. * pytest: fix flake in test_fetchinvoice * unittests: use current version of BOLTS when reading specs. * pytest: add simple test for offers with paths. * plugins/offers: improve reply path logic. * establish_onion_path: allow establishment via any connected peer. * offers: check they use the blinded path if one is specified. * plugins/offers: add dev-paths option for specifying offers blinded paths. * fetchinvoice: don't call establish_onion_path ourselves, now send_message does it. * fetchinvoice: handle blinded paths in offers. * fetchinvoice: don't stash path in sent object, hand directly. * fetchinvoice: use common/onion_message routines. * plugins/offers: don't establish_onion_path at start of processing invoice_request * plugins/offers: make send_onion_reply() handle making transient connections. * plugins/offers: use common/onion_message routines in invoice_request handling. * plugin/offers: add inject_onionmessage helper. * common/onion_message: new unified, documented routines for making onion messages. * plugins/offers: neaten fetchinvoice integration. * plugins: move fetchinvoice functionality into offers plugin. * ci: Adding new step to run basic tests on minimum supported BTC version * doc: Update README for BTC min version * splice: Add funding outnum * git: Ignore .doc_version * Makefile: check formatting of notification schemas too. * doc: fix up added versions for notification schemas. * bolt12: partial support for minimal offer * config: onion messages are now always enabled. * connectd: ratelimit onion messages * connectd: forward onion messages by scid as well as node_id. * lightningd: tell connectd about all scids. * connectd: include map of scid->peer node id. * lightningd: add routine to directly inject an onion message. * common/onion_message_parse: return string, not bool. * chore: add ShortChannelId::to_u64 * Makefile: only run cppcheck on non-generated files. * connectd: tie gossip query responses into ratelimiting code. * connectd: do response to gossip queries, don't hand them to gossipd. * gossipd: drop zlib support. * connectd: use gossmap streaming interface. * gossmap: add linear streaming interface. * gossipd: delete empty gossipd_peerd_wire.csv * connectd: throttle streaming gossip for peers. * doc: fix schema for listconfigs. * config: actually deprecate --experimental-anchors. * plugins/offers: establish_onion_path can have a simple boolean arg for connect_disable. * bitcoin: hoist scid hash into common code. * common: make struct sphinx_hop more generic. * connectd: log incoming onion message IO properly. * lightningd: clarify bolt12 "payment secret" logic. * common: don't pass (unused) has_realm parameter to process_onionpacket. * plugins/pay: fix crash if we try to self-pay a bolt12 invoice. * plugins/pay: don't crash if getroute is empty. * global: we always offer OPT_ROUTE_BLINDING, don't need to test. * connectd: fix missing peer close. * ccan: update to get fdpass fix. * common/msg_queue: make sure to close any pending fds on destruction. * connectd: don't report socket fds twice. * msggen: add `missing-grpc` command * plugins: Remove requirements.txt for python plugins * fundchannel: remove crash on race condition * wire: optimize sending packets now we can return EAGAIN. * ccan: update for more efficient ccan/io. * Makefile: don't make every file depend on version headers. * doc: don't include exact version in manpages. * doc: speed "noop" compilation times by not force-building markdown and man pages every time. * Fix: Add `notifications.rs` to $(CHECK_GEN_ALL) target * ci: Unify bitcoin versions for whole project * doc: Updated warning box for wss-proxy * clnrest: Fixes `ssl_version` deprecated warning * clnrest: Updated requirements.txt (without coincurve) * pyln-proto: update coincurve to v20 * docs: Fixes typo in library name `flask-restx` * docs: Moved python plugins after CLN instructions * fix(deps): add pytest dependency for make check * docs: chore: bump lightning tag in installation.md * fix(deps): add python3-pip to clnrest dependencies * fix(deps): add libsecp256k1-dev to clnrest plugin instructions * fix(deps): add libffi-dev build dependency * docs: notifications schema update * script: Adding Delete action for readme docs * CI: set elements and bitcoind versions in one place. * lightningd: check command should return as much detail as possible. * common: move now-enlarged command_fail_badparam into its own source file. * common: don't mention the contents of invalid parameters when in non-dev mode. * common: new function command_log to log something about a specific command. * lightningd: rename command_log() to command_logger() * pytest: fix flake in test_emergencyrecover * pytest: fix test_feerate_stress race * pytest: fix flake in test_feerate_stress. * reckless: don't use time-based directories. * pytest: fix flake in test_closing.py::test_penalty_htlc_tx_timeout * lightningd: fix race in initialization. * chaintopology: free outstanding requests so we don't get responses while shutting down. * lightningd: pass explicit context for bitcoind request functions. * lightningd: move checkchain_timer out of bitcoind, remove stopped flag. * lightningd: move all chainheight logic into setup_topology. * lightningd: remove unused "min_blockheight" from chain_topology struct. * lightningd: wait for bitcoind at startup if it's fallen behind. * lightningd: remove `first_call` param for getchaininfo. * lightningd: split chaintopology init from normal loops. * lightningd: add arg to bitcoind_estimate_fees callback. * lightningd: pass height back from getrawblockbyheight. * docs: Removed example usage * docs: Example notifications heading * docs: Changes examples format * wallet: Log to chan->log rather than wallet->log * feat(main): Print channel status when loading it from DB * pytest: fix default/description order in zeroconf test plugin. * pyln: Turn the plugin options into a real Type (uppercase T) * pyln-client: allow dynamic option setter to throw exceptions * pyln: Drive-by type fixed * pyln: Add and test a callback based setconfig listener * pyln-client: allow dynamic=True add_option to actually allow updates. * autoclean: call list in easy stages. * autoclean: use filter to reduce processing. * libplugin: add plugin_option_dev_dynamic (our first dynamic dev option coming). * libplugin: support filtering of outgoing commands. * autoclean: rework logic to clarify subsystems. * autoclean: use a pointer for our global clean_info. * autoclean: remove autocleaninvoice command. * lightningd: don't load all forwardings into memory at once. * CI: upgrade CI actions to latest releases * ci: use Bitcoin Core 27.1 * fuzz: Add input for from_bech32_charset off-by-one bug * lightningd: don't try to "save" incoming channel by closing early if incoming already onchain. * lightningd: log BROKEN if we lost funds, only unusual if we closed abandoned HTLC. * msggen: don't spam every time we rebuild. * pyproject.toml: Update ancient flake8 * doc: Added Miscellaneous section for Docker users * doc: fixed broken link to `setting-up-multiarchqemu-user-static` * docker: Update github actions for Docker releases * docs: Updated scripts for rpc examples * docs: Removed null values from example request params * docs: Updated req params type from string to object * onchaind: remove support for rescuing non-static channels when we've lost data. * Gossip: BOLT catch, remove initial_routing_sync. * Doc: update bolts to assume gossip_queries under the new meaning. * Doc: update to BOLT which assumes option_var_onion_optin. * common: BOLT update: option_anchors_zero_fee_htlc_tx is now simply "option_anchors". * channeld: BOLT update, no longer allow creation of old (experimental-only!) non-zero-fee anchor channels. * common: No longer support new channels without option_static_remotekey. * common: update BOLTs to bring LaTeX formatting into onion spec. * common/onion_decode: update to latest BOLT spec, reject encrypted_recipient_data with both scid and node. * tools/check-bolt: rewrite to give more accurate diagnostics when we're out of sync. * devtools/gossipwith: use timestamp filter message not obsolete INIT_ROUTING_SYNC. * poetry: Use non-package mode * script: Adjusting the script with ubuntu:noble updates * script: Adding cl-repro script for easier builder image building * doc: fix markdown in configuration node options * commando: update removal version for rpcs * doc: Fixed decode `first_scid` and `first_scid_dir` descriptions * configure: don't sanitize function call types. * offers: remove deprecated @ prefix. * bcli: remove deprecated `estimatefees` responses. * Makefile: next version is 24.08. * offer: make the blockheight nullable * docker: poetry install before make * script: Added caching option for docker publish and shellcheck fixes ------------------------------------------------------------------- Wed Jun 05 05:49:17 UTC 2024 - marec@detebe.org - Update to version 24.5.0: * CHANGELOG.md: make more consistent. * meta: update changelog for v24.05 * meta: bump rust versions for 24.05 release * gossipd: fix crash when processing pending node_announcements. * script: Add reproducible build script for Ubuntu v24.04 * docs: Update chat links * meta: update changelog for 24.05rc2 * common: translate legacy onion payloads. * pytest: test for forwarding legacy onion. * lightningd: `pay` dev_legacy_hop arg to make second-last onion a v0. * common: re-add legacy v0 hop generation to Sphinx for testing. * lightningd: send CHANNEL_REESTABLISH ourselves on closed channels. * pytest: test that we get REESTABLISH msg reply on closed channels. * chanbackup: typo fix. * docs: Type change for request amount too * schemas: use sat and sat_or_all again where appropiate * pytest: remove xfail in test (accidental commit?) * docker: Add missing git dependency for Fedora Dockerfile * docker: Adding Docker release action * meta: update python versions to 24.05 * Meta: update the changelog for 24.05rc1 * doc: Including docs references for addpsbtoutput where it seems to be missing * msggen: Include AddPsbtOutput to RPC list and meta field map; regenerate libraries * renepay: relax knowledge mod * renepay: put global variables into renepayconfig.h * common: make gossmap ignore redundant channel_announcements. * gossipd: make extra-sure we don't put in redundant channel_announcement messages. * Update Dockerfile and build-release.sh to PR7214 * Use cross compile in dockerfile, fix arm32 support * lightningd: don't ignore fee limits on mutual close. * pytest: test that ignore-fee-limits doesn't apply for mutual close. * offer: fix type to the getchaininfo rpc * bcli: make last height optional * doc: Updating python grpcio-tool dependency * wallet: recoverchannel RPC only stubs a channel if the id is vacant in the DB as well. Closed channels are not loaded inside the peer struct, hence find_channel_by_id fails when the id is occupied by a closed channel. So we need to create a function to directly check if the ID is occupied or not. * doc: Adding manual trigger event * doc: listpeerchannel typo and description fix * renepay: clean unit tests * renepay: fix memleak in CI * renepay: test channel capacity unavailable * unit-test: update mocks * common: Fix off-by-one in `from_bech32_charset` * renepay: fix 32-bit compilation. * bcli: fix compilation on 32-bit platforms. * doc: update rpc docs before pushing to readme * pytest: use reserve_unused_port even more * docs: adds documentation about the log notification * core: notify plugins when a log line is emitted. * msggen: add checkrune method * msggen: add blacklistrune method * msggen: add createrune method * msggen: add showrunes method * msggen: add multiwithdraw method * msggen: add makesecret method * fixup! msggen: add listconfigs method * msggen: add listconfigs method * msggen: fix edge cases in rust composite field names * tools: allow spelling of HtlcTlv * msggen: allow backfilling of already deprecated fields * msggen: add listinvoicerequests, disableinvoicerequest, invoicerequest methods * msggen: add routes to decode and decodepay * msggen: fix invoice request missing exposeprivatechannels * msggen: add help method * msggen: add funderupdate method * msggen: add emergencyrecover, recover, recoverchannel methods * msggen: add dev-forget-channel method * pytests: use reserve_unused_port() everywhere * Updated poetry.lock file * Makefile: Include pyln-grpc proto in PYTHONPATH * gci: Try to fix the pyln-grpc installation in CI * Fix test for notification of custommsg * Include `grpc-msg-buffer-size` in docs * msggen: Allow non_camel_case for notification * Test for notifications over grpc * cln_rpc: Test deserialize_connect_notification * msggen: Schema for ConnectNotification + collisions * grpc-plugin: Implement notifications * msggen: Implement notification-server * msggen: Add notifications to proto-file * msggen: Add conversions for notification pb to json * msggen : Add notification-messages to proto-file * msggen: RequestStructs for notification-streams * cln-rpc: Test for serialization of block_added * msggen: Add test for serialization of notifications * msggen: Choose better when a subfield is optional * msggen: Provide parent in `visit`-method of patch * msggen: Add notification-structs to `cln_rpc` * msggen: Move rpc-model generator * msggen: Cut gprc into pieces!!! * msggen: Add notifications to the `Model` * Add schema's notifications * libplugin: allow display of default values. * libplugin: add test for dynamic setting values. * plugins: two minor things I noticed. * msggen: add upgradewallet method * msggen: add setpsbtversion method * msggen: add setconfig method * msggen: add sendonionmessage method * pay: fix bolt12 blinded path cltv logic. * offers: add final node's CLTV delta in when creating blinded path payinfo. * offers: use node_id not scid inside blinded path. * offers: put correct CLTV limit inside blinded paths. * pytest: add test for paying an invoice with a one-hop blinded path. * cln-plugin: fix over-escaping rpc errors * cln-plugin: Add dynamic configs and a callback for changes * msggen: add splice_update method * msggen: add splice_signed method * msggen: add unreserveinputs method * msggen: add reserveinputs method * pytest: test that we kick out pending transient connections too. * connectd: --dev-handshake-no-reply so we can test pending connections. * pytest: test dropping transient connections. * connectd: implement "transient" connections. * pytest: test fetchinvoice reply path which is not a direct peer. * plugin/offers: connect if necessary for replying to invoice_request. * plugins/fetchinvoice: use new generic connect-if-needed infrastructure. * plugins: generalize "connect if we can't route" logic, link into offers plugin. * connectd: fix exhaustion code where we pick random peer. * doc: Updating installation instructions for python plugins * hsmd: support HSM_VERSION 6 * hsmd: HSM_VERSION 6: get_per_commitment_point never returns secret * hsmd: make the negotiated hsmd version available to libhsmd * channeld: split get_per_commitment_point uses into separate functions * channeld: factor out unneeded make_revocation_msg_from_secret * hsmd: prune unreachable pre HSM_VERSION 5 code * bcli: Add rpcclienttimeout parameter and use max value of it and retry_timeout * common: add unit test for marginal feerate function. * channeld: Adjust the feerate security margin profile * msggen: add sendinvoice method * msggen: add renepaystatus method * msggen: add renepay method * msggen: add plugin method * devtools: make bolt12-cli print blinded paths properly. * offers: handle scid in blinded reply path `first_node_id` field. * bolt12: allow first_node_id in blinded path to be a scid. * common: move json_to_blinded_path into its own file. * plugins/fetchinvoice: remove "blindedpath" command altogether, since we're the only user. * lightningd: move undocumented "blindedpath" command into fetchinvoice plugin. * common: add sciddir_or_pubkey type. * docs: fixing the json response in documentation * pytest: fix flake in test_splice_disconnect_sig * pytest: fix flake in test_rbf_non_last_mined * pytest: update bookkeeper chan lease fee * pytest: update dual-fund tests * dualopend: remove min_witness_weight requirement * global: remove tags from BOLT quotes now dual-funding is in master * multifundchannel: where does this minimum weight appear in the BOLTs? * openingd: be more flexible in allowing unknown segwit inputs. * lightningd, dualopend: update minimum RBF increment from 1/64 to 1/24. * dualopend: make is_dust test match the spec. * common: don't set DF feature in invoices. * BOLT: catch up now Dual-funding is included. * BOLT: update to include 2016 for max_htlc_cltv. * BOLTs: catch up with minor clarifications. * global: fix up quotes which are now in the master branch. * pytest: relax test_low_fd_limit for Github CI. * lightningd: hell, let's start with 64k fds. * lightningd: increase fd limit if we can, to double number of starting channels. * channeld: close an unimportant connection when fds get low. * connectd: use hash table, not linked list, for connecting structs. * lightningd: log when we fail to get an fd from hsmd. * connectd: log when we fail to receive an fd from lightningd. * connectd: log if we fail an accept() call. * ccan: update to get extended ccan/io error callbacks. * Typo fixes from Alex's review * pay: Enable chainlag offset in direct override paymod * pytest: Remove outdated tests * pay: Implement offset and chainlag computation when paying * pay: Add the `chainlag` to the payment * pay: Remove the `waitheight` paymod * pytest: Add a test for the height mismatch logic * renepay: fix some memory leaks * renepay: fix sources * renepay: wake-up result collector every 50msec * renepay: a bit more verbose information in logs * renepay: update test files * renepay: bug fix, local channel information * renepay: bug fix on the routebuilder * renepay: remove payment from route * renepay: add disabledmap abstraction * renepay: refactor routefail * renepay: disable channels not in chan_extra_map * renepay: add a test for routehints * renepay: uncertainty network update revisited * renepay: pay selfpayments on spot * renepay: minor fixes * renepay: update unit tests * renepay: add a test for concurrent payments * renepay: update the Makefile * renepay: add a helper module json.c * renepay: add error codes * renepay: add routefail.c * renepay: add routetracker.c * renepay: add routebuilder.c * renepay: payment state machine (mods) * renepay: the plugin executable is main.c * renepay: THE plugin global structure * renepay: refactor uncertainty API * renepay: rename uncertainty_network for uncertanty * renepay: abandon pay_flow for route * renepay: concurrent pay calls * renepay: separate flow and chan_extra * plugin: notify only the plugins with init state * pytest: clean up tests/test_db.py::test_last_tx_psbt_upgrade * pytest: catch more broken messages in test_important_plugins. * Add missing docs and Changelog-Added line. * drive-by: fix valgrind complaint about uninitialized memory * logging: move noisy gossip recv message to TRACE * logging: move two noisy plugin_hook logging entries to TRACE * logging: add TRACE between DEBUG and IO * lightningd: avoid crash from hsmd not signing anchor spend * pytest: attempt to reproduce hsm signing failure. * docker: Fix jq installation error for focal * pytest: test that the preapprove hooks get called by check. * pay: move preapproveinvoice command out so it can be called bu `check`. * keysend: move preapprovekeysend outside pay loop, so it can be called by `check`. * pytest: test various preapprove scenarios. * lightningd: extend preapproveinvoice and preapprovekeysend to check with HSM. * hsmd: add variant of preapprove commands to have it check only, not do anything. * hsmd: add preinit message to set HSM dev flags. * setconfig: more thorough check for internal config options. * libplugin: allow check setconfig on all dynamic options. * pytest: test the check command in libplugin. * libplugin: handle check for our commands. * lightningd: allow plugins to specify `cancheck` for us to pass check commands through. * lightningd: add optional ->check callback to struct json_command. * param: generalize check handling a little. * renepay: fix gcc -O3 overzealous warning. * ccan: update tal so we can annote allcators as never returning NULL. * pyln-client: pass through level parameter on command notifications. * cln_plugin: Support wildcard subscriptions * Repro: cln-plugin cannot subscribe to wildcard "*" * cln_plugin: Example package subscribing to "*" * pay: ignore uncommited channels in listpeerchannels output * pytest: test pay during channel open * msggen: add disableoffer method * msggen: add missing bkpr-* methods * msggen: add getlog method * plugin/clnrest: Adding new config param as `clnrest-swagger-root` * plugin/clnrest: Adding new config param as `clnrest-swagger-root` * msggen: add delforward method * poetry: add pyln-grpc-proto dependency * bcli: change iteration order on peerlist * ci: Update elementsd version used for tests * tests: add tests for getblockfrompeer * bcli: add version check * bcli: try to fetch block from peer if unknown * bcli: add peers array to the stash * bcli: move getblock call to a separate function * bcli: allow non 0 exit status for getblock call * msggen: add low level fundchannel methods * msggen: add methods autoclean-once and autoclean-status, fix bkpr-listincome * msggen: rust fix for methods containing '-' * msggen: add delpay method * plugin/wss-proxy: Documentation * plugin/wss-proxy: Testing WSS connection * plugins/wss-proxy: Plugin included in core lightning * plugins/wss-proxy: Adding WSS Proxy plugin * msggen: add low level openchannel methods * msggen: fix for methods that contain '_' in their name * spelling: its `listinvoices` and `listinvoicerequests` * lightningd: use return value of peer_start_channeld() * lightningd: trivial cleanup: use tmpctx. * pyln-testing: require explicit pattern for BROKEN messages. * msggen: fix walk through nested json schemas * newaddr: various fixes for msggen and docs * msggen: use field numbers from .msggen.json for rust model if available * schema: add multifundchannel generation * mssgen: preserve existing camel cases an edge case like MultifundchannelChannel_idsChannel_type was previously converted to MultifundchannelChannelIdschannelType instead of the correct MultifundchannelChannelIdsChannelType * plugins/clnrest: simple wrapper to handle missing python3. * libplugin: cleanly separate apply and unapplying payment route. * doc: Updated dev command warning * doc: updated schemas for json_examples * ci: use Bitcoin Core 26.1 * tests: add test_pay_avoid_low_fee_chan * pay: fix removal of channel hint estimated capacity * remove is_local variable from listpeerchannels cb * renepay: limit arc capacities by htlc_max * add more fields to gossmods_listpeerchannels * lightningd: fix memleak where we didn't free plugin_hook_request when it was finished. * lightningd: don't allow mindepth != 0 if they specify channel_type zeroconf. * fundchannel_start: delay, don't refuse, if node not synced. * pytest: rename lightning nodes to reduce confusion. * openchannel_bump: delay, don't refuse, if node not synced. * openchannel: delay, don't refuse, if node not synced. * lightningd: debug log when we send a notification to a command. * lightningd: allow *outgoing* HTLCs before full bitcoind sync. * lightningd: use get_network_blockheight() not get_block_height() for pay. * fix some typos * gossipd: be stricter with non-gossip_query nodes. * lightningd: revert f450dfeb5591d69f78c66a1e1e6fbbd455be6902 to allow non-gossip_query nodes. * Remove spurious import * gitignore: Ignore clangd related things * pytest: test parsing of bolt12 invoice strings in runes. * runes: bolt12 parsing. * runes: handle `lightning:` prefix in invoice arguments * pytest: add test for rune pinv bolt11 parsing. * runes: add support for pinv bolt11 parsing. * doc: document the rune conditions for parsing invoice strings. * plugins: remove allowance for missing JSON `id` in commando. * lightningd: remove createrune restrictions as strings. * lightningd: remove `channels` from `listpeers`. * lightningd: remove handling of invalid `sendpay` route first `channel` * lightningd/connectd: remove --experimental-websocket-port * lightningd: remove `funding_locked` from channel_opened notification. * lightningd: remove obsolete feerate output fields. * autoclean: remove deprecated configs. * doc: correctly remove generated schemas on `make clean`. * lightningd: remove delexpiredinvoice * lightnind: remove deprecated feerate names. * lightningd: remove failure_code from invoice hook and htlc_accepted hook. * Makefile: update CLN_NEXT_VERSION so deprecations tick over. * lightningd: fix name for deprecated APIs, and fix crash with listconfigs when --i-promise-to-fix-broken-api-user is used. * doc: add dev-forget-channel to Makefile * doc: Add json schema for dev-forget-channel * plugins/spender: fix multifundchannel all on more than 1 channel. * multifundchannel: fix `amount` argument in schema & docs (it can be "all"). * tests: multifundchannel and "all" amount * Update hsmtool.c * bitcoin/short_channel_id: pass by copy everywhere. * wallet: fix wallet_forwarded_payment_add to use scid_out arg. * wallet: fix parameter order to hand `const tal_t *ctx` first. * common: remove type_to_string files altogther. * global: change all type_to_string to fmt_X. * global: expose all fmt_X functions for direct use, make uniform. * lightningd: fix type_to_string on `struct channel`. * Makefiles: remove leftover references to $(EXP) * Makefiles: remove dependency on wire/onion_wiregen.o where unnecessary. * lightningd: check rune parameter names with and without punctuation. * lightningd: name error messages a bit more readable. * ccan: update to get rune error message fix. * pytest: make runes amount test more specific. * pyln-testing: use DEBUG_LIGHTNINGD to start gdb on lightningd during test. * pay: add `partial_msat` option to make partial payment. * libplugin: conceptually separate "total due" from "amount we're paying". * doc: Delete markdowns as they will be generated by scripts now * docs: Updates script and fixed test to generate lightning-sql * doc: Schema generation instructions update * doc: Remove unused request.json and schema.json files * doc: updated markdown files * script: Update scripts * script: Updated sql-schema_gen and documented it's jq dependency. * doc: Merge request and schema jsons in a single file * doc: Markdown cleanup * doc: Updated schemas *.schema.json * doc: Updated parameter descriptions for *.request.json * doc: Adding missing `*.schema.json` * doc: Adding missing `*.request.json` * Add type `currency` for offers amount field. Eg.`1.50USD`. * Add type `sat`. * JSON: don't return a raw result string for "stop" and "recover". * configure: disable ASAN for buggy clang. * tx: remove allocating script fetchers * wallet: perform wallet_can_spend exit logic only once * wallet: avoid computing the key fingerprint while looking for matches * wallet: remove output_is_p2sh from wallet_can_spend * addr: avoid a redundant allocation for unknown script types * wallet: avoid database lookups for non-interesting script types * script: consistently take the script length in identification functions * script: remove repeated tal_count calls from scripteq * Revert "core: Add a function check if a script is P2WSH without allocating" * Revert "core: Defer extracting the script until we're sure we'll use it" ------------------------------------------------------------------- Sat Apr 06 12:01:42 UTC 2024 - marec@detebe.org - Update to version 24.2.2: * meta: Update the CHANGELOG.md for the point release * gossipd: be stricter with non-gossip_query nodes. * lightningd: revert f450dfeb5591d69f78c66a1e1e6fbbd455be6902 to allow non-gossip_query nodes. ------------------------------------------------------------------- Sat Mar 09 10:16:58 UTC 2024 - marec@detebe.org - Update to version 24.2.1: * release: Update the changelog to include v24.02.1 * gci: Add manual dispatch trigger to the crate publication * crates workflow: add protoc to workflow environment * bump setup-python version for pypi workflow * fix: pypi github workflow does not allow double quotes * rs: bump crate versions for v24.02.1 * crate-io workflow: replace unmaintained actions-rs * renepay: bugfix situation with htlcmax=htlcmin * renepay: refuse to pay BOLT12 invoice * common, pay: actually test Dijkstra and route finding. * pay: ignore fees on our own channels when determining routing. * common: fix dijkstra scoring. * plugins/pay: fix route scoring calculation. * plugins/pay: fix capacity bias. * common: add routing test using real data which crashes. * common: fix uninitialized member in gossmap. * plugin: Make the recover plugin less noisy * pay: Prevent repeating the preapproveinvoice check * release: Unbreak the Fedora build * release: Bump pyln package versions * release: Update the CHANGELOG.md ------------------------------------------------------------------- Tue Feb 27 21:14:52 UTC 2024 - marec@detebe.org - Update to version 24.2.0: * release: Unbreak the Fedora build * release: Bump pyln package versions * release: Update the CHANGELOG.md * gossmap: fix OpenBSD crash. * ci: Remove the alping compilation test * hsmd: remove deprecated init v2 * hsmd: increase the min version * cln_plugin: switch lifetimes of ConfigOption from static to non-static * reckless: populate local submodules for searching * reckless: prefer to search a local copy of a remote source * reckless: Clone github sources when API access fails * reckless: set relative path of local git repo directories * reckless: initialize repos with submodules and other cleanup * reckless: reduce github API calls * tx_parts: use wally to clone tx inputs * gh: Recude parallelism of Pre-flight checks' * make: The protobuf files are a grouped target * core: Defer extracting the script until we're sure we'll use it * core: Add a function check if a script is P2WSH without allocating * pytest: Stabilize the `test_recover_plugin` test * libwally: upgrade to latest master, fixes testnet block with huge tx. * channeld: Splice resume check should log to DEBUG * tests/test_misc.py: Add test_recover_plugin to check if the funds are being recovered when we lose some state and enter recover mode. * plugins/recover: Load gossmap, connect to old peers and fetch peer_storage and then call restorefrompeer repeatedly. * plugins/recover: entering_recovering_mode should connect to most connected nodes on the network and call emergency_recover immediately. * plugins/recover: Initialise process to detect lost state through listpeerchannels. * Plugins: Add recover plugin which'd identify if we've lost some state and try to recover the node by entering mode. * peer_control: Add lost_state param inside listpeerchannels rpc, which'd help us identify if we've fall behind or lost some state. * Docfix: connec-notification * gossipd: handle case where block closes multiple channels requiring node_announcement moves more than once. * gossipd: don't try to delete node_announcement twice if it's the same node. * Fix incorrect hex value for Signet port * lightningd: add --dev-allow-shutdown-destination-change to unstick existing nodes. * lightningd: clean up close logic, fix bug where we can't override destination. * common: add tal_arr_eq helper. * lightningd: reindent closing_control.c * pytest: add test for issue #7014 * renepay: update the gossmap after addgossip * autoformat test_renepay.py * renepay bug fix: list previous sendpays * lightnignd: Fix another assert crash. * connectd: fix bad assert. * lightningd: don't force state if gossipd gives us an unexpected channel_update. * Add no-reconnect-private option to disable automatic reconnect-attempts to peers connected to our node with private channels only * pyetst: Fix up a test checking fees after fee calc change * channeld: fix update_fee cap. * gossipd: don't assert on redundant flag write, just log message. * connectd: don't suppress channel_announcement without channel_update yet. * gossipd: make sure to mark node_announcement dying if we fast-path remove last channel. * gossipd: don't consider dying channels when seeking preceeding channel_announcements. * gossipd: set dying flag on node_announcements when initially created. * gossipd: preserve the dying flag when moving node_announcement. * gossipd: when applying a new channel_update, preserve the dying flag. * gossipd: make sure to unmark dying node_announcement if we see a new channel! * gossipd: set dying flag on node_announcement when all channels are dying. * gossipd: clean up flags accessors. * common: add gossmap_chan_is_dying() helper to check flags. * devtools: remove warning message from dump-gossipstore. * pyln-testing: dump gossip store when we complain about bad gossip. * doc: fix typos * fixup! dual-fund: add `require_confirmed_inputs` to RBF flows * dual-fund: add `require_confirmed_inputs` to RBF flows * fixup! lightningd: Add tx_abort routine to lightningd * fixup! inflight: Add ability to delete an inflight * fixup! splice: Add support for tx_abort to channeld * splice: Add support for out-of-bound tx_sig * splice: Add support for tx_abort to channeld * lightningd: Add tx_abort routine to lightningd * interactive_tx: Add tx_abort support * inflight: Add ability to delete an inflight * dualopen: update test to handle ABORT * msggen: Regenerate with correct protobuf version * msggen: Regenerate to include `offer` and `listoffers` * make: Add missing dependency between schemas and schema bundle * `cln_plugin` : Test default values for ConfigOptions * cln_plugin: Add documentation * cln_plugin: Request value as rust primitive * Allow `ConfigOption` to be specified as const * cln_rpc: Refactor `ConfigOption` and `Value` * cln_rpc: Store ConfigOptions in HashMap * ci: Install python build dependencies in alpine image * make: always generate proto files sans rust * msggen: Start making the `msggen` library a standalone tool * wallet: ensure all established channels have aliases. * Fixed indentation errors. * Fixed raising TypeError instead of ValueError when there is an invalid type. * pytest: fix flake in test_htlc_in_timeout with anchors. * pytest: fix bad gossip flake in test_closing_specified_destination. * options: make anchors enabled by default, ignore experimental-anchors. * pytest: changed tests if we're using experimental-anchors. * lightningd: allow dev-force-features to unset even if not set. * funder: don't try to spend emergency_reserve * wallet: be a little more flexible with change for emergency reserve. * lightningd: correctly handle case where we have no fee estimates opening anchor channel. * msggen: Fix the `offer` schemas * doc: Fix schema for `offer.amount`, it can be a non-BTC unit too * feat: added listoffers grpc command * doc: adding schema to create offer requests * feat: adding offer command to pyln-testing and msggen * feat: grpc - adding support for `Offer` command * fix: fixing start_any_period type for offer recurrence * doc: update reckless documentation * pytest: add tests for pip and poetry installed virtual envs * reckless: update timeouts * reckless: create virtual environments for python plugins * reckless: add option to check out a specific commit ... using the syntax reckless install <plugin_name>@<commit hash> * reckless: add installation metadata * reckless: add staging directory and symlink * reckless: properly remove entry from reckless sources * Add a method to the pyln-client for using the sendcustommsg api * lightningd: deprecate @-prefix hack for offer recurrence_base. * common: don't insist on unique param() arguments. * cln-grpc: Patches rpc-file path * Removed duplicate keys in some dictionaries. * fuzz: use explicit fetching for digest algorithms * Revert "ci: Split out the fuzz testing" * regtest: fix check for bitcoin-cli * pytest: another bad gossip flake * pytest: fix YA gossip flake. * gossipd: simplify gossip store API. * gossipd: clean up gossip_store offsets. * gossipd: clean up gossip_store routines. * gossipd: remove routing.c and other unused functions. * gossipd: switch over to using gossmap_manage, not routing.c. * gossipd: make the query response code use gossmap_manage. * gossipd: make the seeker code use gossmap_manage. * gossipd: re-implement flood protection. * gossipd: don't validate UTXOs on our own channels. * gossipd: interface to have gossmap_manage send updates on init. * gossipd: implement pruning timer inside gossmap_manage. * gossipd: gossmap_manage helper to get a node's address, if any. * gossmap_manage: new file for managing the gossip store. * gossip_store: remove infratructure and bits for marking ratelimited entries. * gossipd: remove spam handling. * gossipd: remove zombie handling. * gossipd: move gossip_store pointer from struct routing_state to daemon. * gossipd: simplify seeker startup. * gossipd: move timestamp_reasonable out of routing.c * gossipd: move dev flags from routing struct to daemon struct. * gossipd: have seeker quert interfaces take an id, not a struct peer. * gossipd: pass node_id to queue_peer_msg, not peer. * gossipd: make gossip_store hold daemon ptr, not rstate. * gossipd: remove online gossip_store compaction. * gossipd: move tell_lightningd_peer_update from routing.c into gossipd.c * gossipd: take signature checks out of routing.c * gossipd: take txout failure cache out of routing.c * common: optional gossmap callbacks for better failure handling. * common: gossmap method to load fd directly, not filename. * gossipd: new routines to support gossmap compatibility. * common/gossmap: remove now-unused private flag. * Convert fee result into msats * Remove update_count from find_account_onchain_fees * Remove grouping by update_count in finding onchain fees * Add xfail tests for onchain fees lookup query * regtest: error message if bitcoin-cli is not found * pytest: fix another gossip flake. * pytest: fix gossip_store load test for EXPERIMENTAL_SPLICING=1 * lightningd: fix reference to uninitialized channel_type during opening. * fixed source (using raw millisatohis) * renepay: flow routes limited by htlc_max/min * rpc: Regenerate derived files after adding bkpr-listincome * msggen: Add classes for MethodName and TypeName * fix : implements the `bkpr-listincome` plugin for grpc interface. * ci: Split out the fuzz testing * pyln: Include py.typed file * configure: abort on UBSan failure * channeld: use explicit revoke on retry when HSM_VERSION >= 5 * Rename `FeatureBitsPlace` to `FeatureBitsKind` * cln-plugin: Allow user to set featurebits * cln-grpc: Derive serde Serialize/Deserialize for types * tests: augment test_onchain_their_unilateral_out to check to_remote witness generation * Update LICENSE * cln_plugin: custommessages in `get_manifest` * update the documentation * add --show-secrets option for hsmtool dumponchain * hsmtool: add feature to dump private keys * ci: fix the CI configuration for rust release * lightningd: don't generate node_announcements with identical timestamps. * pytest: don't trigger bad gossip message in test_routing_gossip_reconnect. * pytest: fix flake in test_wumbo_channels * lightingd: corrections from Alex Myers's review. * gossipd: make seeker uintmap visible to memleak code. * lightningd: expose last_stable_connection in RPC. * lightningd: set last_stable_connection 1 minute after channel reestablished. * lightningd: add last_stable_connection field to db, channel. * lightningd: expose `reestablished` field. * channeld: notify lightningd once we've exchanged reestablish messages. * gossipd: remove node_announcement generation. * lightningd: turn on node_announcement generation. * lightningd: move node_announcement unit test from gossipd/ * connectd: channel_gossip when we've tried to connect to all peers. * lightningd: refine heuristics for remote address selection. * lightningd: don't create node_announcement until gossipd is finished. * gossipd: tell lightningd its node_announcement on startup, if any. * lightningd: add routines to generate node_announcement. * lightningd: check channel_announcement signatures we get from peer. * gossipd: remove channel update/annoounce creation and local update handling. * common: remote private_channel_announcement.[ch]. * gossipd: don't put private channel info into store at all. * gossipd: no longer take private channel updates from lightningd * dualopend: remove channel_update message creation. * channeld: remove channel_update message creation. * lightningd: ignore gossip messages from channeld, switch to our own. * lightningd/channel_gossip: new file for handing and generating gossip for our channels. * lightningd: add gossip_generation.c * lightningd: tweak db remote channel_announcement sigs API. * channeld: tell lightningd scid from `announcement_signatures` message. * connectd: separate routine to inject message without closing connection. * lightningd: set channel's local alias at init. * dualopend: tell lightningd what remote alias the peer sent. * gossipd: get told about new block before splicing code tries to feed us channel_announcement. * gossipd: strip private updates from gossip_store on startup. * hsmd: add command to allow lightningd to sign channel announcement. * lightningd: use explicit accessor to get channel_update for errors. * lnprototest: remove. * pytest: fix flake in test_upgrade_statickey_onchaind * topology: don't call gossmap for locall added channels. * pytest: test_gossip_pruning should not assume redundant node_announcement. * pytest: prepare sql test for faster gossip generation. * pytest: fix up test_zeroconf_multichan_forward once routehints fixed. * pytest: fix splicing tests to be more explicit, and not rely in private gossip messages. * pytest: check for short_channel_id, not status. * pytest: Remove check for to-be-removed message about msg from channeld. * pytest: make sure listincoming sees remote update in test_zeroconf_forward. * pytest: make sure gossipd sees channel before we close it. * pytest: ignore private updates in test_addgossip. * pytest: fix psbt db test. * Schema: Add schema's for detailed fundchannel methods * Doc: Undocumented parameters in fundchannel_start * pyln: Consolidate on pyln.testing.db * wipe_db() implementation * pytest: fix flake in test_peer_anchor_push * pytest: fix flake in test_custommsg_triggers_notification * openingd: make channel-type rejection message clearer. * pytest: check that listpeerchannels gives the same channel type as returned from fund/openchannel * fundchannel_start & multifundchannel: add channel_type. * dualopend: fix channel_type if we don't negotiate the default. * common: move json_add_channel_type from lightningd/ * lightningd: make option_static_remotekey compulsory. * lightningd: add dev-any-channel-type to allow obsolete channel types. * spenderp: add channel_type parameter to fundchannel / multifundchannel. * fundchannel_start / openchannel_init: add a channel_type parameter to force channel type. * common: channel_type_accept doesn't use `their_features`; remove parameter. * common: add param_channel_type helper. * openingd: correctly mark channel type zeroconf if negotiated and they offered minimum_depth=0. * lightningd: make option_gossip_queries compulsory. * lightningd: make option_data_loss_protect compulsory. * lightningd: fix fd leak with check command on openchannel_init. * msggen: Make `ChannelTypeName` a native enum * docs: Fix missing escapes for underscores in multifundchannel docs * lnprototest: update to origin/no-empty-channeltype for now. * channeld: use revoke_commitment_tx if hsmd supports * hsmd: separate revoke_commitment_tx * channeld: defer construction of revocation message until master sync * channeld: add hsm_capabilities and add hsm_is_capable to common * renepay: accomodate fees in the payment flow * bkpr: now that we're not doing empty acct logging, we dont need this bool * bkpr: use currency in balance snapshot * bkpr: dont log unknown accounts with zero balances * libplugin: remove global deprecated_apis flag. * commando: use deprecation API for missing ids. * plugins/bcli: use per-command deprecation flags. * libplugin: handle `deprecated_oneshot` notification. * plugins: allow plugins to get per-connection deprecated state. * libplugin: get `i-promise-to-fix-broken-api-user` list from lightningd. * lightningd: mark all internal deprecations by version. * pyln.plugin: Allow deprecated to be a string. * libplugin: support version strings for deprecations. * plugins: allow plugin options `deprecated` to be an array of strings. * plugins: allow `deprecated` for registered commands to be an array of versions. * common: allow JSON-RPC parameters to specify deprecation versions. * jsonrpc: add deprecations command to locally set deprecated apis on/off. * plugins/sql: use per-command deprecations. * common: add command_deprecated_param_ok() and command_deprecated_out_ok() * pytest: note where we use very deprecated features, so tests don't rbeak. * doc: start a list of current deprecations. * lightningd: i-promise-to-fix-broken-api-user option. * plugins/sql: do deprecations at request time, not schema loading. * plugins/sql: make columns an array of pointers. * schemas: deprecated is now a range. * msggen: generalize version handling. * Makefile: add CLN_NEXT_VERSION, functions encoding deprecation schedule. * lightningd: reuse code for "dynamic" getmanifest response parsing. * config: remove disable-ip-discovery * funder: remove lease-fee-base-msat * wallet: remove P2SH support. * pyln-client: remove long-obsolete support for msatoshi arg. * common: remove parameter aliases. * common: remove no-longer-required p_req_dup_ok hack. * lightningd: remove msatoshi alias for amount_msat. * lightningd: clean up notification infrastructure. * renepay: add self-pay feature * Update configuration.md * fuzz: initial fuzz-bolt12-invoice-decode corpus * fuzz: initial fuzz-bolt12-invrequest-decode corpus * fuzz: initial fuzz-bolt12-offer-decode corpus * fuzz: initial fuzz-bolt12-bech32-decode corpus * fuzz: test bolt12 decoding * fuzz: test bolt12 bech32 decoding * fix CI test: uninitialized variable * fix CIL maybe-uninitialized variable * error handling in mcf.c * move function linear_fee_cost to mcf * remove unnecessary arguments * remove declaration of derive_mu * renepay: low level update knowledge failure * uniform error handling pattern for flow.c * renepay: remove unused derive_mu function * remove amount_accumulate and amount_reduce * remove flow dependency on payment * rs: Temporarily remove a test that is failing due to log changes * hsmd: Implement hsmd_forget_channel for Channel Deletion in HSMD * py: Publish pyln-grpc-proto following new methods * doc: add a nice explainer about how to update PR branches * Update installation.md * cln-plugin: Switch from `env_logger` to `tracing-subscriber` * cln-plugin: Allow setting `usage` of RPC-method * chaintopology: fetch item counts just once when iterating * chaintopology: do less work when iterating blocks * bitcoin_block_from_hex: avoid creating PSBT wrappers for finalized block txs * fromwire_bitcoin_tx: avoid creating a PSBT wrapper just to throw it away again * dualfund: avoid creating PSBT wrappers for input UTXOs * interactive-tx: avoid creating PSBT wrappers for input UTXOs * tx: add pull_bitcoin_tx_only to pull without creating a psbt wrapper * tx_parts: use wally to clone tx outputs * base58: use _n api variant to avoid duplicating our input string * psbt: use wally_map_get instead of wally_map_find * psbt: use wally_map_replace instead of coding our own * wally: build debug or release according to the top-level configuration * wally: remove now-redundant BUILD_ELEMENTS define * wally: update libwally to 1.0.0 * wally: rename functions that clash with wally v1.0.0 * devtools: Deduplicate links from linkify * `cln_plugin` : Add basic test to `cln_plugin` * Rust cln-rpc crate: Fix `call_enum` and `call` * Some basic documentation for `cln_rpc` * cln_rpc : use `call_typed` with external structs * cln_rpc: Split low- from high-level API calls. * configure: Use the system `PKG_CONFIG_PATH` * doc:correct listrunes's link * gci: Add build step to PyPI publication * meta: Fix the `tools/build-release.sh` script to match repro instructions * docker: Use `git clone` as the build basis for reprobuilds * misc: Improve generated traces from common/trace.c * fix: load script * invoice: force inclusion of min_final_cltv_expiry for compatibility * rust: use "map_or" instead of "if let" * pyln-client: don't automatically turn JSON into Millisatoshi class. * pytest: don't assume that msat fields get turned into Millisatoshi. * Update pyln-testing cheroot dependency * Update and rename backup-and-recovery.md to backup.md * Update recovery.md * doc: Recovery using database * Docs: Add `custommsg` notification * Implement tests for notifications on `custommsg` * Create notification on `customssg` * Publish code-generation.md (#6948) * tracing: Add an example exporter to zipkin and document it * test: Add a `run-trace` test and document tracing overhead * common: Add trace_cleanup() function for unit tests * psbt-fix: set to NULL, which will cleanup dangling pointer * doc: documentation update for hooks and testing * Docs: Consistent whitespace in developers guide * Sync bind-addr docs * rs: Bump versions of `cln-plugin` and `cln-rpc` * grpc: Add metadata to Cargo.toml * pyln: Also generate type stubs in pyln-grpc-proto * gci: Upload to test.pypi.org only from `master` * build: 32-bit status_debug fix * fix typo * fix typos and add suggested edits from community * splice: Agressive restart testing during splices * Doc fix: dynamic options * Allow dynamic option in python plugin * gha: Add `--skip-existing` option to publish to testpypi * plugins: re-enable listchannels local info in deprecated mode. * topology: deprecate overriding `active` in `listchannels` with `listpeerchannels` info. * common: gossmap now always ignores private gossip_store messages. * topology: don't show private nodes in listchannels. * pytest: skip test_hook_crash on liquid CI * pytest: fix tests in assumption that listchannels will no longer show private gossip. * topology: don't show private nodes in listnodes. * pytest: remove test waits for routehints. * lightningd: make listincoming use gossmods_from_listpeerchannels(). * lightningd: permit wrong-alias lookups for private channel_update. * pytest: fix test_zeroconf_open() * plugins/renepay: use gossmods_from_listpeerchannels instead of private gossip_store records. * plugins/renepay: add localmods later. * common: add tests that json_scan can omit an entire object member. * plugins/pay: use gossmods_from_listpeerchannels instead of private gossip_store records. * libplugin-pay: make sure get_gossmap calls are balanced by put_gossmap. * plugins/fetchinvoice: use gossmods_from_listpeerchannels instead of private gossip_store records. * plugins/topology: temporary hack to allow per-access private gossmap suppression. * common: generic routine to turn listpeerchannels into gossmap local updates. * gossmap: insert temporary per-caller flag to turn off private gossip. * pyln-testing: when waiting for active channel, make sure remote has sent fee info. * listpeerchannels: show gossip updates. * gossipd: always send information about our channels' gossip updates. * gossipd: pass remote private channel update to ld and stash in the database. * db: add private channel inbound fields * gossipd: correctly attribute our own channel update to ourselves * pytest: use wait_local_channel_active * common: prepare unit tests for lack of private channels. * compiler: fix for -O3 errors. * Implement `is_some` for `cln_rpc::options::Value` * Doc: Formatting * Fix docs for invoice_payment hook and notification * doc: Documentation update * docker: Creating docker `latest` image * subd: Do not send feerate updates to non-channeld subds * misc: Add Alex's gpg key * misc: Add Shahana's gpg key * startup_regtest: test for clnrest dependencies before writing config * startup_regtest: configure clnrest and generate rune * coin_mvt: use the `lightning_hrp` for *all* coin movement currency * startup-regtest: fund_nodes make balanced, funded channels * startup-regtest: flag every use as developer * startup-regtest: add ability to set bitcoin-bin + lightning-dir * ci: Force flaky of tests * chore(CI): Add manually triggerable PyPI publication * chore(CI): Increase the timeout to publish to test.pypi.org * misc: Add peter's gpg key * CHANGELOG.md: Update for 23.11 final ------------------------------------------------------------------- Fri Dec 22 17:16:30 UTC 2023 - marec@detebe.org - Update to version 23.11.2: * meta: update CHANGELOG.md for v23.11.2 * invoice: force inclusion of min_final_cltv_expiry for compatibility ------------------------------------------------------------------- Sat Dec 16 13:17:39 UTC 2023 - marec@detebe.org - Update to version 23.11.1: * meta: Update CHANGELOG.md for v23.11.1 hotfix * subd: Do not send feerate updates to non-channeld subds * build: 32-bit status_debug fix * CHANGELOG.md: Update for 23.11 final ------------------------------------------------------------------- Tue Nov 28 19:23:30 UTC 2023 - marec@detebe.org - Update to version 23.11.0: * CHANGELOG.md: Update for 23.11 final * doc: Update example to be valid codex32 string * doc: Add `getemergencyrecover` to the hsmtool doc * hsmtool: Fix argc check on getcodexsecret * CHANGELOG.md: Update for 23.11rc3 * clnrest: Import sys in except clause explicitly * lightningd: don't print out notification msat fields as strings. * tests/clnrest: Test to confirm that `msat` suffix has been removed from notifications * tests/clnrest: Added rpc method name and params to use the same `notifications_received_via_websocket` function for other rpc requests too * CHANGELOG.md: Update for 23.11rc2 * splice: Reestablish when commit or sig sends fail * lightningd: fix up deprecated `rest-port`, `rest-protocol`, `rest-host` and `rest-certs` option if we would otherwise fail. * meta: Update release checklist * clnrest: prefixing all rest config options with `cln` * doc: update decode instruction for emergency.recover * clnrest: add gevent ssl monkey patch to remove warning log * clnrest: don't convert *_msat fields to strings. * clnrest: don't try to handle plugin.rpc.call results. * pytest: adds test suite for clnrest plugin * poetry: add `requests` dependency for clnrest tests in contrib/pyln-testing * msggen: Add `Wait` method to managed API * msggen: Add `FetchInvoice` method to Rust and grpc bindings * splice: fix WIRE_HSMD_SIGN_SPLICE_TX capability check * doc: fix cln-grpc example. * CHANGELOG.md: Update for 23.11rc1 * reprobuild: Upgrade rust version * docker: Add fix for broken setuptools installation * Possible fix for CI_SERVER issue in VLS CI * dualfund: error on out of order sigs * dualfund: add test to make sure that tx-sigs sent before commitment * dualfund, test: add test for dropping to chain during RBF * dualfund, bump: when bumping a channel make sure it's in ok state * dualfund, memleak: don't leak the msg on error * dualfund, tests: break out "peer forgets" test * dual-fund: have accepter send their commitment sigs asap * dual-fund: don't re-notify plugin on arrival of sigs (2nd time) * tests: update opening tests for new reconnect behavior * mfc, nit: print out the error reason when an open fails * dualfund, reconnects: update dual-fund to use next-funding-id * dualfund, tx-abort: only check for abort state if we're sending * dualfund: update handling of tx-sigs * dualfund: handle commitment signed * dualfund, nit: make method for "their_role" * dualfund, cleanup: reuse code for verifying peer's commitment sigs * dualfund, cleanup: move common remote commit tx code into single place * dualfund, cleanup: make method for reporting channel state to HSMD * listpeerchannels: only add the scratch_txid if it exists * dualfund: when dropping to chain, only drop if we have a commitment tx * dualfund: report on whether or not we've gotten commitments * dualfund: on error, handle different states differently * dualfund: if we get an abort, clean up dangling inflights * openchannel_update: check if we've got an inflight record * dualfund: remove reliance on open_attempt on commit_received * dualfund: if we don't have commitments, error openchannel_signed * dualfund: wait til after we've sigs on disk before network check * dualfund: when updating an inflight, check for existing data * nit: spelling error (int -> in) * db, inflights: add method to remove any 'dangling' inflights * init channel: only fill in wscript if requested * dualfund: split 'commit-received' into two parts * dualfund: add switch for if the incoming channel is "too early" * wallet: allow inflights to have empty last_tx * inflights: split up adding sigs from making a new inflight * wallet: allow the channel to not have a last_tx * dual-fund: add new open-commit-ready state * lightningd: wumbo is now the default, setting has no effect. * ld: add commit-fee-offset option, update config schema * lightningd: add a feerate offset when updating feerates as opener * splice, vls: Fix missing check_mutual_channel_ready check. * splice, vls: fix missing rename in logging * add data field to RpcError * lightningd: rewrite anchor spend to use multiple UTXOs if needed. * pytest: detect RBF more reliably * lightningd: use wallet_utxo_boost for zero-fee htlc_tx. * wallet: routine to gather UTXOs to meet a certain feerate. * common: add amount_feerate helper. * wallet: specialize get_utxos interfaces. * pytest: force anchor tests to use more than one UTXO. * addpsbtoutput: allow command to specify output address. * doc: update clnrest documentation. * clnrest: code refactoring for: * Remove error from startup_regtest when loading wallet * gossipd: never regard our own updates as spam. * gossipd: don't blanket disable channels on startup. * cln-plugin: don't require redundant method reference * cln-rpc: Update the Makefile to always rebuild each example when the example files themselves are changed * cln-plugin: Add test for cln-plugin send_custom_notification * cln-plugin: implement registering and sending custom notifications to lightningd * splice: unwrap the old_secret during tx candidates * poetry: Add requests dependency * doc: document wait for new subsystems, add request & response schemas. * wait forwards: add in_htlc_id * pytest: add test for wait forwards. * lightningd: add ordering and pagination to listforwards. * listforwards: add created_index and updated_index fields. * forwards: hook into the wait subsystem. * lightningd: hook forwards into the wait system. * lightningd: split up peer_htlcs a little, create forwards.c * lightningd: add ordering and pagination to listsendpays. * listsendpays: add created_index and updated_index fields. * lightningd: don't log that we're trying to pay if we're not. * wallet: use standard-style iterators for payments. * lightningd: rename struct sendpay_command to waitsendpay_command. * wallet: don't expose wallet_payment constructor, use wallet_add_payment. * wallet: remove premature optimization when making payment. * sendpays: hook into the wait subsystem. * db: initialize variable to hold max payments id. * sendpays: add to wait subsystem. * tests/test_misc.py: Add test_decode to test the decryption of bech32 encoding of emergency.recover * tools/hsmtool.c: Add to hsmtools to get content of emergency.recover in bech32 format. * doc/lightning-decode.7: Update the doc * plugins/offers: Extend the capability of decode to decrypt the contents of emergency.recover file. * splice: Update test for new logs * channeld: use anchors on peer's commitment(s) if we can't broadcast our own. * lightningd: save peer's commitment anchor info into db. * lightningd: routines to deal with peer commitment anchor info in db. * channeld: tell lightningd about local anchor for each commitment tx. * channeld: have commit_tx tell is what output was the other's anchor. * channeld: rename channel_splice_txs to channel_txs. * channeld: get rid of unused commit_sig and htlc_sigs in channeld_sending_commitsig * pytest: test (fails!) that we push anchor on peer's commitment tx. * gci: Handle skipped tests correctly * pyln: Add missing upgrade-version target to pyln-grpc * pytest: Mark `test_plugin_startdir_lol` as `@unittest.skip()` * pytest: Mark test_exclude_adjacent_routehint as skip * pytest: Mark `test_splice` as falky * rs: Fix the manifests for publication * gci: Fix up the crates.io publishing job * pytest: Fix a falke in test_gossip_pruning * pytest: Skip `test_commit_crash_splice` as it is too flaky * cln-plugin: Make logging optional * add test, update schemas, fix warning_custom_fallbacks * display paid_outpoint on invoices when present * mark invoice as paid when we detect an on-chain payment to a fallback * lightningd: adapt invoice_try_pay declaration for onchain payment. * add autogenerated fallbacks (not custom fallbacks) to invoice_fallbacks table * track fallbacks for invoices, pt 1 * wallet: add invoice_fallbacks table and paid_outpoint to invoices * add invoices-onchain-fallback config option and warnings * bitcoin: expose fmt_bitcoin_txid * wallet: add newaddr_inner() helper * fallback: json add fallback cleanup * hsmd: implement the hsmd outpoint check * splice: Add hsmd_check_outpoint and hsmd_lock_outpoint * Splice: Better balance checking * doc: recover. * lightningd: implement `recover` command. * lightningd: allow --recover to take a 64-char hex string. * lightningd: use param_check on all commands which do extra checks. * lightningd: preapproveinvoice doesn't need to string lightning: prefix. * lightningd, pyln-testing: do extra checks to make sure check *cannot* write to db. * common: add new param_check() which doesn't abort if we're simply checking. * connectd: fix fd leak for --offline. * lightningd: add direct close outputs to listfunds (mutual close). * pytest: test that we can CPFP a mutual close. * test: Add test for datastoreusage * pyln: Add datastoreusage command * grpc: Add datastoreusage command * rpc: Add datastoreusage command * msggen: Add datastoreusage command * datastore: Add datastoreusage command * splice[FLAKE]: stale channel announcement fix * lightningd: fix crash when blocks come it fast. * pytest: fix flake in test_remote_addr_port * Restore any missing psbt metadata that resource constrained signers strip * lightningd: don't re-enter transaction if we have to call plugin_exclusive_loop. * lightningd: really fill in our own details when channeld says to make channel_update. * channeld/gossipd/lightningd: reverse polarity of channel_update 'disable_flag' * lightningd: make sure that channeld_local_channel_update fields are redundant. * channeld: don't attach channel_update to errors, let lightningd do it. * pyln-testing: fix LihgtningNode.openchannel() * switch from ToString to Display for ShortChannelId * pytest: fix flake in test_excluded_adjacent_routehint * pytest: fix flake in test_listforwards_and_listhtlcs * pytest: fix flake in test_even_sendcustommsg * lightningd: call finished callback *every* time we re-xmit a transaction. * lightningd: have broadcast_tx take a ctx. * lightningd: add context arg to bitcoind_sendrawtx() * lightningd: don't process request twice if plugin dies. * lightningd: handle properly if our own request to plugin is freed. * lightningd: simplify plugin hook handling. * lightningd: generically fail requests when plugin dies. * lightingnd: wrap all JSON reply callbacks in db transactions. * lightningd: make pending_request map inside plugin struct. * lightningd: fix --dev-debugger use with listconfigs * psbt: fix PSBT mutation in the changeset * fuzz: new bolt11 seeds * fuzz: add bolt11 cross-over mutator * fuzz: add custom cross-over functions * fuzz: set minimum bolt11 size to 9 * pytest: test for accepting even unknown messages. * lightningd: allow sending of even messages. * lightningd: tell connectd about the custom messages. * plugins: allow plugins to specify even messages they accept. * proto: update the bitstring dep * grpc: Regenerate proto stubs * grpc: Add missing feature attribute * doc: Remove redundant schema check * grpc: Add WaitBlockHeight method to grpc interface * rs: Implement a best-effort msat to sat conversion * rs: Fix up grpc conversions * rs: Bump crates cln-rpc and cln-grpc to 0.1.5 and cln-plugin to 0.1.6 * ci: use Bitcoin Core 25.1 * connectd: fix forwarding after tx_abort. * pytest: fix flake in test_excluded_adjacent_routehint * channeld: fix up compilation after conflicting changes. * libhsmd: removing assertion on push_value * splice: signer must be informed of splice params * hsmd: name the hsmd_ready_channel to hsmd_setup_channel * wallet: `db_get_runes` will only get rune from DB select * runes: bug fix for db failure to upgrade due to runes table alteration #6770 * connectd: use shutdown() not close() on TCP sockets for dev-disconnect. * lightningd: let channeld/dual_openingd send error to peer. * pytest: be sure that we receive error on datalose failure. * channeld: don't hang up immediately after sending bad reestablish warning. * lightningd: close channel ourselves, if we receive an error. * pyln-testing: make disconnects continue after restart. * common: move is_peer_error/is_peer_warning from read_peer_msg.c to wire_error.c * pytest: revert warning delivery failure workaround * lightningd: pass disconnect flag to subd's errcb. * common: add peer_failed_warn_nodisconnect routine for non-disconnecting warnings * connectd: don't disconnect automatically on sending warning or error. * connectd: give subds a chance to drain when lightningd says to disconnect. * lightningd: stop all subds when we want to disconnect. * tools: make sure we don't use optimization when building mocks. * Makefile: fix update-mocks dependencies. * connectd: Ignore sodium fd for Mac tests * bolt11: don't abort on invalid pubkey * bolt11: validate recovery ID * fuzz: check invoice signatures * bolt11: return error on unexpected lightning prefix * github: re-enable ASan and UBSan for check-fuzz * bolt11: avoid reading uninitialized memory * bolt11: check return value of pull_all * lightningd: simplify lease update blockheight code. * lightningd: fix crash caused by sending channeld message to closingd * Revert "runes: fix `checkrune` when `method` parameter is the empty string." * clnrest: correctly omit `method` to checkrune call if none is supplied. * pytest: correctly test missing method for runes. * pytests: make pay helper able to route (optionaly) * channeld: comment typo fix * build: fix compilation issue on OpenBSD * doc: fix display of lists and code blocks in manpages. * lightningd: print the version of cln early * cln-grpc: do not generate cerficates when no autostart. * pytest: test that cln-grpc certificates are not generated when cln-grpc is not started. * runes: fix `checkrune` when `method` parameter is the empty string. * pytest: test `checkrune` when `method` parameter is the empty string. * fuzz: bolt11 inputs * fuzz: test bolt11 decoding * contrib: add `developer` option to config file in startup_regtest.sh. * pytest: wean many tests off the assumption that listchannels shows private channels. * pytest: remove old test_tlv_or_legacy * renepay: work around change in fundchannel in tests. * plugins/topology: convert listpeerchannels into local overlay. * plugins/topology: split getroute logic. * plugins/topology: remove unused fuzz factor. * channeld: fix memleak where tx gets leaked. * channeld: fix memleak when inflights is NULL. * splice: minor code cleanups * addpsbtoutput: New onchain command for PSBT output * common/memleak: show tal_steal operations on memleak. * common/memleak: implement callback arg for dump_memleak. * lightningd: fix assertion when funding depth changes fast. * db: removed deprecated `runes_uniqueid` from `vars` table * runes: Runes will use unique id returned from `runes` table not `runes_uniqueid` from `vars` * runes: Get next unique id from runes table * runes: Reimplement `rate` in terms of `per` * pytest: trim mostly-reundant commando-rune test. * tools: Fixed release verify typo * lightningd: simplify funding_depth_cb now it only handles main funding tx. * lightningd: fix dual-funding case where we coop close and an RBF confirms. * lightningd: ensure we *always* watch channel spend. * lightningd: fix bug where we didn't correctly change outpoint of splice scid. * lightningd: don't share funding_depth_cb for non-funding txs. * lightningd: fix watch on existing tx. * lightningd: make watch_txid more generic. * lightningd: remove watch_tx() in favor of watch_txid(). * lightningd: make channel-query functions all take state. * lightningd: split DUALOPEND_OPEN_INIT into DUALOPEND_OPEN_INIT and DUALOPEND_OPEN_COMMITTED. * doc: introduce new state DUALOPEND_OPEN_COMMITTED. * common/json_stream, lightningd/notification: clean up function APIs * lightningd/channel.h: rename channel_unsaved to the more explicit channel_state_uncommitted. * wallet: add standard sanity-check function for channel_state. * lightningd: remove peer_any_unsaved_channel and use peer_any_channel. * lightningd/channel.h: clean up channel states. * lightningd: generalize peer_any_active_channel to peer_any_channel. * lightningd: fold funding tx depth into a single function. * lightningd: make dualopen_tell_depth match channeld_tell_depth. * lightningd: don't report original depth once splice started. * lightningd: clean up channel_tell_depth. * lightningd: disconnect on *any* transient error, except abort * pytest: make test_splice_gossip more precise. * doc: fix listpeerchannels schema to allow CHANNELD_AWAITING_SPLICE in state. * patch remove-developer-test-annotations.patch * pytest: fix flake in upfront warning. * contrib/pyln-grpc-proto/ regenerate. * poetry: run poetry update. * plugin/clnrest: Adjusting flask-cors install after rebase * lnproto: Remove unused report upload on GH Actions * lnproto: Fix up the lnprototest docker image * py: Add the grpc extra to pyln-testing dependency * gen: Update generated files * plugins/clnrest: Updated clnrest's requirements.txt * plugins/clnrest: Updated clnrest install command in Docker images and CI * plugins/clnrest: Adding pyproject.toml for Poetry install * lnprototest: bump to the new version * ci: autopublish rust crates on tag event * rust: fixes the dependencies build with the rustc 1.73.0 * Makefile: use `xargs -0` consistently * pytest: Fix elementsd tests * gci: Update apt before installing * gci: Print envvars and configuration when testing * gci: Only install one of elementsd or bitcoind * ci: Unbreak the liquid-regtest run on Github Actions * Only 3 question marks for 4 variables * renepay: remove custom debug, in favor of normal operations. * lightningd/test: use tmpdir_mkstemp() to create log file * configure: recognize --disable-ub-sanitizer option * lnprototest: included the last lnprototest fixes * pyln-grpc-proto: regenerate after last patch. * add CHANNELD_AWAITING_SPLICE state to cln-rpc * Makefile: don't fail find -delete if we run in parallel. * build: remove all trace of DEVELOPER. * pytest: use --developer instead of environment variable. * tools: remove DEVELOPER. * plugins/renepay: remove DEVELOPER build-time, in favor of runtime. * lightningd: remove #if DEVELOPER. * plugins: remove #if DEVELOPER. * common: add option for dev-only parameters. * lightningd: mark explicitly what commands are dev-only. * db: remove #if DEVELOPER in favor of runtime flag inside db struct. * closingd: remove #if DEVELOPER in favor of runtime flag. * hsmd: remove #if DEVELOPER in favor of runtime flag. * gossipd: remove #if DEVELOPER in favor of runtime flag. * channeld: remove #if DEVELOPER in favor of runtime flag. * openingd: remove #if DEVELOPER in favor of runtime flag. * onchaind: remove #if DEVELOPER. * connectd: remove #if DEVELOPER * bitcoin: remove DEVELOPER options. * common: remove #if DEVELOPER. * common/bolt11: remove #if DEVELOPER. * common/daemon.c: remove #ifdef DEVELOPER in favor of runtime flag. * lightningd: pass --developer down to plugins, subdaemons when it set. * lightningd: add --developer runtime option. * lightningd: fix bogus unused var warning. * lnprototest: update to branch which supports --developer * wallet/test: regenerate mocks to fix build. * splice: Fixes from splice-out test * tests/runes: per restriction test cases * lightningd/runes: Added last_time_check for validating rune for `per` restriction * common: make json_add_timeabs full precision & renamed json_add_time * lightning/runes: added last_used in showrunes * lightningd/runes: update last_used timestamp when a rune successfully used * wallet: add last_used_nsec in runes table * common: add test to check bolt12/offers-test.json test vectors. * common/bolt12: roll all necessary checks into offer_decode. * common: helper to generate bolt12 test vector bolt12/offers-test.json. * bolt12-cli: add encode routine to make bolt12 vectors. * wallet: re-run update-mocks. * docker: Deleting redundant arm64v8 and arm32v7 Dockerfiles * script: Updated build-release script * script: Fixed bitcoin and litecoin archs used with buildx * doc: update to BOLT 6e85df448bfee7d10f26aabb06b8eba3d7505888 "BOLT2: change "reject" to "fail" in channel opening requirements (#1104)" * bolt11: update to BOLT 7dda8f84edb8655766d83619cb41110c1bf43f85 "Bolt11: min_final_cltv_expiry_delta is optional, not required (#1100)" * wire: update to BOLT 17ceba42dc6607b637a713a9bc051f63c1357edd "BOLT 4: onion message support." * doc: update to BOLT cded2df1fd0095c947af02a52e28b16c66bdfc5c "Fix undeclared reference in onion-routing" * lightningd: update to BOLT 7d3ef5a6b20eb84982ea2bfc029497082adf20d8 "Allow unset `onion_hash` in `invalid_onion_blinding` (#1093)" * doc: update to BOLT 7f53a3e46ed59f19463447d5fa63b3e484c708a5 "BOLT 8: make it clear that there are *two* chaining keys." * wire: update to BOLT aad959a297ff66946effb165518143be15777dd6 "tools: remove option handling now last optional field was removed." * doc: update to BOLT 3747ba83022cd385093df2696ed342f1e41e31b3 "Remove requirements to disconnect on warnings" * connectd: also don't disconnect on "all-channel" warnings. * common: don't hang up when we receive a warning. * common: don't send channel_id on peer error. * common: remove channel_id check. * doc: update to BOLT 50b2df24a27879e8329712c275db78876fd022fe "Update onion errors since we allow overpaying or under-CLTVing" * doc: update to BOLT 29c14c6e12cbdf33f6b724094c81658a614d2e02 "fix typos" * common: rename tlv_payload's "blinding_point" to "current_blinding_point" as per spec. * wire: fix up messy patches (make extract-onion-csv) * wire: fix up messy patches (make extract-peer-csv) * docker: make available bitcoind binary * docs: Update documents with `rest-csp` option * plugins/clnrest: Update content security policy for server * plugins/clnrest: Add `rest-csp` option for config * plugins/clnrest: Update Websocket server to dynamically add CORS settings * docs: Update documents with `rest-cors-origins` option * plugins/clnrest: Update App server to dynamically add CORS settings * plugins/clnrest: Add `rest-cors-origins` option for config * plugins/clnrest: Add CORS library * plugins/clnrest: Updated websocket client examples * plugins/clnrest: Rune authentication for websocket server * plugins/clnrest: Generate all three certs if missing * Remove GMP from arm64v8 build * Remove deprecated GMP dependency * update the app dev interface img * fix the rest api reference link * fix paths for doc files and make them public * pytest: fix renepay test_shadow flake. * fuzz: initial wire message seeds * fuzz: target for your_peer_storage * fuzz: target for update_blockheight * fuzz: target for tx_signatures * fuzz: target for tx_remove_output * fuzz: target for tx_remove_input * fuzz: target for tx_init_rbf * fuzz: target for tx_complete * fuzz: target for tx_add_output * fuzz: target for tx_add_input * fuzz: target for tx_ack_rbf * fuzz: target for tx_abort * fuzz: target for stfu * fuzz: target for splice_locked * fuzz: target for splice_ack * fuzz: target for splice * fuzz: target for peer_storage * fuzz: target for open_channel2 * fuzz: target for onion_message * fuzz: target for accept_channel2 * fuzz: target for warning * fuzz: target for update_fulfill_htlc * fuzz: target for update_fee * fuzz: target for update_fail_malformed_htlc * fuzz: target for update_fail_htlc * fuzz: target for update_add_htlc * fuzz: target for shutdown * fuzz: target for revoke_and_ack * fuzz: target for reply_short_channel_ids_end * fuzz: target for reply_channel_range * fuzz: target for query_short_channel_ids * fuzz: target for query_channel_range * fuzz: target for pong * fuzz: target for ping * fuzz: target for open_channel * fuzz: target for node_announcement * fuzz: target for init * fuzz: target for gossip_timestamp_filter * fuzz: target for funding_signed * fuzz: target for funding_created * fuzz: target for error * fuzz: target for commitment_signed * fuzz: target for closing_signed * fuzz: target for channel_update * fuzz: target for channel_reestablish * fuzz: target for channel_ready * fuzz: target for channel_announcement * common: disable node ID check when fuzzing * configure: define macro for fuzzing builds * fuzz: target for announcement_signatures * fuzz: target for accept_channel * wire: correct argument order * Makefile: don't delete python-generated files in distclean. * gci: Update apt cache before installing * configure: typo fix * gci: Fix wrong guard on rebase step * ci: Add a server to track flakyness in pytest tests ------------------------------------------------------------------- Wed Sep 13 20:06:49 UTC 2023 - marec@detebe.org - Update to version 23.8.1: * CHANGELOG.md: update for v23.08.1 release * Makefile: regnerate config.vars if configure changes (recently PYTHON var). * wallet: slightly neaten fundpsbt code. * wallet: Exclude uneconomical UTXOs from fundchannel * tests: test for fundchannel fail with tiny utxo * tools: Make shellcheck happy * tools: Fix accidental override on checksum copy * tools: Add sha256sum check for detailed hint * tools: Add check if the checksum file exists * plugin/clnrest: Deleting redundant README.md * doc/rest: add documentation for REST interface * runes: fix weird unnecessary zero-test. * runes: ensure that uniqueid is a valid number. * clnrest: don't require nodeid (FIXME: needs screenshot updates!) * lightningd: refuse to create rune with empty fields. * checkrune: make nodeid and method optional. * doc: point checkrune at createrune for rune field details. * doc: clean up checkrune man page. * doc: fix checkrune request schema. * doc: update to reflect new bitcoind requirement * doc: Updated instructions for signing, co-signing and Docker publishing * script: docker build * docker: Install tini in final stage * add Eq, PartialOrd, Ord, Hash to ShortChannelId * Dockerfiles must not invalidate the docker cache * doc: Updated old links and deleted old file * doc: Updated Backup and Recovery * schema: Fix wrong type of CLTV in route * doc: add tip on extracting binary release * plugin/clnrest: do not read json payload if data length is zero * renepay: fix case where shadow increases amount past payment amount. * renepay: fix shadow route extension logic. * pytest: test for shadow route additions. * renepay: calculate fee and delay accumulation correctly. * pytest: write test to ensure renepay the correct channel's fee/cltv info. * renepay: don't crash if we get failure from sendpay. * db: migrate old runes table to fix up id fields. * runes: insert rune with correct id field. * pytest: test to show that we got db ids incorrect. * configure: don't assume python3, create $PYTHON var. * rs: Bump crate versions before publishing * Use python3 instead of python in Makefile * gci: Do not try to rebase if we're on master already * clnrest: Change process method to fork ------------------------------------------------------------------- Wed Aug 23 08:27:04 UTC 2023 - marec@detebe.org - Update to version 23.8.0: * CHANGELOG.md: update for 23.08 release final. * renepay: hack in workaround spendable reduction for release. * plugins/renepay: don't drop min to 0 if it's shown wrong, but drop 50%. * renepay: remove remaining calls to debug_paynote in favor of payflow_note(). * plugins/renepay: convenience routing to log when we disable a chan. * plugins/renepay: neaten the command notifications. * renepay: clarify chan_extra_can_send. * plugins/renepay: use struct short_channel_id_dir. * plugins/renepay: clean up unused fmt_payflows. * pay_flow: downgrade messages about shadow route delays. * plugins/renepay: Add infra for per-flow notes. * announce-addr not announce only * docs: update the details of emergency.recover and --recovery for users. * docker: bookworm image for arm32v7 * pytest: fix plugin_connected_hook_chaining flake. * lightningd: don't restart subds unnecessarily on reconnect. * pytest: fix autoclean test flake * plugins/renepay: don't free waiting-for-addgossip flow! * meta: ignore renepay binary * CHANGELOG.md: Update for -rc3 * openingd: Clean up channel_type_accept * openingd: Reject zeroconf if the peer is not allowed * openingd: Tentatively accept zeroconf when proposed * renepay: reduce minimum "known" if it proves incorrect. * CI: add clnrest requirements so --help lists rest params. * lightningd: documentation updates. * plugins/sql: make testing-only sqlfilename option `dev-sqlfilename`. * lightningd: don't print plugin deprecated options in --help. * doc: document lightning-cli options. * tools/check-manpage.sh: update undoc to a simple list. * build: have check-manpage.sh ignore options documented as deprecated. * build: fix check-manpage.sh * listchannels: also list splicing channels as active: true. * pay: don't filter out local channels which are splicing. * renepay: splicing channels are also usable for payments. * gossmap: fix false valgrind uninitialized error on arm64, ppc. * pay: don't discard high-htlc_min channels when searching for routehint starts. * pytest: test that we can pay via routehints even if a 1000-min-htlc channel is in path. * lightningd: clean up properly if we fail to exec plugin. * pytest: test that failing to start a plugin due to failure to exec does not put it in plugins list. * pytest: test (failing) for not crashing if plugin fails to exec via `plugin startdir` * Don't restart my machine if I add code to kill a failed plugin. * splice: prevent splice going to onchaind & race prevention * nixos install: tweaks to get CLN building from fresh start * doc: Drive-by fixes of JSON examples in hooks.md * pytest: add no argument cli test * cli: don't crash when there's no argument * splice: quieter in more cases * pay: still use channels for routehints even if peer says it's disabled. * common/test: fix gcc-12.3.0 -O3 warning. * common: fix gcc-12.3.0 -O3 warning. * plugins/renepay: fix gcc-12.3.0 -O3 warning. * plugins: fix compile warning with some gcc versions and -Og * channeld: fix gcc-12.3.0 -O3 warning. * CI: add -O3 to one of the builds, to test for errors. * plugins/renepay/test: fix access-after-free. * renepay: don't free adg twice after adding gossip. * plugins/renepay: don't require raw_message in error reply. * splice: STFU -> disable announce and commit timers * Release candidate rc2. * closingd: fix case where we we can pass under min-relay-fee for mutual close. * lightingd: fix minfee when we're ignoring limits. * pytest: test for mutual close fail under very low feerates. * Pypi: publish to real repository. * renepay: add command notifications * renepay: trivial cleanup to rename `flow` to `pf` everywhere. * renepay: do less work in destroy_pay_flow, and reorder pay_flow.c * renepay: add dummy pf_resuly type to ensure we deal with the flow. * renepay: drive *all* progress from termination of struct pay_flow. * renepay: grab update from WIRE_TEMPORARY_CHANNEL_FAILURE if present. * renepay: make pay_plugin a tal object. * renepay: fix up handling of errors from final node. * renepay: remove always-true "first_time" and "unlikely_ok" flags. * renepay: remove unused result member. * renepay: put the entire hash in the key struct. * renepay: don't re-parse bolt11 to get routehints. * renepay: simplify JSON handling in notification_sendpay_success. * renepay: simplify JSON handling. * renepay: get max group_id in single iteration. * renepay: merge `struct renepay` and `struct payment` into one. * renepay: make memleak simpler. * renepay: move list_node to first member of struct payment. * renepay: use more formal allocator pattern. * renepay: fix localmods. * renepay: remove attempt limit. * renepay: remove unused all_flows field. * gossipd: load pending spam node announcements without forgetting previous node announcements * pytest: test gossip_store loading of spam node announcement before channel update * gossipd: reduce gossip self-advertizement if we have many peers/channels. * common: add tal_arr_randomize() to shuffle an array. * gossipd: ignore redundant node_announcement in gossip_store. * common: restore OPT_SPLICE definition for the day it's ratified. * splice: Move splice to experimental feature bit * plugins: fix error report from bitcoin-cli exec failure. * renepay: add help for renepay & renepaystatus * pytest: run splicing test on every run. * hsmd: fix capability check for signing splices. * hsmd: rename "capabilities" flags for hsm fds to "permissions" * Fix: Remove Sync requirements on Futures returned in the Rust plugin library. * doc: Fix typo in the description of fields `private` * lightningd: use fsync not fdatasync. * fix flake8 errors in the script file (#6528) * lightningd: fix bolt11 parsing in preapproveinvocie, sendonion, listsendpays and renepay * pytest: test for listsendpays and lightning: prefix crash. * lightningd: close connection when HTLC addition times out. * pytest: test that we close connection if adding an HTLC times out. * renepay: allow it to die gracefully without crashing lightningd. * plugins/bcli: update minimum required bitcoind version. * docker: bitcoin and elements version update * channeld: don't send splice TLV fields unless negotiated. * meta: Add the VLS team as codeowners of hsmd/hsmd_wire.csv * meta: Remove ZmnSCPxj from codeowners * splicing: Update documentation * docs: try to fix up readthedocs.io, so it has the latest man pages, but points to docs.corelighting.org for the rest * random doc changes to test. * add workflow to sync rpc commands * add script to sync rpc commands * build: more 32-bit printf fixes. * lightningd: don't return to a default filter level if there are no per-file filters. * pytest: test for filtering bug. * doc: make staticbackup output autogenerated in man page. * msggen: Add `staticbackup` call to list of managed methods * doc: The SCB fields are hex-encoded binary TLVs, not strings * docker: Install clnrest dependencies from rquirements.txt * plugin/clnrest: Downgrading pyln-proto versino from 23.08 to 23.5.2 * splicing: Remove dependency on experimental_dual_fund * makefile: py plugins installed twice with `make install` * renepay: Use macros instead of manual format types * wallet: don't BROKEN log id payments.total_msat is null. * doc: give an example for multifundchannel * channeld: replace `struct splice` with `struct splicing` for OpenBSD. * CHANGELOG: Update. * pyln-client, pyln-proto, pyln-testing: update to new versions. * codex32: use "cl" instead of "ms" as our HRP. * renepay: clean up ARC_ADDITIONAL_BITS. * renepay: switch from arc_t to struct arc. * renepay: implement an alternative to a union with bitsets. * CCAN: update for base64 compile fix on ARM. * docs: document hsmtool getcodexsecret, refer to it for recover. * hsmtool: remove threshold arg. * hsmtool: tighten checks for validity on getcodexsecret * lightningd: pass signed tx through to close callback. * lightningd: clean up signing of last_tx. * pytest: test for bad 'tx' result from close. * Fixing env reference for API key and path to doc (#6478) * fix readme workflow (yet again) and test a dummy change (#6477) * fix workflow file and test changes (#6476) * docker: Dockerfile aarch images with pyln and clnrest * docker: Dockerfile amd64 with pyln and clnrest * docker: fedora builder update * tools: build release docker script update * plugin/clnrest: Updated pyln version to 23.08 in requirements.txt * CI: Try not running group 2/10 UBSAN in parallel. * signed types: integer fix * inflights: use ctx for making new ones * contrib/startup: ignore db-updates * channeld: Code to implement splicing * interactive-tx: Renaming for clarity and cleaning up * splicing: add DB details for splice HTLCs * psbt: Fix psbt linseraize_input bug * splicing: Add channel state AWAITING_SPLICE * psbt: routine for lining up sigantures in a witness stack * hsmd: Add sign splice command * Mac: Change format type to use PRIu64 instead of %ld * Mac: Update Makefile to work on Mac zsh * tests/test_misc.py: Add a subprocess to make sure getcodexsecret fetches out the hsm_secret in bip93 format. * tools/hsmtool: Add getcodexsecret to get the hsm_secret in bip93 * common/codex32: Add codex32_secret_encode to encode seeds in bip93 format. * channeld: don't backtrace if master closes during sync read. * wallet: don't load indexes inside outpointfilters_init trace span. * lightningd: fix crash on startup expirations. * pytest: add test for expiration-on-startup crash. * pytest: correctly annotate invoice tests needing sqlite3. * devtools/credit: ! not ^ to invert the match set in sh. * use of PATH_TO_BITCOIN in regtest * pytest: fix the invoice typo migration if you ran and created both. * pytest: test for grubles' screwed node. * renepay: add test * renepay: more cleanups - adopt "const <type> *"convention - remove use_shadow option for some pyln tests - show prob. information of flows into paynotes - show prob. of success of entire payment flow in paynotes - minflow: We were not releasing the memory of flow arrays when replacing them with a new canditate. - use memleak_scan_obj in memleak_check - replace u64 with size_t * plugins/renepay/dijkstra: improve API to remove global. * renepay: fixups after comments * renepay: add information decay linear in time * renepay: an experimental payment plugin * Import lqueue from CCAN * codex32: rework. * codex32: document the complete representations from BIP, adjust numbers. * codex32: minor cleanups. * doc: Add documentation for --recover flag. * lightningd: Added plugin hook to notify whenever started in rocover mode. * lightningd/options: Always start in offline mode when run using --recover flag. * tests: Added test_recover to test --recover flag. * lightningd/common: Added --recover flag to populate HSM_secret with appropriate secret through a valid codex32 secret. * common/test: Added run-codex32 to test codex32.c decoding function. * common/codex32: Added checksum verification for codex32 encoding. * common/codex32: Added codex32_decode() without checksum verification. * plugin/clnrest: Improved readme and comments * plugin/clnrest: Updating package * dual-fund: keep track of aborted requests, seamlessly restart daemon * dual-fund tests: add tests for reported incompat with Eclair * funder: don't re-reserve utxos on retries * gossipd: set timer for node_announcement regen * gossipd: actually update own node announcement if needed * added frontmatter and fix github workflow * gossipd: tell lightningd about all our previous channel_updates at startup. * lightningd: don't crash if peer manages to spend onchain HTLC after we've abandoned upstream. * pay: Fix a null-dereference introduced in #6428 * pytest: follow RBF txids properly, but ignoring identical "RBF" txs. * pytest: fix flake in penalty tests * build: avoid printing PLUGIN_BASES on every build. * pytest: fix flake in test_gossip_store_compact_on_load. * update dead links * doc: Add documentation for the tracing subsystem * trace: Use User Statically-Defined Tracepoints to emit tracing spans * trace: Instrument topology functions * trace: Instrument the wallet startup functions * trace: Instrument lightningd startup sequence * trace: First draft of API * config: Detect presence of USDT header in configure * SECURITY.md: fix email address * wallet: fix migration of existing invoices's id variable. * pytest: add test for db crash on conflicting invoice.id * msggen: Add two missing fields to the `listpays` schema * pytest: add reckless local install test * reckless: add installation capability for additional sources * reckless: style cleanups, type hints, simplify create_dir * gossipd: handle upgrade from version 11 gossip_store * pay: Fix a non-rebased PR merge error * lightningd: fail incoming HTLCs if peer would close channel. * onchaind: keep trying to collect HTLC even if we're past timeout. * pytest: test that we proactively close incoming HTLCs to avoid them closing on us. * plugins/bcli: plug temporary leak on retry. * plugins/bcli: fix leak report when bitcoind goes away. * ci: Update lnprototest * channeld: Verify the signature sent by the counterparty * libplugin/pay: allow shortcut for self-pay. * lightningd: sendpay implement zero-length path == self-pay. * pay: generalize payment_succeeded(). * invoice: return error string from invoice_check_payment. * wallet: implement and use wallet_payment_new. * lightningd: extract check_progress() from send_payment_core(). * db/bindings: allow db_col_short_channel_id_arr and db_col_node_id_arr on null columns * common/utils: add tal_strdup_or_null helper. * db: add optional column string helper, and make db_col amounts return sat/msat. * pytest: test self-payments using sendpay. * bolt11: Don't push the size of the witness program for v1+ scripts * plugins/clnrest: Websocket Server * plugins/clnrest: Update clnrest dependencies in scripts and Dockerfiles * plugins/clnrest: removing notifications queue in favor of websocket server * plugins/clnrest: disable ourselves unless rest-port is set. * plugin: On-boarding default python plugins * lightningd: if a plugin disables at getmanfest time, use INFO log level. * plugins/clnrest: fail gracefully if dependencies not installed. * doc: update installation to not refer to PPA, and indicate how to get clnrest support. * plugin: clnrest * docs: Incorrect params sequence for checkrune command * lightningd: clean up commando rune_counter from datastore on migration. * lightningd: migrate (and delete) old commando blacklists. * lightningd: migrate (and delete) old commando runes. * lightningd: simplify datastore internal db API. * wallet: internal exposure to db accessors. * lightningd: initialize runes way earlier, and add rune_is_ours helper. * commando: pass through commands to lightningd. * commando: remove now-unused internal checking routines. * commando: use CLN's checkrune() instead of our own for for rune validation. * plugins/commando: make struct cond_info self-contained, split try_command() * plugins/commando: try_command can reuse incoming `struct commando`. * plugins/commando: create new_commando() allocator, use it. * commando: always active. * common: fix wrong comment on json_scan() return. * pytest: use native l1 commando secret. * common/json_param: use assert() for param correctness checks. * runes: fix `stored` field. * runes: Renamed `listrunes` to `showrunes` * runes: create and blacklist commands aliases * makefile: use recursive delete to target extraneous .baks * fix: dont send blockheight updates if we're still syncing * gossipd: don't "unmark" dying channels' updates if we receive them. * lightningd: fix false memleak report (test flake)! * devtools/print_wire: long long instead of longing * channeld: don't close connection on invalid badonion code (LND bug?) * ci: Update the upload-artifact step to v3 * added listhtlcs model * pay: Consider amounts when selecting a channel to send over * pay: Annotate both local alias and real scid with channel hints * lightningd: expose created_index and updated_index fields. * listinvoices: add limit param. * listinvoices: add index and start params. * invoice: include invoice label to wait subsystem when it's paid. * lightningd: trigger changed wait when delinvoice desconly used. * invoices: pass through info to delete function, call invoice_index_deleted. * wallet: hoist gathering of expired_ids. * wallet: hook up updated_index for invoices. * wallet: hook up created_index for invoices. * invoices: routines to maintain invoice indices. * db: infrastructure to initialize indexes. * wait: new command to wait on indexes. * Swap out doc links with link to new doc website * parital eq for channel state enum * pytest: test for setchannel's new ignorefeelimits. * wallet: persist the ignore_fee_limits bit for channels.4 * lightningd: `setchannel` can set `ignorefeelimits`. * lightningd: remove now-unused flag in set_channel_config. * lightningd: expose ignore_fee_limits field in listpeerchannels. * lightningd: don't override fee limits in feerate_min/max, do so in callers. * lightningd: put ignore_fee_limits bool into channel struct. * build(deps): bump cryptography from 41.0.1 to 41.0.2 * pytest: make tor-requiring tests clearly distinguishable * Makefile: check-source-no-cppcheck. * pytest: fix timeout in test_sql * pytest: fix flake handling intest_restorefrompeer. * connectd: fix memleak report. * pytest: fix timeout in test_channel_lease_unilat_closes * pytest: fix test_anchor_min_emergency "bad gossip" messages. * connectd: fix transient memleak report. * pytest: fix race in test_gossip_not_dying * lightningd: avoid notleak markers: move global into struct runes. * docs: document all the rune commands which look like the now-deprecated commando ones. * lightningd: implement `blacklistrune` command. * lightningd: implement `checkrune` command. * pytest: test for runes, based on commando tests. * lightningd: implement `createrune` command. * lightningd: implement listrunes command. * lightningd: new runes infrastructure. * db: add runes tables and accessors. * pyln: Make the grpcio dependencies optional * libplugin: Remove presplitter from the API * pay: Remove the presplitter modifier * pytest: Remove tests that rely on presplit behavior * gossipd: aggressively advertize *both* sides of channel_update. * gossipd: when we dump our own gossip, include our node_announcement. * pytest: test for force-broadcasting our own gossip. * gossipd: clean up dump_our_gossip. * connectd: don't gossip dying channels. * pytest: test for whether we gossip spent channels. * gossipd: add dying marker to channel_announcement/channel_update. * gossip_store: add "dying" flag to indicate not to gossip dying channels. * lightningd: updating python v3.7 to v3.8 * lightningd: removing bionic and python 3.7 support * lightningd: fix up log filtering for generic daemons (e.g. connectd). * lightningd: extend log-level to support filter per-file. * lightningd: add support for per-logfile filters. * lightningd: clean up logging initializations. * lightningd: keep linked list of all loggers. * lightningd: have opt_log_level take the log_book. * lightningd: add "has_io_logging" helper. * lightningd/log: clean up nomenclature. * wallet: Fixes the compilation error due the db_binindings change * commando: added check for empty params * commando: changed listrunes schema for `restrictions` `english` key * db/bindings: now bindings are always in order, remove index. * bkpr: use BIND_NEXT. * wallet/invoices.c: use BIND_NEXT * wallet: convert to BIND_NEXT. * db: implement BIND_NEXT for autocounting. * db: enforce that bindings be done in order. * wallet: use standard-style iterators for invoices. * wallet/invoice: remove indirection. * wallet: rename enum wallet_payment_status to payment_status. * invoice: expose invoice_status_str function. * invoices: keep wallet pointer. * docs: fix up internal references now files have moved. * update workflow to sync readme files * move old misc file to another folder * remove mkdocs and sphinx files * move old misc files to another folder * remove legacy docs * update new documentation and move to doc/ * pytest: fix flake in test_feerates * pytest: fix flake in tests/test_closing.py::test_penalty_rbf_normal[True] * pytest: test for bad estimatefees results. * lightningd: don't crash when we get a bogus estimatefees subfield. * fix(db): normalize all the invoice string stored in the database * fix(jsonrpc): trim the lightning: prefix from invoice * dual-fund: send the next_funding_txid to peer * spec: update to latest dual-funding wire changes * varint: make helper public * signed types: add handlers for signed types * plugins: fix shutdown notification to contain object. * libplugin: support wildcard subscriptions. * pyln: add ability to subscribe to all notifications. * plugins: add support for wildcard subscription. * mac-fix: `sed` doesn't work the same on macos * ci: Fix flakiness of lnprototest on CI * feat: increase the lnprototest version * hsmtool: Add support for Signet network * build: Regenerate the generated file as the default target * Support p2tr deposit addresses * cleanup comments, variables with respect to bech32(m) * txfilter_add_derkey: track p2tr scripts * is_segwit_output: properly detect all known segwit script types * encode_scriptpubkey_to_addr: support p2tr * p2tr script utilities * psbt_input_add_pubkey: support taproot * lightnind: cleanup not necessary logs * pyln-proto: make point release. * pyln-client: make point release to prepare for next release. * CHANGELOG.md: import changes from 23.05.2 branch. * cln-plugin: Remove unused `nonnumericids` field in `Builder` * cln-rpc: Allow access to deprecated fields in conversions * cln-rpc: Generate ToString for Enums * msggen: Derive `Eq` on generated Enums as well * rs: derive PartialEq for simple rust enums * pytest: fix flake in test_closing_anchorspend_htlc_tx_rbf * pytest: fix flake in test_routing_gossip. * bcli: don't feed CLN massive feerates. * Makefile: rebuild ccan when ccan_compat.h changes. * lightningd: fix rebase clash. * log: increase the log for ignore fee to unsual * This commit fixes a bug introduced in 64b1ddd761de30152154714433974b72bfb7f278 that does not ignore the min fee as specified by the user setting. * fuzz: initial sha256 seeds * fuzz: initial ripemd160 seeds * fuzz: add sha256 target * fuzz: add ripemd160 target * ccan: rename ripemd160 functions * lightningd: actually deprecate old notification fields now tests and plugins are all updated. * plugins: update for new deprecation API where all fields are nicely wrapped. * tests: notification response fixes * doc: Typo in balance_snapshot notification object * notification: block_added json key updated from block to block_added * notification: disconnect with json key disconnect * notification: connect with json key connect * docs: remove the ppa reference from the docs * fix(docs): Fix instructions for installing cln on FreeBSD * cln-rpc: Remove wildcard import from model * ci: Add pyln-grpc-proto to auto-publication workflow * pyln: Create a dedicated pyln-grpc-proto package * doc: Updated instructions * build: updated build-release script * build: passing release version for make * docker: cl-repro Dockerfile update * docker: Core lightning Dockerfile updates * gitignore: Ignore SHA256SUMS from release verify * pytest: fix flake in test_onchain_middleman_simple with anchors. * Makefile: add dependencies to fix parallel build. * CI: use prebuild for (almost all) ci steps. * CI: clean up workflow files. * configure: --enable-debugbuild flag for debug builds. * lightningd: remove deprecated_apis global, put into lightningd. * common/utils: remove now-unused softref facility. * gossipd: don't use softref in the seeker. * gossipd: don't use peer softrefs for gossip credit. * gossipd: put ->daemon pointer into routing_state. * gossipd: use htable, not linked list for peers. * connectd: don't try to keep going if gossipd dies. * db: add generic warn/error function. * libplugin: expose plugin_logv. * Remove --daemon from lightningd.service * contrib: fix id parsing in IO loop of cowsay.sh plugin * pytest: test for emergency minimum fund violations. * plugins: don't let multifundchannel open a new anchor channel without reserves. * lightningd: add option `opening_anchor_channel` to fundpsbt, utxopsbt. * lightningd: don't let them fundpsbt below emergency reserve. * config: add min-emergency-msat option. * multifundchannel: use `excess_as_change` flag to simplify logic. * txprepare: use `excess_as_change` flag to simplify withdraw/txprepare logic. * fundpsbt/utxopsbt: handle `excess_as_change` and `all` correctly. * plugin: make sure multifundchannel / fundchannel unreserve correctly on failure. * pytest: adapt all the anchor-iff-EXPERIMENTAL tests to --experimental-anchors. * pytest: make dual-funding tests using option_anchors_zero_fee_htlc_tx. * pytest: test CPFP using anchors. * lightningd: turn on experimental support for zero-fee htlc anchors. * global: thread zero fee option everywhere. * common/channel_type: add zero-fee-anchor type. * onchaind: don't grind for htlc fees if option_anchors_zero_fee_htlc_tx set. * onchaind: use our own inputs to add to HTLC txs (iff using anchors). * hsmd: routine to sign HTLC tx merged with our own tx. * channeld: for anchors, use minrelayfee as minimum commit tx fee to allow. * dualopend: use a lower feerate for first commitment tx for anchors. * openingd: use a lower feerate for first commitment tx for anchors. * lightningd: use lowball feerate for commit_tx on anchor channels. * dual-funding: don't override default if multifundchannel doesn't set feerate. * lightningd: code to spend anchor outputs if we want to boost commitment tx. * common: simple helpers to test for either anchor feature. * lightningd/chaintopology: change form of finished callback. * bitcoin/psbt: handle anchor spends. * hsmd: command to sign anchor spends. * pytest: test for commitment feerate. * lightningd: don't make htlc_timeout_satoshis/htlc_success_satoshis zero if we support anchors. * lightningd: don't spam with RBF messages if fee hasn't changed. * bitcoin: set PSBT amount inside tx for bitcoin_tx_output_set_amount. * wallet: fix psbt_using_utxos when used with base PSBT. * msggen: add new version string. * lightningd: ignore any new options we add in deprecated output. * lightningd: initialize channel_type field on dualopend channel creation. * wallet: don't try to set a timer past 2038 on 32-bit platforms. * add m1 native lib paths to Makefile * Correct INSTALL.md requirements for OpenBSD * cln-grpc: Add some missing states to HtlcState * pytest: fix another flake in test_restorefrompeer. * pytest: fix reconnect flake in test_plugin_connected_hook_chaining * lightningd: fix crash on shutdown while if channel being cancelled at same time. * poetry, CI: insist in protobuf v21.12 in both Python and CI. * Makefile: correctly erase generated contrib/pyln-testing/pyln/testing/ in distclean, and rebuild by default. * Makefile: use grouped targets for recipes with multiple fixed outputs * pyln-proto: expose the tlv types * fuzz: initial fuzz-cryptomsg seeds * fuzz: test message encryption and decryption * common: make hdr parameter const * fuzz: initial handshake target seeds * fuzz: add handshake act 3 target * fuzz: add handshake act 2 target * fuzz: add handshake act 1 target * fuzz: add connectd_handshake.h * CI: run 2, not 3 UBSAN/ASAN pytests at once. * pay: don't require description for hashdesc invoices (i.e. undeprecate). * pay, decodepay: handle descriptions with " inside them where we use hashed descriptions. * Added sample config file * pytest: test_pay flake fix. * pytest: fix unexpected reconnection check in test_setconfig() * autoclean: allow dynamic changes to autoclean-cycle. * autoclean: various configuration options now dynamic. * plugins: libplugin support for dynamic config options. * libplugin: make set callback for options take plugin ptr, check correct type. * lightningd: call setconfig on plugins' dynamic options. * plugin: allow plugins to set `dynamic` on options. * setconfig: hook into plugin infrastructure for setconfig. * setconfig: comment out now-overridden lines. * lightningd: setconfig command. * lightningd, libplugins: allocate opt strings from tmpctx, not NULL. * common: allow configvars to be marked dynamic. * lightningd: move listconfigs into configs.c * pytest: handle v fast disconnect during test_funding_external_wallet_corners() * ccan: update to fix hang in ./configure with clang. * Revert "CI: allow postgres and ASAN longer to run." * pytest: fix race in test_penalty_rbf_normal * tools/gossipwith: shutdown, don't close, TCP socket. * lightningd: tell connectd to disconnect even if it's connecting now. * CI: allow postgres and ASAN longer to run. * CI: suppress Postgres vacuuming during tests. * CI: Remove unused CI build script. * CHANGELOG.md: updates from 23.05.1 branch. * Persist feature bits across restarts * py: Update dependencies and relax constraints * common: add test to generate BOLT formatting vectors. * doc: add section about Clang code coverage * contrib: script to generate Clang coverage report * make: allow Clang coverage instrumentation * fix: build on alpine linux * fix(pyln): regenerate the autogenerated files * Revert "ci: re-enable the compilation workflow" * Revert "ci: add the pre-build check as dependencies on lnprototetest" * ci: add the pre-build check as dependencies on lnprototetest * ci: re-enable the compilation workflow * feat(bitcoind): pass the current known block height * msggen: Extend support range to v0.10.1 and mark address, added in v23.02 * openingd: work harder to intuit OPT_SCID_ALIAS. * fix: increase version cryptography python dep * ci: run integration tests with sanitizers * ci: run unit tests with sanitizers * tools: fix memory leaks in hsmtool * lightningd: allow `false` as a default for flags. * pytest: mark test_commando_stress as slow. * commando: integer command parameters can be compared with < and >. * pytest: test for amount comparison. * doc: document that we strip punctuation from parameter names. * doc: Add guides and GitHub workflow for doc sync * fix: prefixing external build dir * lightningd: close plugin dir on return * devtools: call common_setup in main * plugins: initialize destination_reachable * db: avoid calling memcpy on NULL * common: remove json_stream_log_suppress * bitcoin: avoid adding to NULL * doc: making release document update * doc/Makefile: restore CI check for schema changes. * CI: use rebasing instead of merge. * listconfigs: add `plugin` field if config is for a plugin. * lightningd: deprecate listconfigs direct fields. * reckless: use modern listconfigs. * pytest: use modern listconfigs. * listconfigs: show plugin options in 'configs' with normal options. * config: replace accept-htlc-tlv-types with accept-htlc-tlv-type * lightningd: don't simply ignore defaults on flags, deprecate. * lightningd: listconfigs update, using configvars code. * lightningd: fix `listconfigs` `rpc-file-mode` * lightningd: annotate configuration settings which contain JSON literals. * lightningd: use OPT_EXITS for options which exit. * lightningd: make `--clear-plugins` override prior plugin configvars. * lightningd: allow --regtest. * lightningd: switch parsing to common/configvar * lightningd: explicitly mark developer-only options with OPT_DEV. * dev-allowdustreserve: make this a DEVELOPER option. * lightningd: explicitly note what parameters can be specified multiple times. * common/configvar: routines to manage config settings. * lightningd: implement more show commands to reduce listconfigs special casing. * lightningd: simplify listconfigs loop. * ccan: update to latest ccan/opt * bcli: fix type of rpcport. * pyln-client: don't try to convert objects to Millisatoshi. * lightningd: remove deprecated `null` for missing plugin options. * common: allow optional fields in json_scan(). * rs: Bump version of cln-plugin * gossipd: don't fail on gossip deletion * grpc: The plugin needs to use the `server` feature on `cln-grpc` * rs: Guard the cln-rpc dependency behind "server" feature in cln-grpc * doc/Makefile: suppress stupid check. * lightningd: deprecated experimental-websocket-port now we can use bind=ws: * lightningd: allow --bind=ws: * common: add new internal type for websockets. * lightningd: deprecate weird listening options. * lightningd: deprecated --announce-addr-dns. * wireaddr: clean up option parsing, support `dns:` prefix directly. * common: remove websocket type from wireaddr. * connectd: don't advertise websocket addresses. * common/wireaddr.h: simplify parse_wireaddr API. * connectd: clean up add_gossip loops. * staticbackup: don't use wireaddr_internal. * poetry: update grpcio to 1.54.0 * pyln-testing: allow protobuf v4 * Gitlab: updating outdated references * psbt: remove redundant re-setting of tx modification flags. * psbt: replace init/create_psbt with just create_psbt, and simplify it. * psbt: implement psbt_append_output using psbt_insert_output. * rust: Bump the versions of cln-rpc, cln-grpc, cln-plugin and grpc-plugin * ci: use Bitcoin Core 25.0 * Revert "Put bitcoind logging in stdout for pytest" * wallet: fix crash on listtransactions. * db_col_tx: report txn hex if transaction pulled is invalid * msggen: Add preapproveinvoice and preapprovekeysend to msggen * lightningd: fix incorrect reuse of dualopend, leading to dev_queryfeerates race * dualopend: fix minor memleak report. * pytest: slow down for bitcoind. * CI: restore dual-funding tests. * pytest: fix up DF tests after onchaind broadcast rework. * pytest: update test_v2_fail_second for listpeers changes. * lightningd: fix DF crash from libwally update. * fuzz: new fuzz-bech32 seeds * fuzz: simplify fuzz-bech32.c * fuzz: check decoded values * fuzz: allow bech32_encode to succeed * common: rename bech32_encode parameter * db_col_psbt: convert to PSBTv2 on load from db * Put bitcoind logging in stdout for pytest * Add segwit_addr_decode unit tests for invalid bech32(m) * test_withdraw_bech32m: test that expected outputs exist for bech32(m) withdrawals * make: clean up lowdown install * build: remove gmp * signpsbt_done: don't try to access global tx in exception * Update libwally to 0.9.0 * build: remove --enable-experimental-features / EXPERIMENTAL_FEATURES * pytest: remove EXPERIMENTAL_FEATURES as a consideration. * listtransactions: remove annotations, which were only in EXPERIMENTAL_FEATURES. * build: don't generate experimental variants of wire files. * plugins/fetchinvoice: remove obsolete check for feature 102. * channeld: use explicit --experimental-upgrade flag, not #ifdef EXPERIMENTAL_FEATURES * channeld: allow stfu based on peer features, not EXPERIMENTAL_FEATURES. * wire: remove our own blinded payments patch since it's in spec. * tools/generate-wire.py: intuit tlvs field if a tlv type is mentioned. * tools: remove optional csv field handling. * lnprototest: update to avoid compulsory option_anchors init. * fuzz: new fuzz-bip32 seeds * fuzz: correct buffer length for bip32_key_version * fuzz: check key validity before serializing * make: build dependencies with fuzzing flags * ci: use Bitcoin Core 24.1 * doc: remove duplicate wumbo key from listconfigs * fix commas to semicolons * add missing htlc states from incoming htlcs * CI: re-add python timeout. * pytest: fix tests/test_cln_rs.py to avoid race. ------------------------------------------------------------------- Fri Jun 30 11:11:30 UTC 2023 - marec@detebe.org - Update to version 23.5.2: * CHANGELOG.md: update for release. * cln-grpc: Add some missing states to HtlcState * doc/Makefile: restore CI check for schema changes. * CI: use rebasing instead of merge. * CI: don't merge into master * pay: don't require description for hashdesc invoices (i.e. undeprecate). * pay, decodepay: handle descriptions with " inside them where we use hashed descriptions. * pyln-proto: make point release. * pyln-proto: expose the tlv types * pyln-client: make point release to prepare for next release. * pyln-client: don't try to convert objects to Millisatoshi. * listtransactions: remove annotations ------------------------------------------------------------------- Thu Jun 08 09:26:08 UTC 2023 - marec@detebe.org - Update to version 23.5.1: * CHANGELOG.md: update for release. * openingd: work harder to intuit OPT_SCID_ALIAS. * gossipd: don't fail on gossip deletion * lightningd: fix DF crash from libwally update. * wallet: fix crash on listtransactions. ------------------------------------------------------------------- Wed May 10 18:36:26 UTC 2023 - marec@detebe.org - Update to version 23.5.0: * meta: update changelog and pyln version for 23.05 release * meta: Add changelog for 23.05rc4 * jsonrpc: Remove the old "_msat" prefix in the listpeerchannels command * pyln: remove unused variable * meta: Add changelog for 23.05rc3 * pytest: Add helper to get a grpc stub and test decode * msggen: Switch signatures to string instead of bytes * pyln: Set the correct envvar for logging for rust plugins * msggen: Add DecodePay to the mappings * msggen: Map arrays of hashes and add HtlcState enum * msggen: Move overrides into the model itself * jsonrpc: Add request schemas for decode and decodepay * pytest: Extend ListPeerChannels test to include ListClosedChannels * cln-grpc: Rename the ChannelSide variants * msggen: Add ListClosedChannels and overrides * pytest: Add a test for the grpc conversion of listpeerchannels * msggen: Disable grpc response -> json response temporarily * msggen: Normalize enum and field names if they contain a '/' * msggen: Add ListPeerChannels to generated interfaces * jsonrpc: Add versioning annotation to listpeerchannels * meta: Add changelog for 23.05rc2 * fix: build with gcc 13 with enum and int mismatch * connectd: pass correct buflen to memmem * pytest: ignore pip warning * reckless: simplify installer registration * meta: Add changelog for 23.05rc1 * reckless: provide response when failing to add source * reckless: enable case-insensitive searching * reckless: add missing type hints * reckless: match name using installer entry formats * reckless: install command now uses `Installer` class methods * reckless: add installer methods * reckless: add Installer class to support additional languages * reckless: fix crash on non-verbose output * reckless: remove extraneous web request * reckless: avoid superfluous config rewrites * reckless: fix CLI redirect, minor cleanup * Update libwally to 0.8.9 * pyln: Exclude all `cln-` plugins from valgrind * make: Use the CLN_PLUGIN_EXAMPLES variable for testbin * rs: Run hooks, methods and notification handlers in tokio tasks * pytest: Highlight the re-entrancy issue for cln-plugin events * gossip_store: remove now-redundant push bit * connectd: remove handling of push only gossip * connectd: implement timestamp-as-trinary. * connectd: no longer stream our own generated gossip, let gossipd do it. * gossipd: make sure we also spam private channels with the peer involved. * gossipd: send our own gossip aggressively when a new peer connects. * pytest: fix up test_gossip_ratelimit. * pytest: fix test_penalty_htlc_tx_timeout accounting flake. * pytest: make sure we wait for all feerates to be gathered. * lightningd: fix 100% CPU hang on shutdown. * Makefile: remove plugins/sql-schema_gen.h and plugins on `make clean` * plugins/Makefile: don't use echo -n. * ci: run fuzz regression tests * doc: document that urgent doesn't use the 2-block estimate, but the 6-block. * doc: give helpful examples for feerate values. * hsmtool: makerune command. * hsmtool: move sodium_init() to top level. * hsmtool: rework common hsm_secret fetch/decode. * Fix Typo in startup_regtest.sh * have towire_wally_psbt and fromwire_wally_psbt set safe psbt version * ld: Add a couple of logging statements when forwarding * cln: Fix routehints conversion from cln-rpc and cln-grpc * pytest: Reproduce #6143 * lightningd: use channel_type as we're supposed to for forward descisions. * wallet: add channel_type field to db. * zeroconf: don't accept channel_type with option_zeroconf unless we're really zeroconf. * common/channel_type: routines to set known variants, set scid_alias. * wallet: allow psbt_using_utxos to take a starter psbt. * bitcoin: bitcoin_tx_remove_output() * lightningd: fix parent reporting for memleaks. * pyln-testing: add support to tell bitcoind not to include txs if fee is too low. * bitcoin: rename confusing functions. * lightningd: expose default_locktime for wider usage. * db: print nice message and not just backtrace on bad column name. * lightningd: create small hsm_sync_req() helper for hsm queries. * connectd: dev-report-fds to do file descriptor audit. * common: lookup function for symnames. * connectd: don't leak fds if we have both IPv4 and IPv6. * connectd: log broken if TCP_CORK fails. * connectd: don't try to set TCP_CORK on websocket pipe. * lightningd: do RBF again for all the txs. * lightningd: rebroadcast all pending txs each 30-60 seconds. * lightningd: remember if they set "allowhighfees" when we rebroadcast. * lightningd: split the simple onchain tx signing code. * lightningd: base feerate for onchain txs on deadlines, not fixed fees. * feerates: add `floor` field for the current minimum feerate bitcoind will accept * plugins/bcli: use the new feerate levels, and the floor. * lightningd: handle bcli plugins returning fee_floor and feerates parameters. * lightningd: allow "NNblocks" and "minimum" as feerates. * lightningd: clean up feerate handling, deprecate old terms. * lightningd: handle fees as blockcount + range. * lightningd: clarify uses of dynamic (mempool) feerate floor, and static. * pytest: test parsefeerate explicitly. * common: add tal_arr_insert helper to utils.h * plugins/bcli: move commit-fee (dev-max-fee-multiplier) and into core. * msggen: fix incorrect assertion. * doc: add section about improving fuzzing corpora * doc: document "make check-fuzz" * make: add check-fuzz target * fuzz: add check-fuzz.sh * fuzz: improve corpus merging * fuzz: add initial seed corpora * pytest: add blackbox tests for reckless * reckless: use environment variable redirects * reckless: add support for additional networks * reckless: don't crash on subprocess calls * tests: split fetchinvoice recurrence tests into separate test. * Makefile: update to latest BOLT text. * Makefile: update to BOLT 20066dc2aba906f37f3be5a810ae67040f265377 * Makefile: update bolts a0bbe47b0278b4f152dbaa4f5fab2562413a217c * Makefile: bolt version b38156b9510c0562cf50f8758a64602cc0315c19 * Makefile: update bolts fc40879995ebc61cc50dfd729512f17afb15b355. * Makefile: update bolts to 60cfb5972ad4bec4c49ee0f9e729fb3352fcdc6a. * wire: fix extracted patch. * lightningd: update comments now channel-type is merged. * test: add the timeout to the waitpay command * Update INSTALL.md * onchaind: no longer need information about current feerates. * onchaind: remove now-unused direct tx creation. * onchaind: propose_ignore specifically to ignore if output reaches depth. * pytest: clean up wait_for_onchaind_tx interface, remove wait_for_onchaind_broadcast * onchaind: use lightningd to sign and broadcast htlc expired txs. * onchaind: use lightningd to sign and broadcast htlc_timeout transactions. * onchaind: use lightningd to sign and broadcast htlc spending txs. * onchaind: use lightningd to sign and broadcast htlc_success transactions. * common: expose low-level htlc_tx function. * onchaind: have lightningd create our penalty txs. * onchaind, pytest: disable RBF logic. * lightningd: remember depth of closing transaction. * onchaind: use lightningd for spending our unilateral "to us" output. * onchaind: use lightningd to send "delayed_output_to_us" from HTLC txs. * pyln-testing: adapt wait_for_onchaind_broadcast function for when onchaind uses lightningd for broadcast. * onchaind: infrastructure to offload tx creation to lightningd. * lightningd: handle first case of onchaind handing a tx to us to create. * hsmd: add support for lightningd signing onchain txs. * lightningd: add initializing state. * tests: test for stopping node while it's starting. * common: fix build of run-channel_type.c * msggen: Add VersioningCheck * msggen: Use the inferred optional field * msggen: Add an optional patch * msggen: Add patching system, add `added` and `deprecated` to Field * db: db_set_intvar/db_get_var should take a const char *. * db: make db_exec_prepared_v2 return void. * db: catch SQL errors unless we're expecting them. * postgres: add missing 'update_count' to stmt * pyln.testing: remove Throttler. * Makefile: fix check-gen-update to diff *all* files. * plugins/bcli: use getmempoolinfo to determine minimum possible fee. * onchaind: two minor tidyups. * onchaind: helper to read and queue unwanted messages. * bitcoind: fix clone_bitcoin_tx() when tx is take(). * onchaind: simplify lightningd message handling into a switch statement. * lightningd: use tx_feerate() for calculating fallback feerate for onchaind. * bitcoin: add tx_feerate() to reverse-calculate feerate a tx paid. * chaintopology: allow minblock for broadcast_tx. * lightningd: don't use notleak in chaintopology.c * lightningd: provide callback in broadcast_tx() for refreshing tx. * lightningd: rebroadcast code save actual tx, not just hex encoding. * bitcoin: helpers to clone a bitcoin_tx, and format one. * chaintopology: rename broadcast_tx callback name. * channel: don't log scary disconnect message on unowned channels. * pytest: add connection test for gratuitous transient failure message. * doc: fix commando-listrunes SHA256SUM line. * fix helloworld.py example in README for pyln-client * delpay: delete the payment by status from the db * pygossmap: rename GossipStoreHeader to GossipStoreMsgHeader * pygossmap: adds statistic and filter module * pygossmap: store features for nodes and channels * pygossmap: read .disabled from channel_flags * pygossmap: parse node addresses and other data * pygossmap: adds get_neighbors and get_neighbors_hc flodding method * pygossmap: adds get_halfchannel * pygossmap: adds a more complete mesh testcase * pygossmap: adds missing __str__, __eq__ and __hash__ * pygossmap: adds GossmapHalfchannel to module exports * pygossmap: cleanups and optimizations * pytest: adds skipped test_create_gossip_mesh * pyln-testing: fundbalancedchannel default total_capacity to FUNDAMOUNT * pyln-testing: remove deprecated fund_channel * pyln-proto: shorten ShortChannelId.from_str() * fuzz: fix check-src/includes when fuzzing enabled. * channeld: don't asort(NULL). * plugins/pay: fix capacity bias. * bitcoin/script: don't memcmp NULL. * common/gossmap: don't memcpy NULL, 0, and don't add 0 to NULL pointer. * configure: support sanitizers properly. * ccan: update to include versions which pass -fsanitize=address and -fsanitize=undefined * fuzz: fix invalid pubkey error * tests: commando-blacklist * tests: commando-listrunes * doc: commando-listrunes & commando-blacklist * doc: schemas for commando-listrunes & commando-blacklist * commando: add restrictions information in listrune command * commando: Save blacklist runes to datastore * commando: make blacklist effective. * commando: blacklist support * commando: listrunes command * commando: save runes as we generate them * gitignore: Somebody uses vscode: make their life easier! * wallet: don't silently load invalid last_tx psbts. * wallet: fix up PSBTs as a migration. * wallet/test/run-psbt_fixup.c: test for psbt fixups. * wallet/psbt_fixup: routine to fix invalid PBSTs which modern libwally won't load. * docs: update autogenerate file * docs: Add docs on code generation * docs: Use admonition markup for warnings and notes * docs: Use blockreplace.py to include all manpages * docs: Remove redundant ToC in FAQs * tools: Add yml mode to `blockreplace.py` * docs: Add LICENSE to the About section * docs: Fix a number of broken links in the generated docs * docs: Structure the files in mkdocs * docs: Switch to mkdocs for documentation * plugins/sql: add listclosedchannels * lightningd: add listclosedchannels command. * listpeerchannels: add channel_type, both in hex and as array of names. * plugins/sql: recurse correctly into complex objects during processing. * plugins/sql: handle case of subobject with sub-arrays. * common: expose routine to map channel_type to feature names. * wallet: add accessor for closed channels. * wallet: make it clear that `enum state_change` is in db. * wallet: use db_col_optional. * db_col_optional: wrapper for case where a field is allowed to be NULL. * db_bind_scid: rename to db_bind_short_channel_id * wallet: don't clear reference from channel to peers table when we close channel. * wallet: only delete peer from db if it's unused. * wallet: add comment on db noting that `ON DELETE CASCADE` is never used. * msggen: Allow using deprecated fields in the rpc -> grpc conversion * grpc: make the mTLS private keys user-readable only * plugin: autoclean: cleanup the forwards with localfailed * Docker: run directory for post-start if present. * Report failure to sign psbt inputs by hsmd * test_closing_different_fees: b vs balance in loop * Make startup_regtest.sh more robust to bitcoind wallet state * Add PSBT version setting RPC to aid with debugging and compatibility * Re-enable PSBT tests for Liquid except test_sign_and_send_psbt * Update libwally to 0.8.8, support PSBTv2 * test: add PSBT field that doesn't collide with PSBTv2 fields * fuzz: fix UBSan nullability error * fuzz: avoid buffer overflow in bech32 target * wallet/db: don't use migration_context. * lightningd: move bip32_pubkey here from common/, add hsm check. * lightningd: move bip32_base pointer into struct lightningd. * hsmd: capability addition: ability to check pubkeys. * hsmd: new version, which tells us the HSM version, and capabilities. * common: update HSM_MIN_VERSION to reflect reality. * build(deps): bump werkzeug from 2.2.2 to 2.2.3 * channeld: don't spin trying to send commitment while waiting. * pytest: add timeout to test_feerate_stress. * connectd: log status_failed on TOR problems * Output channel_id in listfunds * plugin: fetchinvoice: set the quantity in invreq * test_backfill_scriptpubkeys: stop first cln node before second sub-test * ci: add timeout field to 2h for each task * cln_plugin: add `shutdown()` method to `Plugin` * pytest: don't run test_backfill_scriptpubkeys under valgrind in CI. * lightningd: require "jsonrpc": "2.0" as per JSONRPC spec. * cleanup: rename json_add_amount_msat_only to json_add_amount_msat * global: remove deprecated "msat" suffix on msat fields. * global: remove deprecated non-msat-named msat fields. * lightningd: remove deprecated local_msat, remote_msat from listpeers. * lightningd: remove deprecated behavior where checkmessage would fail quietly. * hsmd: deprecate reply_v1. * common/hsm_version: list sha256 for every known version. * fix: partial fix lnprototest runner * doc: update release procedure ------------------------------------------------------------------- Tue Mar 14 18:18:42 UTC 2023 - marec@detebe.org - Update to version 23.2.2: * CHANGELOG.md: v23.03.2 * plugins/pay: revert removal of paying invoice without description. ------------------------------------------------------------------- Sat Mar 11 18:20:57 UTC 2023 - marec@detebe.org - Update to version 23.2.1: * meta: update changelog for v23.02.1 * offers: enable label for invoicerequest * doc: update documentation for fetchinvoice(7) and offer(7). * doc: fix modern usage of sendinvoice (changed in v22.11) * doc: add documentation for invoicerequest commands. * chanbackup: even if they enable experimental-peer-storage, check peers * wallet: really allow broken migrations. * connectd: fix crash on freed context for new connections. * gossipd: don't make new zombies, just prune channels as we did before. * gossipd: ignore zombie flag when loading gossip_store. * pytest: remove zombie test. * sql: fix nodes table update. * sql: fix bug where nodes table would get duplicate entries. * gossipd: remove zombie spam cupdate when resurrecting * meta: update changelog and pyln version for 23.02 release * reprobuild: use pyenv for python installation * repro: Add `protoc` dependency to repro-build * gossipd: load and store node_announcements correctly * gossipd: flag zombie channels when loading from gossip_store * gossipd: correct node_announcement order when zombifying channels * keysend: Do not strip even TLV types that were allowlisted * keysend: Extract `accept-extra-tlv-types` from `listconfigs` * libplugin: Expose the `jsonrpc_request_sync` method * json: Add method to parse a u64 array * pytest: remove openchannel('v2') marker from test_sql * sql: fix crash on fresh node_announcment. * sql: fix schema tests since num_channels added to listpeers. * fix: do not send send peerstorage msg when disabled * pay: fix delpay to actually delete. * autoclean: fix timer crash when we're cleaning two things at once. * db-fix: update NULL lease_satoshi fields to zero * meta: Update changelog for 23.02rc3 * gossipd: remember to squelch node announcements when shuffling * gossipd: remove redundant is_node_zombie() in routing_add_node_announcement. * gossipd: remove any zombified node_announcements on load. * gossipd: don't zombify node_announcements, just forget them. * gossipd: don't complain about unknown node_announcements if it's a zombie. * gossipd: don't broadcast node_announcement if we have no public channels. * gossipd: neaten node_has_broadcastable_channels logic. * devtools: fix ZOMBIE detection in devtools/dump-gossipstore. * Change year to 2023 in LICENSE * fix: fixes `FATAL SIGNAL 11` on gossmap node * pytest: allow ipv6 in test_announce_dns_suppressed * peer storage: advertise features as optional * meta: Add changelog for 23.02rc2 * pytest: adds xfail test that shows datastore issues * lightningd: unescape JSON strings for db. * commando: don't try putting an integer as the 'string' parameter to "datastore". * libplugin: don't spew datastore errors to LOG_DEBUG. * fix: compilation error on armv7l 32 bit * gossipd: don't resurrect deleted half_chans * add a log message when it is not possible upgrade the db * meta: Add changelog for 23.02rc1 * msggen: Regenerate for addition of SendCustomMsg * msggen: Enable SendCustomMsg * lightningd: don't put old deprecated `local_msat` and `remote_msat` in listpeerchannels. * ci: include rust tests inside the pre build checks * fix(grpc): add the num_channels field inside the tests * ignore sql binary plugin * cln-grpc: add roundtrip tests for test_getinfo and test_listppers * cln-rpc: explicitly enumerate ChannelState enum * Implement GRPC -> JSON conversions also for response types * Implement conversion JSON->GRPC also for requests type * options: create enable/disable option for peer storage. * plugins/chanbackup: make get_file_data take ctx. * plugins/chanbackup: neaten a little. * features: make name of peer storage features match spec. * plugins/chanbackup: switch to normal indentation. * tests/test_misc.py: Add test_restorefrompeer. * Plugins/chanbackup: Add RPC for recovering from the latestscb received from peers. * Plugins/chanbackup: Add hook for exchanging msgs on connect with a peer * Plugins/chanbackup: Add hook for receiving custommsg * Plugins/chanbackup: Add SCB on CHANNELD_AWAITING_LOCKING stage * plugins/chanbackup: use grab_file. * plugins/chanbackup: Define FILENAME globally (Good Manners) * Plugins/chanbackup: Add featurebit Peerstrg and YourPeerStrg. * plugins/chanbackup: PLUGIN_RESTARTABLE to PLUGIN_STATIC... * connectd: make exception for peer storage msgs. * peer_wire_is_internal helper. * feature(PEER_STORAGE and YOUR_PEER_STORAGE) added in feature.c and internal message. * wire: Add patch file for peer storage bkp * lightningd: allow sendcustommsg even if plugins are still processing peer_connected. * lightningd: don't access peer after free if it disconnects during peer_connected hook. * lightningd: fix leak report from peer_connected. * df: remove static remote key dependency * df: remove minimum witness weight for input calculations * df: fetch both the first+second commitment point * rfc-dual-fund: update to latest spec for dual-funding * test (df): check 'require-confirmed-inputs' for v2 opens * df: persist our setting to disk, read back to dualopend at reinit * dualopend: if required, validate inputs rcvd from peer * df: add new config option for v2 opens `--require_confirmed_inputs` * df: reuse psbt validation for the psbts incoming from dualopend * df: for dryruns, inform on requires-confirmation value * df: push back psbt to validate iff peer requests confirmed inputs * df: wire up peer's "require-confirmed-inputs" * df: persist channel open preference to database * tx_roles: allow to be serialized btw processes * v2 open: if flagged, check that all our inputs are confirmed * Update Bitcoin Core to 24.0.1 in other git ci locations * Fix 'extreme cases' logging of many commit timer failures * mssgen: adds num_channels * pytest: listpeers new attribute num_channels * doc: listpeers new attribute num_channels * rpc: adds num_channels to listpeers * grpc: Allow conversion code to use deprecated fields * grpc: Silence a warning about `nonnumericids` being unused * More accurate elements commitment tx size estimation * msggen: Regenerate for addition of SignInvoice * msggen: Enable SignInvoice * lightningd: Add `signinvoice` to sign a BOLT11 invoice. * doc: Correct `createinvoice`'s `invstring` description * cli: accepts long paths as options * make: Add doc/index.rst to generated files * lightningd: don't print zero blockheight while we're syncing. * SECURITY.md: Tell them to spam me, and include our GPG fingerprints. * pytest: fix flake in test_closing_disconnected_notify * Makefile: don't try to build sql plugin if there's no sqlite3 support. * lightningd: tell connectd we're shutting down. * connectd: add new start_shutdown message. * connectd: keep array of our listening sockets. * pytest: fix flake in test_bolt11_null_after_pay * tests: de-flake test that was failing on cltv expiry * df: echo back "tx-abort" when we receive 'tx-abort' * dual-open: use tx-abort instead of warning/errors * lightningd: remove duplicate routine `fail_transient_delayreconnect` * v2 open tests: don't drop connection when an openchannel fails * lightningd: don't send channeld message to onchaind. * relax log check for test_closing_higherfee * gci: Re-Add `TEST_NETWORK=liquid-regtest` to CI run * Need `sudo` for `make install` * Add protobuf as a Dependency * Must Specify pip3 on macOS * Use Python 3.7.8 Instead of 3.7.4 for macOS Install * Update CI to Bitcoin Core 24.0.1 * build(deps): bump tokio from 1.23.1 to 1.24.2 * v2 opens: don't use p2sh inputs for v2 opens * upgradewallet: JSONRPC call to update p2sh outputs to a native segwit * fundpsbt: add option to filter out wrapped p2sh inputs * dual-open-rbf: remember the requested lease amount btw restarts * test: restart node during rbf * dual-fund: patch in channel_type logic * tests: check that non-anchor opens can't use liquidity ads * tests: anchors is only EXPERIMENTAL_FEATURES * connectd: patch valgrind error w/ buffers for error msgs * dual-fund: only allow for liquidity ads if both nodes support anchors * dual-fund: remove anchor assumption for all dual-funded channels * dual-fund: check features to print (anchors not assumed) * opening: helper for anchor flagged, use in dualopend also * dual-fund: validate upfront shutdown using taproot + anchors * openingd: pull out validation for shutdown script * df tests: use the amount from the logs to check for! * dual-fund: on re-init, re-populate opener_funding/accepter_funding * dual-fund: update to latest, add in updates to rbf amounts * dual-fund: update extracted CSVs to latest bolt draft * offers: fix pay where we are using deprecated apis. * pytest: add test for using offers with allow-deprecated-apis=True * pyln-client: adds utils cln_parse_rpcversion * valgrind-fix: patch valgrind error on log statement in pay plugin * lightningd: don't run more than one reconnect timer at once. * pytest: neaten test_cln_sendpay_weirdness, rename. * pay: specify the channel to send the first hop. * libplugin-pay: fix (transitory) memleak which memleak detection complains about. * pytest: limit test_cln_sendpay_weirdness to only failures. * tests: add Carl Dong's example exhaustive zeroconf channel pay test. * pytest: fix flake in test_closing_simple when we mine too fast. * plugins/sql: fix foreign keys. * pyln-testing: fix wait_for_htlcs helper * lightningd: flag false-positive memleak in lightningd * plugins/pay: fix htlc_budget calc when we get temporary_channel_failure * libwally: update to cln_0.8.5_patch * doc-schema: make address field in getinfo response required * make: fix make doc error * db: catch postgres error on uninitialized database * datastore: Add check for empty key array * doc: remove sections on litestream, .dump and vacuum into * typo fixes found by @niftynei * doc: add examples for sql plugin. * plugins/sql: listsqlschemas command to retrieve schemas. * plugins/sql: add bkpr-listaccountevents and bkpr-listincome support. * plugins/sql: allow some simple functions. * doc: document the sql command. * plugins/sql: print out part of man page referring to schemas. * plugins/sql: pay attention to `deprecated` in schema. * doc/schemas: fix old deprecations. * plugins/sql: refresh listnodes and listchannels by monitoring the gossip_store. * plugins/sql: include the obvious indexes. * pytest: perform more thorough testing. * plugins/sql: add listpeerchannels support. * plugins/sql: make tables for non-object arrays. * plugins/sql: rework to parse schemas. * plugins/sql: create `struct column` to encode column details. * plugins/sql: initial commit of new plugin. * doc: use specific types in schema rather than "hex". * doc/schemas: remove unnecessary length restrictions. * common: add routine to get double from JSON. * tools/fromschema.py: don't try to handle more complex cases. * common/gossip_store: move subdaemon-only routines to connectd. * common/gossip_store: expose routine to read one header. * common/gossip_store: clean up header. * doc: fix listsendpays man page. * lightningd: fix type of listhtlcs payment_hash. * plugins/topology: add direction field to listchannels. * listtransactions: get rid of per-tx type annotations. * wallet: remove unused TX_ANNOTATION type in transaction_annotations table. * doc: remove unused offerout schema. * pytest: test_channel_resurrection now succeeds * gossipd: zombify inactive channels instead of pruning * gossip_store: add a flag for zombie entries * gossipd: avoid gossipd crash due to double freeing timer * pytest: gossipd: test resumption of pruned channels * common/bolt11: fix 32-bit compilation. * decode: fix handling of blinded_payinfo. * plugins: update to match latest offers text. * common: update to latest onion-message spec. * common/test: fix up name of test file to match latest version. * common: update to latest route-blinding spec. * common/onion_decode: put final flag in onion_payload. * tools/check-bolt.c: don't leak open directory. * wire: use correct number of update_add_tlvs blinding field. * common/test: remove unused padding in bolt04/blinded-onion-message-onion-test.json * gossipd: Do not send warning when node_announcement parsing fails * hsmd: increase HSM_MAX_VERSION to 3 * hsmd: add hsmd_preapprove_keysend and check_preapprovekeysend pay modifier * hsmd: add hsmd_preapprove_invoice and check_preapproveinvoice pay modifier * docker: Install protobuf-compiler for builder * cln-rpc: use serde rename instead of alias * common/bolt11: convert to table-driven. * common/bolt11: add pull_all helper for common case of entire field. * common/bolt11: const cleanup, fix parsing errors. * pytest: test ip discovery for custom port * doc: usage of --announce-addr-discovered-port option * opts: adds --announce-addr-discovered-port config option * cli: getinfo output to regard --ip-discovery * opts: deprecate --disable-ip-discovery switch * pytest: fix and adapt test_remote_addr_disabled * doc: announce-addr-discovered config switch * opts: announce-addr-discovered on/off/auto switch * opts: adds the autobool on/off/auto feature * ci: adds git fetch before doing schema checks * lightningd: use a hash table for peer->dbid. * lightningd: use hash map for peers instead of linked list. * pytest: fix output order assumption in test_setchannel_all * common: expose node_id_hash functions. * wallet: add dependency on lightningd/ headers. * channeld: remove dead HTLCs from htable and free them (eventually) * add PartialEq to ShortChannelId * comm: make sure that our version check is reliable * lightningd: fix valgrind reported leak when we exit early. * ccan: update to fix recent gcc "comparison will always evaluate as 'false'" warning * gha: Temporarily disable `test_notify` * ci: Downgrade the upload-artifact action to v2.2.4 * gci: Add a test for the postgres backend * gci: Add an explicit name to the matrix for display * gci: Split out installation of elements and bitcoin into a script * make: Clean up duplicate cargo examples build rules * ci: Use bzip2 and release to reduce artifact size * ci: Add gather step * ci: Build and test in ubuntu:22.04 and install lowdown * tests: Fix a small memory leak in the onion test vector tester * ci: Add bitcoind to integration test job * ci: Automatically cancel CI runs if we push a new version to the PR * ci: Split out the unit tests * ci: Split the pre-flight checks into separate steps * ci: Add a testpack.tar target to tranfer artifacts between CI jobs * gci: Re-add tests of pre-compiled binaries * gci: Clone BOLTs repository in pre-build checks * py: Ignore missing whitespace after keyword for now * gci: Split out the stages better * gci: Update Github Actions steps to their latest versions * gci: Stop uploading unit test results * gci: Split out the lnprototest from the larger CI run * pytest: fix flake in tests/test_closing.py::test_closing_specified_destination * lightningd: deprecate p2sh-segwit addresses for `newaddr` `addresstype` * pytest: use bech32 addresses everywhere. * pyln-testing: don't default openchannel and fundwallet to p2sh-segwit, use bech32. * gossip: Do not send warnings if we fail to parse a `channel_update` * CI: fix schema diff check. * cln-grpc: update listpeers json fixing tests * doc: remove manual field descriptions from listpeerchannels(7). * lightningd: deprecate listpeers.channels * pytest: fix race in test_bookkeeping_closing_subsat_htlcs * tests: use listpeerchannels. * tests/utils.py: use listpeerchannels. * contrib/pyln-testing: use listpeerchannels. * plugins/topology: use listpeerchannels. * plugins: use listpeerchannels instead of listpeers. * pay: use json_to_listpeers_channels() for local_channel_hints. * libplugin: don't return unopened channels from json_to_listpeers_channels(). * plugins/libplugin: flatten return from json_to_listpeers_result. * plugins: make bookkeeper use the new listpeerchannels command. * lightningd: add listpeerchannels command * lightningd: prepare internal json routines for listpeerchannels. * proposal_meets_depth tracked output always has a proposal * lightningd: don't call memcpy with NULL. * all: no longer need to call htable_clear to free htable contents. * setup: make all htables use tal. * plugins/command: ensure htables are always tal objects. * plugins/pay: ensure htables are always tal objects. * gossipd: use pointer to hash table for channels in node. * gossipd: ensure htables are always tal objects. * memleak: prepare for htable to be a tal object. * connectd: ensure htables are always tal objects. * gossmap: ensure htables are always tal objects. * lightningd: ensure htlc htables are always tal objects. * lightningd/chaintopology: ensure htables are always tal objects. * tests: add account_id's and match by acct id, not test ordering * tests: mark test as dev (times out otherwise) * CI: rough check that schema changes seem to mention added, don't delete non-deprcated. * doc: add recent additions, fix annotation on listpeers to actually deprecate. * docs: handle "added": "version" and "deprecated": "version" from schemas. * doc: document that we should annotate added and deprecated schemas. * cli: add -c/--commando support. * commando: add filtering support. * commando: correctly replace the `id` field in responses. * commando: track incoming and outgoing JSON IDs. Get upset if they don't match! * commando: build ID of command based on the id they give us. * commando: send `id` inside JSON request. * commando: require that we have an `id` field in JSON request. * lightning-cli: fix error code on invalid options, document them. * lightningd: treat JSON ids as direct tokens. * libplugin: don't turn non-string JSON ids into strings. * pytest: add tests for devtools/mkfunding * doc: channel_state_changed has a timestamp field * Ping request types are changed from number to u16 * build(deps): bump tokio from 1.23.0 to 1.23.1 * doc: fix accept-htlc-tlv-types description * common/test: fix typo in bolt12 test vector generation. * doc: "--enable-experimental-features" hard-codes experimental-offers + experimental-onion-messages * unify the zlib version * change zlib download path * common: update comments documenting the use of param() * cli: fix buffer overflow in (currently unused!) code for progress bars. * Replace head -n with sort -R for better random peer selection * remove unnecessary CPU_TO_LE32 * git: Mark node_pb2.py as text so we can see changes * generate composite fields in grpc * lightningd: Look for channels by alias when finding channels * topology: fix memleak in listchannels * connectd: don't ask DNS seeds for addresses on every reconnect. * rs: Add cln-plugin metadata * rs: Add cln-grpc metadata * rs: Add cln-rpc metadata * db: Fix the ordering of `channel_htlcs` in postgres * pytest: adds test for msat to int comparison * Update release tag * pyln-client: make Millisatoshi comparable to int * Fix link to github.com/lightningd/plugins not clickable in README * rs: Update cargo dependencies * cln-plugin: Re-export anyhow::anyhow macro * cln-plugin: make available the configuration in plugin * doc: note that grpc needs the protobuf-compiler * ci: Add `protoc` compiler to setup * ci: Unconditionally install `protoc` * rs: Bump crate versions for publication * rs: Remove unused dependency from cln-plugin -> cln-rpc * rs: Update outdated dependencies * Turn on logging for key topics in bitcoind for black box tests * tests: make test_libplugin use the datastore. * libplugin: more datastore helpers. * pyln: add datastore routines. * Adds helper functions to cast Value variants * doc: sphinx-build fix external links (urls), language warning and broken footnote * dox: fix "sphinx-build -b html" warning: reference target not found * Properly raise ValueError message in wait_for * lightningd: remove `setchannelfee`. * sendpay: remove style `legacy` setting. * lightningd: only allow closing to native segwit * wallet: fundpsbt, utxopsbt reserve cannot be bool. * invoice: expiry must be in seconds. * pay: remove description_hash without description. * doc: replace deprecated parameter keyword "msatoshi" with "amount_msat" * add reserve to the fundchannel docs * docs: fix typo * docs: remove the table inside the reoribuild docs * doc: drive-by spelling corrections * fromschema.py: escape underscores in descriptions * doc: escape more naughty underscores * doc: check-manpages: add check for unescaped underscores ------------------------------------------------------------------- Sat Dec 17 13:57:47 UTC 2022 - marec@detebe.org - Update to version 22.11.1: * meta: Add changelog for hotfix release v22.11.1 * build(deps): bump secp256k1 from 0.22.1 to 0.22.2 * lightningd: don't announce names as DNS by default. * wireaddr: allow for UpperCase DNS names * doc: document the usage of DNS hostnames * misc: Update cdecker's gpg key * wireaddr: adds test for punycode * wireaddr: is_dnsaddr allow underscore in hostname * Update the contrib arm32v7 and arm64v8 dockerfiles * wallet: only log broken if we have duplicate scids in channels. * common: fix arm32 compile breakage. * Revert "lightningd: always require "jsonrpc": "2.0" in request." * reckless: further verbosity/squelch of pip output * reckless: avoid redundant include statement in config * reckless: fix verbose option * jsonrpc: fix error when we abort batching due to timeout. * autoclean: fix uncleaned stats when we don't clean due to being too new. * doc: document autoclean-once command. * tools: Add multi-language support to blockreplace.py * doc: Create a blockreplace tool to update generated blocks in docs * pyln: Fix an issue with the LightningConnection short-reading * meta: Update changelog for v22.11 final * lightningd: don't cap spendable_msat/receivable_msat for wumbo channels. * pytest: test for wumbo direct payments. * connectd: don't spam logs when we're under load. * lightningd: do not abort while parsing hsm pwd. * docs: Add the `reckless` manpage to the readthedocs generation * lightningd: correctly exit when an important-plugin fails to start. * submod: Switch lnprototest to clone from github.com/rustyrussell/lnprototest * ci: Temporarily disable lnprototest tests * db: Backfill missing HTLC IDs in the forwards table * meta: Adjust changelog for v22.11rc3 * lightningd: don't timeout plugins if init is slow! * cln-plugin: Adjust visibility of some internals * cln-plugin: Make the configuration in `init` public * CHANGELOG.md: include the v0.12.1 CHANGELOG entries! * CHANGELOG.md: order into a more user-first ordering. * doc: Add a readme to the `cln-grpc` proxy * autoclean: Fix a null-pointer derefence when checking HTLC age * pyln-client, libplugin, rust cln-plugin: explicitly flag that we allow non-numeric JSON ids. * lightningd: only use non-numeric JSON ids if plugin says we can. * plugins: set non_numeric_ids flag based on getmanifest `nonnumericids` field. * doc: document how to construct JSON ids in modern plugins and utilities. * meta: Update CHANGELOG for release candidate v22.11rc2 * rs: Bump cln crate versions to 0.1.1 * docker: Fix the dockerfile * msggen: Be less magic in detecting the repo root * grpc: Add the experimental optional flag to protoc * make: Fix external/lowdown clean targets * make: Make the Makefile make 4.4 compatible * ld: Replace list of outgoing_txs with a hash table * ld: Add an outgoing_txs_map htable to avoid costly lookups * ld: Do not blindly add rebroadcasts to outgoing_tx set * onchaind: cap RBF penalty fee for testnet/regtest * wallet: fix typo in debug message. * wallet/wallet.c: don't declare unused variable. * tools/generate_wire.py: don't declare unused for variable. * add some stuff to gitignores * pyln: Bump versions * meta: Add version v22.11rc1 changelog * lightningd.service: note that the hardening setting seems to break node.js plugins * onchaind: Adjust witness weight estimate to be more conservative * onchain: Document how the expected witness weight for the close tx * reckless: Replace custom logging with the logging crate * reckless: Replace urllib3 with urllib * reckless: neaten path conversions * reckless: avoid changing directory during install * reckless: Use urllib3 instead of requests * check: fix warnings from shellcheck 0.8.0 * bolt12: update comments to match latest spec. * lightningd: re-add 'offerout' functionality, as 'invoicerequest'. * lightningd: add "savetodb" argument to createinvoicerequest, add listinvoicerequests/disableinvoicerequest * db: add invoicerequests table. * bolt12: routines to hash the invreq parts. * lightningd/invoice.c, plugins/fetchinvoice.c: use tlv_make_fields() instead of towire/fromwire hack. * offers: use existing copied fields. * decode: print unknown fields in bolt12 strings. * bolt12: use spec field names, update decode API. * devtools/bolt12-cli: fix decode to understand modern fields. * bolt12: update to modern signature scheme. * bolt12: import the latest spec, update to fit. * offers: remove 'send-invoice' offers support. * offers: make them always unsigned. * common/features: add explicit bolt12 feature sets. * common/bolt12: code to initialize invreqs from offers, invs from invreqs. * common/bolt12: add code to generate offer_id, extract parts of streams. * offers: create a real blinded path, if necessary. * offers: monitor blockheight. * listincoming: add htlc_min_msat, public and peer_features fields. * common/onion_decode: put the path_id into onion_payload->payment_secret. * createinvoice: make a minimal blinded "path" in bolt12 invoice if none presented. * lightningd: temporarily ignore missing payment_secret for bolt12. * invoice: invert check to reduce indentation. * onion_message: don't use general secret, use per-message secret. * common/blindedpath: expose API at a lower level. * plugins/libplugin-pay: hack in blinded path support. * common/onion: split into decode and encode routines. * common/onion: handle payment by node_id. * lightningd: don't return normal errors on blinded path entry, either. * common: catch up on latest routeblinding spec. * wallet: Add utxo_is_immature helper * wallet: Use boolean to determine whether an output is coinbase * wallet: mark coinbase outputs as 'immature' until spendable * tests: test for coinbase wallet spend. * lightning-cli: support --filter parameter. * doc: add lightingd-rpc documentation. * libplugin: support filters. * pyln: add context manager to simpify filter use. * pytest: add filter tests. * lightningd: set filter when we see 'filter' object. * common/test: add unit tests for JSON filtering. * common/json_stream: support filtering don't print fields not allowed. * tests/fuzz: don't pull in JSON common at all. * common/json_filter: routine to turn "filter" JSON into a filter. * common/json_filter: routines for json filtering. * gci: Force MacOS CI Job to use python 3.10 * common/sphinx: don't use fixed lengths anywhere. * pytest: add test for generating non-standard length onion errors. * lightningd: --dev-onion-reply-length option. * reckless: fix git clone issue with removed dir * Reckless: add man page * reckless: analyze repositories with urlparse * reckless: improve config file handling * reckless: add function for lightning-cli calls * reckless: multiline string style cleanup * reckless: replace os.path with pathlib operations * reckless: refactor argument list handling. * reckless: add type hints * reckless: raise exception or early termination instead of returning None * reckless: update help alias * reckless: use argparse subparsers * reckless: use config that was explicitly passed to lightningd * reckless: detect pip3 or pip * reckless: use the lightning path when invoking lightning-cli * reckless: it turns out the warning is a bit much. * reckless: new tool to manage lightningd plugins * gossipd: Cleanup channel update replacement logic * pytest: test for gossip store corruption by private channel updates * gossipd: ensure old private channel updates are properly deleted * pytest: Add test for optional options in cln-plugin * Plugin config options with no defaults * Collaborative transaction building * doc: Move M1 instructions install to its own section * doc: add `make install` line for macOS instructions * mkfunding: no scriptPubKey on utxo causing crash, so we add one * mkfunding: add missing common_setup * msggen: Map the `extratlvs` field of `keysend` * rs: Fix two small regressions * keysend: Allow quoted numbers in `extratlvs` * json: Add helper for quoted numbers * wire/Makefile: fix missing wire/bolt12_exp_wiregen.c in ALL_C_SOURCES. * common/blindedpath: create onion mesage test vectors. * common/onion_message_parse: generic routine for parsing onion messages. * common: update to latest onion message spec. * common/test: check we meet bolt04/onion-route-blinding-test.json * channeld, lightningd: allow blinded payments with !EXPERIMENTAL_FEATURES. * lightningd: return invalid_onon_blinding for any blinded payment error. * common/blindedpay: routines to construct a blinded payment. * common/onion: cleanup by removing unnecessary local temporary. * common/onion: enforce payment constraints. * common/onion: blinded payment support. * common/features: understand the route_blinding feature (feature 24) * common/blindedpath: generalize construction routines. * common/blindedpath: generalize routines. * channeld: don't calculate blinding shared secret, let lightningd do it. * common: remove old route-blinding-override test, update route-blinding test for new vectors. * wire: add latest Route Blinding htlc fields from https://github.com/lightning/bolts/pull/765 * Remove point32. * offers: print out more details, fix up schema for decode of blinded paths. * BOLTs: update to more recent bolt12 spec. * Makefile: separate bolt12 wireobjects * gossmap: move gossmap_guess_node_id to pay plugin. * bolt12: change our payer_key calculation. * hsmd: don't use point32 for bolt12, but use pubkeys (though still always 02) * lightningd: use 33 byte pubkeys internally. * Makefile: check that hsm_version.h changes if wire/hsmd_wire.csv contents does * hsmd: introduce a simple API versioning scheme. * funder: cleanup datastore on state-change/channel failure * funder: filter prev-outs such that we only use still unspent ones * funder: use utxopsbt to build psbt for RBFs * funder: use previous outputs in count towards available funding * funder: pull out previous input list from datastore on RBF * funder: save utxos of signed txs to datastore * funder: rm quote that makes nifty cringe every time she sees it * df: pass lease data back to funder for rbfs * openchannel2: may re-use rates * df: for rbfs, since we know what they asked for, we can abort * df: put requested_lease onto state, so it persists * funder: we always pass in channel_max, no need to special case it * chainparams/dual-open: set max_supply; use for max on wumbo channels * zlib 1.2.12 yanked, update to 1.2.13 in Dockerfile * pytest: fix flake in test_emergencyrecover * psbt: wipe global tx scriptSig/witness data after saved to PSBT * bitcoin: add test for to/from wiring a bitcoin tx w/ scriptsig data * docker: Add bitcoin-cli to the alpine dockerfile * docker: Separate builder from runner stage in alpine docker image * docker: Build the alpine docker image from a clone * docker: Clean up dependencies for alpine build * Clean up Dockerfile.alpine * move alpine build dependencies to virtual package * update package dependencies for Alpine Linux * py: Update protobuf dependency to silence dependabot * keysend: just strip even unknown fields. * tests/test_misc.py: Check if funds are getting recovered on reconnecting... Changelog-None: Increasing test scope * cln-rpc: implement from Secret to slice conversion * devtools: Make fund_nodes compatible w/zsh * msggen: generate deprecated fields in rust.py * channeld: remove onion objects. * common/onion: remove old blinded payment handling. * bitcoin: add routine to check a Schnorr sig given a 33-byte pubkey. * pytest: test that we don't change our payer_key calculation. * devtools: enhance bolt12-cli to convert to/from hex * plugins/fetchinvoice: remove obsolete string-based API. * onionmessages: remove obsolete onion message parsing. * doc: update references to old BOLTs repo. * lightningd: have `makesecret` take `hex` or `string` (just like `datastore`) * pytest: Bypass schema verification for some RPC calls * schema: Add `enforcedelay` to `setchannel` * Setchannel request is provided * channeld: do not enforce max_accepted_htlcs on LOCAL in channel reinit * sphinx: rename confusing functions, ensure valid payloads. * common/onion: expunge all trace of different onion styles. * common/onion: remove all trace of legacy parsing. * tests: add onion-test-vector from "BOLT 4: Remove legacy format, make var_onion_optin compulsory." * use msat_or_all for fundpsbt request amount * pytest: fix flake in test_gossip_persistence. * pytest: fix flake in test_zeroconf_forward * delforward: allow deletion of "unknown in_htlc_id" and fix autoclean to use it. * delforward: tally up deleted forwards so that getinfo's `fees_collected_msat` doesn't change. * db: correctly migrate forwards for closed incoming channels. * pytest: add test for migrations upgrade which breaks 'fees_collected_msat'. * pyln: Bump versions to v0.12.1 * Add basic arithmetic to `Amount` type * Make eligible types `Copy` * Use `secp256k1` for public key type * Use `bitcoin_hashes` for `Sha256` * Implement a typed version of `call` to avoid useless matching * Impl `std::error::Error` for `RpcError` to make it anyhow compatible * Add ConfigurationDirectory * The project is called Core Lightning * lightningd: figure out optimal channel *before* forward_htlc hook. * channeld: send error, not warning, if peer has old commitment number. * pytest: Configure the plugin logging to debug * msggen: Use owned versions to convert from cln-rpc to cln-grpc * cln-plugin: Prep the logging payload in a let * cln-plugin: Add log filtering support * cln-plugin: Defer binding the plugin state until after configuring * doc/GOSSIP_STORE.md: document the gossip_store file format. * BOLT: update to clarify HTLC tx amount calculation. * gossipd: don't close non-local channels immediately, add 12 block delay. * lnprototest: update gossip test including 12 blocks delay * gossipd: batch outpoints spent, add block height. * gossipd: set no_forward bit on channel_update for private channels. * common: assume htlc_maximum_msat, don't check bit any more. * BOLT: update to version which requires option_channel_htlc_max. * gossipd: bump gossip_store to indicate all channel_update have htlc_max. * gossmap: make API more robust against future changes. * gossipd: actually validate gossip_store checksums at startup. * gossipd: don't try to upgrade ancient gossip_store. * BOLT: update to fix gossip pruning quote. * pytest: fix flake in test_pay_disconnect * pytest: fix flake in test_onchain_different_fees * onchaind/onchaind_wire.c duplicated in ONCHAIND_SRC * pyln-spec: update to latest spec. * pyln-spec: update Makefile to use poetry for release, generate versions * pytest: slow down test_autoclean. * db: set now-unused channels.short_channel_id text column to NULL after migration * db/postgres: avoid memleak. * lightningd: allow delpay to delete a specific payment. * lightningd: listsendpays always has groupid. * lightningd: allow a connection to specify db batching. * lightningd: don't always wrap each command in a db transaction. * libplugin: avoid memmove if we have many outputs to lightningd. * libplugin: optimize parsing lightningd rpc responses. * autoclean: remove per-delete debugging messages. * autoclean: add autoclean-once command. * doc: manpages and schemas for autoclean-status. * autoclean: use config variables, not commands. * autoclean: clean up listforwards as well. * lightningd: add `delforward` command. * lightningd: add `listhtlcs` to list all the HTLCs we know about. * lightningd: add in_htlc_id / out_htlc_id to listforwards. * wallet: use scid not string for failchannel (now failscid) in payments table. * db: add `scid` field to channels table. * wallet: use db_col_scid / db_bind_scid where possible. * db/bindings: rename db_bind_short_channel_id to db_bind_short_channel_id_str, add db_bind_scid. * wallet: replace forwarded_payments table with forwards table. * wallet: simplify payments lookup so sqlite3 uses index. * lightningd: index to speed up sendpay / listsendpays * autoclean: handle cleaning of old payments (not just invoices). * autoclean: allow cleaning of paid invoices too. * autoclean: save stats on how much we cleaned. * lightningd: deprecated "delexpiredinvoice", put functionality in autoclean plugin. * autoclean: new interface * libplugin: handle JSON reply after command freed. * ci: Use the new make upgrade-version target to manage versions * pyln: Bump pyln-client dependency in pyln-testing * pyln: Adjust the auto-publish task to trigger on tags * pyln: results of make update-pyln-versions NEW_VERSION=0.12.0 * Makefile: add targets to upgrade pyln versions, push releases. * pyln: use poetry version, add target to check version, use poetry publish. * build-release: configure before submodcheck * msggen: Add conversion from cln-rpc to cln-grpc for Option<Outpoint> * doc: Fix wrong_funding description in manpage and type in schema * lightningd: allow extra tlv types in non-experimental mode. * keysend: allow extratlvs parameter, even in non-experimental mode. * keysend: try to find description in TLV. * Fix a small typo * gh: Mark some derived files as such * pytest: fix flake in test_sendcustommsg * pyln-client: use f strings to concatenate JSON ids, handle older integer ids. * openingd: Fail if dust and max_htlcs result in 0output commitment tx * common: Add multiplication primitives for amount_msat and amount_sat * pytest: Exercise all dust zeroreserve case * pay: If the channel_hint matches our allocation allow it * pytest: Add test for mixed zeroreserve funding * opening: Add `dev-allowdustreserve` option to opt into dust reserves * openingd: Remove dust check for reserve imposed on us * opening: Add `dev-allowdustreserve` option to opt into dust reserves * pytest: Add a zeroreserve test * openingd: Add zeroconf-no-really-zero mode * pay: Allow using a channel on equality of estimated capacity * openingd: Pass `reserve` down to openingd when funding * openingd: Wire `reserve` value through to `openingd` * plugin: Add `reserve` to `openchannel` hook result * openingd: Add `reserve` to `fundchannel` and `multifundchannel` * openingd: Add method to set absolute reserve * gha: Make the setup and build scripts exit if anything fails * pyln: Reduce dependency strictness for pyln-testing * Don't report redundant feerates to subdaemons * libplugin: mark the cmd notleak() whenever command_still_pending() called. * memleak: remove exclusions from memleak_start() * memleak: simplify API. * gossmap: optimize asserts(). * gossmap: make local_addchan create private channel_announcement in correct order. * gossmap: add functions to map index back to node/chan. * gossmap: fail to get capacity of locally-added chans (don't crash!). * jsonrpc: make error codes an enum. * build: allow DEVELOPER builds with -Og and gcc 9.4.0 * contrib/giantnode.py: populate three regtest nodes with many invoices/forwards/payments. * contrib/startup_regtest.sh: misc fixes and add destroy_ln, print usage. * Update doc/INSTALL.md * docs: Correct a command typo for Ubuntuu * connectd+: Flake/race fix for new channels * doc/install: get rid of out of date mrkd / mistune install instructions * docker: use pip install + poetry export instead of poetry update * pyln: Add grpcio and protobuf dependencies to pyln-testing * gci: Use stable rust instead of nightly * lightningd: use jcon logging for commands where possible. * lightningd: don't log JSON ids as debug, use log io. * libplugin: forget pending requests if associated command freed. * libplugin: use proper JSON id for rpc_scan(). * contrib/pyln-testing: pass through id correctly. * contrib/pyln-client: construct JSON ID correctly. * libplugin: use string ids correctly. * plugin: wire JSON id for commands which caused hooks to fire. * lightningd: wire plugin command JSON id through to plugin commands. * lightningd: trace JSON id prefixes through sendrawtx. * lightningd: derive JSONRPC ids from incoming id (append /cln:<method>#NNN). * lightningd: explicitly remember if JSON id was a string. * lightningd: use string as json req ids when we create them. * lightningd: add jsonrpc_request_start_raw instead of NULL method. * cln-rpc: allow id to be any token. * contrib/pyln-client: allow lightningd to give us non-numeric ids. * libplugin: allow lightningd to give us non-numeric ids. * lightning-cli: use cli:<method>-<pid> for all requests. * lightningd: log JSON request ids. * lightning-cli: don't consume 100% CPU if lightningd crashes. * common: helper to get id field as a string. * plugins/Makefile: regenerate plugins list when config changes. * gossipd: rename remote_addr to discovered_ip within gossipd * peer_control: getinfo show correct port on discovered IPs * lightningd: refuse to upgrade db on non-released versions by default. * plugins/bcli: load RPC password from stdin instead of an argument * rename `block_processed` to `block_added` * Add plugin notification topic "block_processed". * gossmap: don't crash if we see a duplicate channel_announce. * lightningd: fix spurious leak report. * commando: make rune alternatives a JSON array. * commando: unmangle JSON. * CCAN: update to get latest rune decode fix. * pytest: test for escapes in commando values. * Makefile: fix msggen regeneration when schemas change. * gossip_store: fix offset error * pay: Aggregate `completed_at` in `listpays` and `pay` * wallet: Set the `completed_at` timestamp when updating the status * doc: Update generated artifacts to match master * wallet: Load and value `completed_at` timestamp from DB * db: Add completed_at field to payments * bkpr-test: maybe fix race in test_bookkeeping_closing_trimmed_htlcs * bkpr: confirm that replaying the open+lock-in txs at start is ok * bkpr: migration to delete any duplicate lease_fee entries * notif: dont send balance snapshot for not yet opened channel * bkpr: failing test for bookkeeper crash * connectd: DNS Bolt7 #911 no longer EXPERIMENTAL * lightningd: more graceful shutdown. * db: fix sqlite3 code which manipulates columns. * db: fix migrations which write to db. * doc: disallow additional properties in sendcustommsg. * libplugin: allow NULL calllbacks for jsonrpc_set_datastore. * doc: escape output types (esp `short_channel_id`). * db: fix renaming/deleting cols of DBs when there are UNIQUE(x, b, c) constraints. * tools/test: fix very confused code. * lightningd: fix crash with -O3 -flto. * doc: include recent BOLT recommendation on grace period. * doc: upgrade to BOLTs 341ec844f13c0c0abc4fe849059fbb98173f9766 * doc: upgrade to BOLTs 2ecc091f3484f7a3450e7f5543ae851edd1e0761 * channeld/dualopend/lightningd: use channel_ready everywhere. * doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730 * doc: increase BOLT level to 03468e17563650fb9bfe58b2da4d1e5d28e92009 * devtools/bolt-catchup.sh: a tool to update the specs, one commit at a time. * hsmtool: remove hsm_secret passwords on cmdline support in `dumponchaindescriptors`. * offers: remove backwards-compatiblity invoice_request signatures. * offers: update to remove "vendor" and "timestamp" fields. * listchannels: don't show "htlc_maximum_msat" if channel_update didn't set it. * lightningd: remove `use_proxy_always` parameter to plugin init. * plugins: require usage for plugin APIs. * lightningd: do inline parsing for listforwards status parameter * lightningd: don't allow old listforwards arg order. * lightningd: always require "jsonrpc": "2.0" in request. * lightningd: remove double-wrapped rpc_command hook. * common: remove unused parameter "allow_deprecated" from parse_wireaddr_internal. * wire/test: neaten and complete tlv checks. * test: fix tlvs test in funding_locked tlv. * cleanup: fix mixed indentation of json_getinfo * peer_control: fix getinfo showing unannounced addr * builds: cleanup duplicate and unused code, fix spelling * build: use ubuntu 22.04 LTS * docs: Clear up Ubutu documentation * doc: document that we can build lowdown, remove from Alpine. * external: build lowdown if not already found. * external/lowdown: local import of lowdown source. * doc: always escape underscores in property names * Makefile: Revert ba7d4a8f6bab5a3d5e5832d1f9e36749e695320a (make-schema: don't include tools/fromschema.py in SHASUMS) * doc: generate correct markdown from schemas. * doc: format markdown correctly. * doc: remove mrkd requirement, add lowdown requirement. * Makefile: replace mrkd with lowdown(1). * tools: add md2man.sh tool, using lowdown. * connectd: better diagnostics on invalid gossip_store entries. * pyln-client: allow 'msat' fields to be 'null' * build, shadows: fix broken build (no shadows) * pytest: fix test_channel_state_change_history * configure: add -Wshadow=local flag. * Remove names of parameters of callbacks which confuse gcc. * Remove general shadowed variables. * ccan: update to get -Wshadow=local clean build. * Simplify poetry interactions * Use unordered list * Add Arch Linux build instructions * Add Arch Linux build instructions * pytest: Add debugging to test_gossip_store_compact_on_load ------------------------------------------------------------------- Fri Sep 23 07:28:47 UTC 2022 - marec@detebe.org - Update to version 0.12.1: * Changelog for v0.12.1 * pyln: update to v0.12.1 * remove all mrkd references from project * libplugin: handle JSON reply after command freed. * libplugin: forget pending requests if associated command freed. * contrib/pyln-client: allow lightningd to give us non-numeric ids. * build-release: configure before submodcheck * peer_control: getinfo show correct port on discovered IPs * cleanup: fix mixed indentation of json_getinfo * peer_control: fix getinfo showing unannounced addr * connectd+: Flake/race fix for new channels * gossmap: don't crash if we see a duplicate channel_announce. * gossip_store: fix offset error * bkpr-test: maybe fix race in test_bookkeeping_closing_trimmed_htlcs * bkpr: confirm that replaying the open+lock-in txs at start is ok * bkpr: migration to delete any duplicate lease_fee entries * notif: dont send balance snapshot for not yet opened channel * bkpr: failing test for bookkeeper crash * builds: cleanup duplicate and unused code, fix spelling * build: use ubuntu 22.04 LTS * connectd: better diagnostics on invalid gossip_store entries. * doc: document that we can build lowdown, remove from Alpine. * external: build lowdown if not already found. * external/lowdown: local import of lowdown source. * doc: always escape underscores in property names * Makefile: Revert ba7d4a8f6bab5a3d5e5832d1f9e36749e695320a (make-schema: don't include tools/fromschema.py in SHASUMS) * doc: generate correct markdown from schemas. * doc: format markdown correctly. * Makefile: replace mrkd with lowdown(1). * tools: add md2man.sh tool, using lowdown. * pytest: Add debugging to test_gossip_store_compact_on_load ------------------------------------------------------------------- Wed Aug 24 08:09:33 UTC 2022 - marec@detebe.org - Update to version 0.12.0: * v0.12.0 release changelog * use 'postgres' driver (not postgresql). Include plain url to guide, as link is broken on RTD. * Changelog-None * fix MD link where text and url are swapped * v0.12.0rc3 - Release Candidate numero tres * channeld: correct reversed shutdown message in billboard. * ci: Fix the Mac OS build test * tests/test_misc.py: check logs for already existing channel. * doc: Typo * lightningd/opening_control.c: Skip over channels which are already stored, and don't create new peer if it already exits. Changelog-None * doc: update usage of lightning-invoice. * Renumber hsmd_derive_secret for consistency with other hsmd messages * ld: Fix a log message assuming that the `channel->scid` was set * pytest: Reproduce a crash when we have multiple channels and 0conf * bkpr-recorder: fatal if there's an error with a database statement * sqlite3: no NULLS FIRST * signpsbt: don't crash if HSM doesn't like your psbt, just return err * signpsbt: add utxo info to inputs * psbt: dont crash when printing psbt to log * CHANGELOG: v0.12.0rc2 * Fix derived_secret, use correct size of secretstuff.derived secret * wallet: fix incorrect column-width access. * pytest: test loading lease_chan_max_msat from channel_funding_inflights * bkpr: let channel reconnect, flake? * bkpr: dont flake, wait til pay done before mining blocks * bkpr: remove duplicate log stmt * bkpr: create accounts for zero sat channels * doc: note that setchannel maxhtlc/minhtlc only apply to *forwards*. * doc: document that sendonion doesn't have to specify (but can!) the first_hop channel. * lightningd: obey first hop channel id. * pytest: don't use bogus scids for first hop of route. * lightningd: don't route private channels via real scid. * lightnind: use aliases in routehints for private channels. * pytest: test that we implement option_scid_alias privacy. * doc: fix decode schema for bolt11 routehints. * build: ignore docker/image build directories * bkpr: track channel rebalances, display in listincome * pay: Use safe list traversal when completing suspended payments * pay: Remove use-after-free bug * pay: Do not replay results against payments that were not suspended * pay: Annotate suspended payments with the groupid they mirror * lightningd: set the channel_type feature. * common/features: add channel_type definition, neaten header. * am_opener unused * docs: clarify the different way to build cln * pytest: make sure we never break channels in multhtlc test. * pytest: change multihtlc topology for simpler testing. * pytest: simplify test_onchain_multihtlc_our_unilateral/their_unilateral * pytest: clarify test_onchain_multihtlc_our_unilateral / test_onchain_multihtlc_their_unilateral * pay: Fix a memory leak when retrying getroute * lightningd: fix funding_locked in channel_opened notification. * Correct basics of backup plugin usage * CHANGELOG.md: note the Great Msat Migration! * CHANGELOG.md: neaten entries. * CHANGELOG.md: add in CHANGELOG from 0.11.2 branch. * pyln: update versions to v0.12.0 * changelog: v0.12.0rc1 release notes * devtools: if there's a message in the API call, print and exit * bkpr: make unit tests not fail if !HAVE_SQLITE3 * bkpr: insert obscure 60s pop references. * bkpr: use long-uint not size_t for time_t * bkpr/listpeeers: add lease_fees back to funds; separate out in listpeers * balance_snapshot: emit balances for channels that are awaiting_lockin * commando-rune: show warning when creating runes with no restrictions * fix doc: deschashonly * pytest: fix flake in test_gossip_timestamp_filter * connectd: control connect backoff from lightningd. * lightningd: don't consider AWAITING_UNILATERAL to be "active". * pytest: test that we don't try to reconnect in AWAITING_UNILATERAL. * bkpr: cleanup csv_safe_str * memleak: throw away things when we're done with them * nit: send_outreq returns &pending, no need to call sep command * bkpr-zeroconf: Zeroconfs will emit 'channel_proposed' event * bkpr: for zerconfs, we still wanna know you're opening a channel * doc: remove two more generated manpages. * doc: note that bookkeeper-dir and bookkeeper-db are in bookkeeper plugin. * plugins/Makefile: put bitcoin/chainparams.o in PLUGIN_COMMON_OBJS since everyone needs it. * bkpr: Add an option to set the database to something else (postgres) * test-db-provider: if postgres in tests, startup a bookkeeper db * bkpr incomestmt: properly escape things for the CSVs * bkpr csvs: koinly + cointracker only accept fees on the same line * bkpr: save invoice description data to the database and display it * test-utils: add the bolt11 invoice * bkpr: add test for bookkeeper being added after channel has closed * bkpr: fetch originating account, if exists, and make sure is populated * bkpr: correctly pass in command for `jsonrpc_request_start` * bkpr: prevent crash when updating same account all at once * bkpr: more logging * bkpr: new method, "is_external_account" * bkpr: duplicate the name, dont steal it * coin_mvt/bkpr: add "stealable" tag to stealable outputs * bkpr: make sure there's always at least on difference in blockheights * bkpr: add more data to listaccountevents printout * bkpr: check for channel resolution for any "originated" event * bkpr: print more info in bkpr-listbalances * bkpr: account name is required for bkpr-inspect * bkpr: add docs, change names to 'bkpr-*' * json-schema: allow 'required' to not be present in if switches * bkpr: add 'msat' suffix to all msat denominated fields * bkpr, elements: elements tx have one extra output for fees * bkpr: don't use a minus in a sql stmt * bkpr: pass the node_id over for channel_opens, add to account * bkpr: exclude non-wallet events in the balance snapshot * bkpr: get rid of crash in `listincome` * nit,bkpr: add a note about what the tag was to this printout * tests,bkpr: liquid fails all these for different reasons * bkpr: separate the invoice_fees from the invoice paid * wallet: resolve crash when blockheight is null * test nit: wait_for_mempool cleanup * coin_mvt: log channel_open for channels that close before they're locked * bkpr: actually fill in the current blockheight for `channelsapy` * bkpr: command to calculate some APYs/stats on channel routing fees * bkpr: add option --bookkeeper-dir * bkpr: add 'start_time' and 'end_time' to `listincome` * bkpr: add `dumpincomecsv` command * bkpr: add a 'consolidate-fees' flag to the income stmt * bkpr: 'to_miner' spends are considered terminal * bkpr: mark external deposits (withdraws) via blockheight when confirmed * tests: move 'bookkeeper' centric tests to their own file * bkpr: add a 'listincome' event * bkpr: add timestamp filters to event lists * bkpr: move json_to functions to respective type files * bkpr: properly account for onchain fees for channel closes * bkpr: use chain_closed count to do mark account closed * bkpr: save closed_count for account, when known * bkpr: annotate an account with the block at which it's been resolved * bkpr: add an 'inspect' command to the bookkeeper * bkpr: once we get channel_opens, we might need to update the fee records * bkpr: account for pushed amounts and record the output_value * bkpr: listbalances, rename 'account_id' => 'account' * bkpr: listbalances, skip the external account * bkpr: cleanup wallet fee entries if decide they belong to a channel * bkpr: don't try to add fees if this tx didn't touch any accts * bkpr: dont count fees for channel closes if we're not the opener * bkpr: update tests for wallet/external onchain fees * bkpr: parse the 'originating_account' field, save to event * bkpr: invert channel + chain event printouts * bkpr: use tags not str for tag originations * bkpr: add a new command `listaccountevents` * bkpr: if we're missing info about an account, add in journal entry * bkpr: put the account name on the event * bkpr: use pointer for payment_id for channel events * bkpr: turns out these fields are optional * bkpr: have onchain_fee records be write-only, don't update in place * bkpr: add journal entry for offset account balances; report listbalances * bkpr: wire up our chain fee accting to chain event reception * patch db-fatal-plugin_err.patch * bkpr: add bookkeeper to PLUGINS list * bkpr: create onchain fee records for events * bkpr: tests for db crud * bookkeeper: initial crud (no tests) * bkpr-tests: first test of plugin bkpr database * bkpr: first attempt at database code for accounting * bookkeep: first commit, stub of new plugin * sql: use last " as " to find name token for column * doc: fix bolt 12 link (it's not in master), update bolt 11 to new "bolts" repo. * pytest: fix flake in test_wumbo_channels * v2open: dont rely on ordering of interprocess messages * flake: permit test_v2_open_sigs_restart_while_dead to succeed/fail * valgrind: rm ref to cmd when cmd is free'd * tests: valgrind barfing on uninitialized value * tests: local flake fix; l1 was waiting too long to reconnect * msggen: Use tempfile + rename to make changes to .msggen.json atomic * pytest: fix test_commando_rune flake. * pytest: fix flake in test_feerates * pytest: fix test_commando_stress * commando: free incmd as soon as we use it. * pytest: fix flake in test_channel_persistence w/ TEST_CHECK_DBSTMTS * lightningd: ignore default if it's a literal 'null' JSON token. * pytest: set dblog-file when adding the dblog plugin (TEST_CHECK_DBSTMTS=1) * connectd: two logging cleanups. * decode: fix crash when decoding invalid rune. * lightningd: fix fatal() log message in log. * CI: fix CI scripts to fail if a command fails. * fuzz: fix fuzzing compilation. * tests: flake fix; l1 was waiting too long to reconnect * nit: speedup retry timeout test * tests:redirect output, so test log passes * test-flake: dont let `l1` send their unilateral tx * msat: cleanup msat outputs for apis * contrib startup-regtest: turn off deprecated apis, update deprecated * pytest: fix flake in test_zeroconf_forward * hsmtool: implement checkhsm. * commando: add stress test, fix memleak report. * commando: limit to 16 partially-received incoming commands at a time. * pytest: don't redirect stderr by default. * commando: fix crash when rune is completely bogus. * libplugin: don't call callbacks if cmd completed before response. * doc: Spell out the reprobuild instructions for each distro * rs: Add Cargo.lock for reproducible builds * rs: Strip binaries when compiling them for release * py: Update poetry.lock using poetry update * repro: Update ubuntu jammy reprobuild * reprobuild: Add Rust compiler to repro build docker images * dualopend: plug memleak. * pytest: disable autoreconnect in test_rbf_reconnect_tx_construct * pyln-testing: Add listinvoices to grpc shim * schema: Add missing `mindepth` argument to `fundchannel` * pyln: Ignore generated files when linting * cln-plugin: Fix plugin dependencies * cln-plugin: Add metadata required by crates.io * pyln-testing: Add `invoice` RPC method * msggen: Ignore `state_changes` in grpc2py * cln-rpc: Add mindepth after rebase on `master` * pyln-testing: Add a couple of methods used in tests * schema: `minconf` should be an integer (u32) not a float (number) * msggen: Add `stop` method to generators * cln-plugin: Notify waiting tasks if the lightningd connection closes * cln-grpc: Skip serializing fields when Option<Vec<T>> is empty too * msggen: Add fundchannel request * pytest: Add a new RPC interface to talk to grpc * msggen: Add a new generator for grpc -> python converter * pytest: Move generated grpc bindings to pyln-testing * msggen: Do not override method names when loading Schema * lightningd: expand exit codes for various failures. * lightningd: test that hsm_secret is as expected, at startup. * rust: upgrade model with new checkmessage requirements * test: disable schema check of `checkmessage` with deprecated API * rpc: improve error format * rpc: checkmessage return an error if pubkey is not found * Add rune and commando to gitignore. Changelog-None: Small fix * channeld: exit after we send an error at lightningd's request. * pytest: fix flake in test_channel_lease_post_expiry * pytest: work around dualopend issue. * lightningd: reintroduce "slow connect" logic. * lightningd: drive all reconnections out of disconnections. * lightningd: don't explicitly tell connectd to disconnect, have it do it on sending error/warning. * pytest: don't assume disconnect finished atomically, and suppress interfering redirects. * multifundchannel: fix race where we restart fundchannel. * lightningd: don't forget peer if it's still connected. * lightningd: provide peer address for reconnect if connect fails. * gossipd: downgrade broken message that can actually happen. * connectd: spelling and typo fixes. * connectd: give connections a chance to drain when lightningd says to disconnect, or peer disconnects. * common/gossip_store: fix leak on partial read. * lightningd: don't kill subds immediately on disconnect. * connectd: don't move parent while we're being freed. * pytest: enable race test. * connectd: add counters to each peer connection. * lightningd: hand fds to connectd, not receive them from connectd. * common/daemon_conn: add function to read an fd. * lightningd: make "is peer connected" a tristate. * lightningd: remove delay on succeeding connect. * dualopend: fix memleak report. * connectd: don't io_halfclose() * pytest: add another connection stress test, using multiple channels (bug #5254) * pytest: test to reproduce "channeld: sent ERROR bad reestablish revocation_number: 0 vs 3" * connectd: remove the redundant "already connected" logic. * connectd: remove reconnection logic. * connectd: don't let peer close take forever. * connectd: release peer soon as lightingd tells us. * connectd: send cleanup messages however peer is freed. * lightningd: don't tell connectd to disconnect peer if it told us. * lightningd: clean up channels when connectd says peer is gone. * connectd: put peer into "draining" mode when we want to close it. * connectd: fix subd tal parents. * lightningd: remove 'connected' flag from channel structure. * lightningd: fix dev-memleak crash on unown unconfirmed channels. * lightningd: set cid correctly in peer->uncommitted_channel. * startup_regtest: add connect helper * startup_regtest: add experimental-offers * lightningd: reduce log level for remote address reporting. * options: set DNS port to network default if not specified * pytest: add xfail test to show DNS w/o port issue * docs: fix contrib/ docs * Add instructions for checking out a release tag * fix typo in commando documentation * decode: support decoding runes. * commando: add rate for maximum successful rune use per minute. * commando: add support for parameters by array, parameter count. * doc: document commando and commando-rune. * commando: require runes for operation. * commando: don't look at messages *at all* unless they've created a rune. * commando: add commando-rune command. * devtools/rune: simple decode tool. * commando: runes infrastructure. * commando: correctly reflect error data field. * commando: support commands larger than 64k. * plugins/commando: basic commando plugin (no runes yet). * libplugin: datastore helpers. * libplugin: jsonrpc_request_whole_object_start() for more custom request handling. * common/json_stream: make json_add_jsonstr take a length. * ccan: upgrade to get ccan/runes. * Makefile: use a library archive for CCAN * plugin: autoclean fix double free when re-enable, remove xfail mark from test_ * pyln-testing: print content of errlog file when _some_ node failed unexpected * test: test_invoices add a stress test * common/json_stream.c: provide explicit json_add_primitive_fmt and json_add_str_fmt routines. * lightningd/json.h: remove. * common: clean up json routine locations. * common: remove json_tok. * plugins/spender/multifundchannel: remove json_tok. * lightningd: remove gratuitous param_tok from help and config. * lightningd/connect_control: remove param_tok from connect. * lightningd/closing_control: remove param_tok(). * common/json_stream: remove useless attempt at oom handling. * lightningd: remove getsharedsecret. * makesecret: change info_hex arg to simply "hex" to match datastore command. * tests: massively speed up our wait for enormous feerates * coin-moves: only log htlc_timeout pair for penalty txs * connectd: optimize requests for "recent" gossip. * common: add ability for gossip_store to track by timestamp. * devtools: add --print-timestamps to dump-gossipstore. * connectd: optimize case where peer doesn't want gossip. * msggen: Update generated files to add Listpeers.peers[].remote_addr * contrib: Remove obsolete libhsmd_python class * make: Add msggen generated files to check-gen-updated target * Use correct naming * plugin: add check on the type json object during the IO message handling * notify: `channel_state_changed` now receives notice when channel opens * lightningd: re-transmit all closing transactions on startup. * pytest: test (failing) that we rexmit closing transactions on startup. * tests: Add tests for the RPCs Changelog-Added: Static channel backup, to enable smooth fund recovery in case of complete data loss * doc: Add documentation for new RPCs and a FIXME: in fromschema.py * Plugin: Add new plugin for SCB, Updated makefile and gitignore as well * lightningd: Add RPC for populating DB with stub channels and set an error on reconnecting * tools/gen: Always return bool! * lightningd/peer_control: Add RPC for fetching scb for all the in-memory channels. * channel: Add struct scb_chan in channel and making last tx optional. * common: Add scb_wire for serializing the static_chan_backup * hsmd: Create derive_secret and makesecret RPC for deriving pseudorandom keys from HSM * make-schema: don't include tools/fromschema.py in SHASUMS * gossipd: fix of gossip v10 channel removal handling * dual-fund: Fail if you try to buy a liquidity ad w/o dualfunding on * test: run-gossmap update test store instructions * doc: mention ip discovery only active when no addresses are announced * gossipd: only use IP discovery if no addresses are announced * gossipd: nit: update a misleading comment * sendpay: allow route to contain both amount_msat and (deprecated) msatoshi. * pytest: test that we allow both msatoshi and amount_msat is route for sendpay. * Revert "pytest: fix test_gossip_no_backtalk flake." * pytest: fix test_gossip_no_empty_announcements flake. * gossipd: fix crash from gossip_store v10 changes * lightningd: save outgoing information for more forwards. * wallet: allow saving forwarding scid even if we don't have amount. * lightningd: allow outgoing_scid without outgoing amount. * pytest: test that listforwards gives as much outgoing information as possible. * docs: Add reference to issue 5366 for stuck channel in awaiting * doc: Add two entries to the FaQ about channel management * tests: Fix the canned gossmap * git: Ignore arm64-darwin external build directory * mac: Ensure that we compile the configurator with the M1 libs * make: Add macos M1 support * cln_plugin: Configure "dynamic" field in "getmanifest" message * json: add "dynamic" field to plugin list * doc: improve/update lightning-plugin, PLUGINS.md and lightning-listconfigs * lightningd: Make sure plugins don't register the same option "name" * pytest: test plugin does not register same option "name" * lightningd: don't increment plugin state to NEEDS_INIT when error in getmanifest * lightningd: convert plugin->cmd to absolute path, fail plugin early when non-existent * ccan: update to latest htable fixes, and update gossmap to meet new assertions. * add generated man page * Update lightning-newaddr.7.md * lightningd: fire watches on blocks before telling lightningd. * lightningd: fix close of ancient all-dust channel. * invoice: turn assert failure into more informative log_broken message. * pytest: fix flake in test_node_reannounce * connectd: don't insist on ping replies when other traffic is flowing. * channeld: fix uninitializes scid alias for dual-funding. * pytest: fix test_gossip_no_backtalk flake. * lightningd: mark subd->conn notleak() properly in transition. * lightningd: make log-prefix actually prepend all log messages as expected. * options: print empty options properly. * pytest: restore prefix to logging. * pytest: fix flake in test_option_upfront_shutdown_script * options: let log-level subsystem filter also cover nodeid. * update-mocks: handle missing deprecated_apis. * df tests: node still correctly picks up new tx if broadcast fails * lnprototest: updating and patch lnprototest * df: check mempool/block for funding output on broadcast fail * peer_control: Fix check_funding_details assert * Inline URL * Fix broken link * doc: Fix generated `listfonfigs` man-page * gossipd: store and index most recent and last non-rate-limited gossip * pytest: test_gossip_ratelimit checks routing graph and squelch * gossipd: make use of new ratelimit bit in gossip_store length mask * gossip_store: add flag for spam gossip, update to v10 * ld: Use the local alias in the `htlc_accepted` hook * zeroconf: Add channel_type variant support * features: Add function to unset a featurebit * pyln-testing: Add `gossip_store` parser to testing framework * zeroconf: Announce the channel with the real scid as well as aliases * channeld: Track the funding depth while awaiting lockin * ld: Use the local alias when reporting failures with zeroconf * channeld: Explicitly use the first commitment point on reconnect * pytest: Add test for zeroconf channels transitioning to be public * pytest: Add test for forwarding over zeroconf channels * pytest: Add a test for direct payment over zeroconf channels * db: Store the local alias for forwarded incoming payments * ld: Add function to retrieve either the scid or the local alias * pay: Populate the channel hints with either the scid or the alias * jsonrpc: Include the direction also if we have an alias * zeroconf: Trigger coin_movement on first real confirmation * db: Store the alias if that's all we got in a forward * ld: Consider local aliases when forwarding * invoice: Consider aliases too when selecting routehints * ld: Allow lockin despite not having a scid yet * jsonrpc: Add aliases to `listpeers` result * gossip: Add both channel directions with their respective alias * gossipd: Use the remote alias if no real scid is known * channeld: On funding_locked, remember either alias or real scid * channeld: Remember remote alias for channel announcements and update * channeld: Send a depth=0 notification when channeld starts up * zeroconf: Wire the aliases through `channeld` * zeroconf: Add alias_remote and alias_local to channel and DB * wire: Add funding_locked tlv patch from PR lightning/bolts#910 * jsonrpc: Add option_zeroconf handling to `listpeers` * pytest: Test the `mindepth` customizations of `fundchannel` and hook * plugin: Allow plugins to customize the mindepth in accept_channel * openingd: If we have negotiated zeroconf we use our mindepth * bolt: Reflect the zeroconf featurebits in code * jsonrpc: Add `mindepth` argument to fundchannel and multifundchannel * json: Add parser for u32 params * valgrind: ingore plugin build with rust * CI: re-enable checks, by changing errant tab back to spaces. * Update README.md * Add README.md * No funding_wscript arg in initial_commit_tx * CI: make sure *someone* runs check-units under valgrind! * connected/test: fix memleak in test. * connectd: don't keep around more than one old connection. * lightningd: fix crash on rapid reconnect. * connectd: avoid use-after-free upon multiple reconnections by a peer * connectd: don't start connecting in parallel in peer_conn_closed. * doc: document the [] IPv6 address hack. * devtools: revert changes and make sure chainparams always set. * test: introduce the default port function and remove hard coded port * lightnind: remove`DEFAULT_PORT` global definition * docs: update the docs in according with the new code * lightningd: change the regtest default port according with the tests * lightningd: use the standard port derivation in connect command * pytest: fix test_multifunding_feerates * pytest: fix flake in test_node_reannounce * pytest: fix flake in test_query_short_channel_id * pytest: fix gossipwith flake harder. * lightningd: add more information should subd send wrong message. * lightningd: track weird CI crash in test_important_plugin * pytest: don't assume that join_nodes needs only check ends. * pytest: fix test_channel_lease_post_expiry flake. * lightningd: fix transient leak report when openingd shutting down. * pytest: allow more time for test_waitblockheight !DEVELOPER. * lightningd: don't run off end of buffer if db_hook returns nonsense. * pytest: fix timeout in test_option_types * pyln-testing: increase default daemon.wait() timeout. * pyln-testing: restore proper streaming behaviour for lightningd. * pyln-testing: use files for stdout and stderr, not threads. * pytest: fix test_ping_timeout flake. * lightningd: don't print nasty message when onchaind fails partially-failed HTLC * pytest: test to demonstrate BROKEN message on onchaind htlc close. * gossipd: fix gossmap race. * gossmap: handle case where private channel turns into public. * pytest: wait for stderr, rather than asserting. * pytest: fix flake in test_wallet.py::test_hsm_secret_encryption * pytest: fix race in test_multichan * pytest: try to fix bitcoind socket timeout in test_channel_lease_unilat_closes * pytest: fix port allocation race when nodes restart. * connectd: fix debug message on bind fail. * pyln-testing: try harder if cleaning directory fails. * pytest: fix flake in test_htlc_too_dusty_outgoing * pytest: fix flakes in test_gossip_query_channel_range * pytest: fix flake in test_gossip_timestamp_filter * pytest: Remove all trace of python's "flaky" module. * lightningd: multiple log-file options allow more than one log output. * pyln: Fix relative path dependencies when publishing to PyPI * common: downgrade LND 'internal error' properly. * common: downgrade "internal error" errors from lnd. * git: Ignore some more generated files and the cln-grpc plugin * signmessage: improve the UX of the rpc command when zbase is not a valid one * Update hsm error formatting. * common/gossip_store: optimize case where entries are filtered out. * pytest: add (failing) test for db with old Tor v2 address. * lightningd: remove overzealous assertion. * Remove native-tls dependency from cln-rpc * doc: Tor needs CookieAuthFile set * cln-plugin: Make the proxy-related configuration Option<> * cln-plugin: Configuration struct * cln-plugin: Add unittest for parsing "init" message * cln-plugin: Save "configuration" from "init" method * clnrs: Implement backwards compatible mode for Amount as bare u64 * doc/PLUGINS: update to remove deprecated fields, formats. * JSON: deprecate printing msat fields as strings. * lease_rates: prepare for msats fields as raw numbers. * plugins/funder: fix parameter parsing. * funder: prepare for msats fields as raw numbers. * libplugin: don't be so strict on msat fields. * fundchannel, txprepare, multiwithdraw: don't assume "excess_msat" will have "msat" appended. * lightningd: deprecate `msatoshi` in `sendpay` `route`. * JSON: don't print deprecated amount fields any more * pytest: don't use deprecated amount fields * pyln-client: convert every _msat field to Millisatoshi * common: enforce that msat fields are called "xxx_msat". * fetchinvoice: amount_msat not msat. * pay: use amount_msat for amount report inside attempts. * lightningd: use amount_msat not amount in htlc_accepted_hook. * lightningd: change `msatoshi` args to `amount_msat`. * coinmvt: don't use msats in fields not called "_msat". * lightningd: don't add null for unset plugin options. * common: add msat to sat convert helper. * json: fix up msat amounts in non-_msat fields. * common/param: support renaming options using "|<deprecatedname>". * lightningd: fix outgoing IO logging for JSONRPC. * contrib/pylightning: remove lightning-pay helper. * tests: Update expected log prefix length to match truncated log * log: Add termination to log level * log: Add termination to prefix log * Warning added to PLUGIN documentation re: `stdin` * expose short_channel_id and htlc id to htlc_accepted. * docs: Fix manpage title * Fix typo and convert paragraph to a list * bitcoin: add to check-source-bolt, and (minor) quotes fixup. * bitcoin: fix header order for make check-source. * Mistaken default directory in plugin docs * jsonrpc: adds optional `remote_addr` to listpeers * connectd: use dev_allow_localhost for remote_addr testing * jsonrpc: adds dynamicaly detected IP addresses to `getinfo` * wireaddr: moves wireaddr_arr_contains to wireaddr.h * connect: adds nodeid to remote_addr log message * connectd: prefer IPv6 when available * devtools: Add fund_ln command to startup_regtest.sh * lightning-cli plugin start - Assume default relative path * connectd: shrink max filter size. * connectd: optimize gossip_rcvd_filter. * connectd: use is_msg_gossip_broadcast into gossip_rcvd_filter.c * channeld: take over gossip_rcvd_filter.c and is_msg_gossip_broadcast. * connectd: restore gossip filter aging. * psbt: use DER encoded + sighash byte for PSBT_IN_PARTIAL_SIG items * route: Do not require both directions to be active * topo: Fix issue considering the wrong direction for capacity limits * Fixes #5276 by using as docker base image debian bullseye instead of buster * cln-grpc: API updates after 8dd51d127fff01b9302009906dcbdc83ea3b6548 * Makefile: update to include fix for remote_addr generation. * Makefile: update bolts to include remote_pubkey change. * Makefile: update to BOLTs without zlib. * gossipd: neaten code now we don't have to prepend prefix after. * gossipd: don't ever use zlib compression on gossip. * docker: Add rust build support to Dockerfile * Change lightning-rfc to bolts post repo move * More explanation of bolt csv regeneration * pyln-spec: upgrade to the last bolt version * lightningd: remove various deprecated JSON fields. * connectd: remove enable-autotor-v2-mode option * Restore description of "reserved" field for listfunds * README: add links to discord + telegram so people can easily find us! * CHANGELOG.md: release notes for 0.11.1. * connectd: remove assert which can trigger. * connectd: fix accidental handling of old reconnections. * docker: Update name from c-lightning to Core-Lightning * docs: Update references from c-lightning to Core-Lightning * doc: Update lightning.readthedocs.org project name after rebrand * ignore generic binary * gh: Add @cdecker as codeowner for the Rust artifacts * gh: Remove @wythe from the codeowners * msggen: adding example and fixes typo * msggen: introduce chain of responsibility pattern to make msggen extensible * pytest: websocket test can tolerate ipv6 address * external/Makefile: fix `submodcheck` for sudo make install and modern git. * "removed asterisks, in case that's what made build fail" * Slight problem with `./configure help` * pyln: Update the makefile to use poetry for publishing * Makefile: stumble along if git does not work. * repro: Update repro dockerfiles and instructions * repro: Add reprobuild support for ubuntu:22.04 ------------------------------------------------------------------- Mon Jun 27 07:27:21 UTC 2022 - marec@detebe.org - Update to version 0.11.2: * connectd: don't start connecting in parallel in peer_conn_closed. * common: downgrade LND 'internal error' properly. * lightningd: remove overzealous assertion. * lightningd: handle old Tor version 2 addresses in db without crashing. * CHANGELOG.md: release notes for 0.11.2. * common: downgrade "internal error" errors from lnd. * common/gossip_store: optimize case where entries are filtered out. * connectd: avoid use-after-free upon multiple reconnections by a peer * lightningd: fix crash on rapid reconnect. * connectd: don't keep around more than one old connection. * topo: Fix issue considering the wrong direction for capacity limits * route: Do not require both directions to be active * connectd: restore gossip filter aging. ------------------------------------------------------------------- Mon May 16 15:29:18 UTC 2022 - marec@detebe.org - Update to version 0.11.1: * CHANGELOG.md: release notes for 0.11.1. * connectd: remove assert which can trigger. * connectd: fix accidental handling of old reconnections. ------------------------------------------------------------------- Tue Apr 26 08:37:30 UTC 2022 - marec@detebe.org - Update to version 0.11.0.1: * doc: fix getinfo non-ASCII chars for older mrkd. * v0.11.0.1: final fixes for release build ------------------------------------------------------------------- Tue Apr 26 08:11:12 UTC 2022 - marec@detebe.org - Update to version 0.11.0: * CHANGELOG.md: Looks like rc5 was the ticket! * CHANGELOG: rc5 * lightningd: don't crash listpeers if we're opening DF channel. * pytest: add test for crashing listpeers. * Fix miner bug in rust client generator. * v0.11.0rc4 * connectd: disable advertizement of WEBSOCKET addresses. * connectd: remove a noisy debug msg, fix name typo. * pytest: test that node_announcement regeneration works as expected. * gossipd: fix daily node_announcement regeneration. * connectd: send our own gossip, even if peer hasn't sent timestamp_filter. * common/gossip_store: add flag to *only* fetch "push"-marked messages. * gossipd: revert useless bf8cb640b71e15f935ced19934150c921c025981. * corrected typo in description * lightningd: fix overzealous check in htlc_out_check: * CI: fix bsd workflow. * connectd: avoid use-after-free on reconnect with remote_addr. * lightningd: remove over-zealous assert. * CHANGELOG.md: rc3 * dualopend: handle dev_memleak during RBF. * pytest: don't add extra cln-grpc instance now plugin is builtin * Make sure DEFAULT_TARGETS includes rust testing examples. * configure: require rustfmt as well, for rust support. * doc/Makefile: make check-manpages require the cln-grpc plugin. * Makefile: make sure ALL_PROGRAMS doesn't include cln-grpc plugin. * doc: document the grpc-port option, now cln-grpc is a built-in. * plugins: fix cln-grpc to work as a builtin when we're run in-tree. * plugins: add grpc-plugin to PLUGINS, so it gets started and installed. * libwally: update to v0.8.5 * Changed external/libwally-core to test_build_fix * cln-plugin: Handle --help invocations better * use full path to reference `lightningd.c` * cln-grpc: Add midstate between configuration and replying to init * cln-grpc: Do not exit if grpc-port is not set * msggen: Add pay descriptions and maxfee * cln-grpc: Do not log an error when called with `--help` * release: v0.11.0rc2 * docs: use poetry in the osx installation process * Travis is no longer used * there are four tests * doc: update c-lightning to Core Lightning almost everywhere. * CHANGELOG.md: fix typos * lightning-cli: document the real argument handling (for special effects). * doc: note how to do Rust builds on Ubuntu, at least. * Update INSTALL.md * build: bump zlib dependency * wallet: don't get so upset on orphaned HTLC. * CHANGELOG.md: update for final PR merge. * changelog: remove duplicate pr links * changelog: sort pr links * v0.11.0rc1 * pytest: test custom config options for fees and htlc limits * pytest: option disable-ip-discovery * doc: htlc_minimum_msat htlc_maximum_msat announce_discovered_ip * config: adds htlc_minimum_msat htlc_maximum_msat announce_discovered_ip * pytest: test that deduplication for `bolt11` works as expected. * plugins/pay: always include `bolt11` (and `description`) in listpays. * pay/sendpay: also store description in case bolt11 uses description_hash. * pay: require description if bolt11 only has hash. * pay: add absolute "maxfee" parameter. * doc: document `pay` localofferid field. * plugins/pay: remove legacypay. * pytest: fix invalid invoice() call. * cln-grpc: Final generation of derived files * msggen: Don't assume we have properties in the schema * pytest: fix RUST=0 pytest. * cln-rpc: Yet more RPC methods being mapped * cln-grpc: Add conversion test for listpeers * cln-rpc: Test that we forward errors correctly * doc: Annotate secrets and hashes in the JSON Schemas * cln-rpc: Add Sha256 and Secret types * cln-rpc: Log requests and responses * cln-rpc: Skip the HTLC state for now * cln-rpc: The JSON `number` type is a float not an int * cln-rpc: Move tests into separate file * cln-rpc: Make Pubkey and ShortChannelId proper types * msggen: Add RoutehintList as a primitive * msggen: Add model-side overrides * msggen: Add parser for "oneOf" * pyln-testing: check the request schemas. * doc/schemas: fixes for request schemas. * pyln-testing: add new schema types. * pytest: don't hand a string for an integer value. * pyln-testing: add dev-invoice. * pyln-testing: add dev-pay. * doc: set additionalProperties to false in all request schemas. * lightningd: deprecate invoice expiry suffixes. * fundpsbt: deprecate reserve=true/false usage. * pytest: fix if no rust installed. * cln-grpc: Map AmountOrAll and AmountOrAny * cln-rpc: Map PSBT and TX methods * cln-rpc: Add `OutputDesc` for `{addr: amt}` style arguments * cln-rpc: Map feerates and backfill methods using it * grpc: Map `sendonion` and `createonion` * msggen: Add custom mappings for JSON -> grpc conversions * cln-rpc: Add invoice RPC methods * cln-grpc: Add the `connect` method * msggen: Wrap field numberings in the message type * pytest: fix flake in test_multichan. * bolt11: reorder invoice production to match test vectors. * doc: update BOLTs to latest master. * plugins/pay: send payment_metadata if provided in invoice. * sendpay: add payment_metadata argument. * lightningd: metadata received support (log and decline). * bolt11: support payment_metadata. * doc: big BOLT update to incorporate warnings language. * shutdown: don't allow shutdown to p2pkh or p2sh addresses for anchor outputs. * doc: update BOLT references part1, including bolt11 test vectors with payment_secret. * specs: update CSV files. * wire/tlvstream: const. * connectd: take dev-suppress-gossip from gossipd. * lightningd: add "style" to listforwards. * lightningd: Restore forwarding of legacy onions. * pytest: test legacy onion acceptance. * doc: add partid to sendonion schema. * onchaind: fix htlc_stub organization. * channeld: log more information about restoring HTLCs. * lightningd: extra sanity checks and rescue attempts for missing HTLCs. * lightningd: add extra debugging for weird onchain htlc interactions. * pytest: make test_onchain_close_upstream more accurately reflect report. * pytest: simple "does onchaind on missing HTLC cause upstream failure" test. * lightningd: add --dev-no-ping-timer to avoid ping response timeouts. * docker-ci: fixed script path inside the entry point * ci: adding pytest parameter to tracing lnprotetest * docker-ci: upgrade pip version in the bash script * ci: migrate the protocol test on docker image * plugins/pay: don't crash on malformed time. * unit tests: don't crash if !HAVE_SQLITE3. * tests: add test that pay gets update correct. * cln-grpc: Set cln-grpc version to 0.0.1 until it's complete * pytest: fix flake due to cln-grpc starting before "public key" message. * cln-grpc: Do not start unless a `grpc-port` is specified * cln-grpc: Rename the grpc plugin and binary * py: Add grpcio-tools for the cln-grpc-plugin * grpc-plugin: Make the grpc port to listen on configurable * cln: Add grpc port as configurable option * cln-grpc: Add the `connect` method * pytest: Add a test for incorrect credentials * grpc-plugin: Generate mTLS certificates and use them in grpc * pytest: Add a test for the grpc plugin * cln-grpc-plugin: Add basic grpc-plugin * make: Generate grpc bindings if we want to test with rust enabled * make: Add a hook for us to depend on generated files for tests * cln-grpc-plugin: Add scaffolding for the cln-grpc-plugin * subd: make channel/peer own the subd. * py: Update the PyPI job to use poetry and correct versions * connectd: restore obs2 onion support. * plugins/topology: don't get upset if we hit deleted channel. * rust: fixed compiler warning in the example * delinvoice: allow desconly arg to only remove the description. * invoice: add deschashonly parameter. * doc/schemas: handle delinvoice of paid invoice. * pay: fix crash on invalid onion in reply in some cases. * devtools/decodemsg: add --list-tlvs * devtools/decodemsg: don't require --onion for onion tlvs, fail if unknown tlvname. * common: add const to accepted_extra_tlvs arg. * common/tlvstream: allow fromwire_tlv to accept *any* unknown type. * common/tlvstream: put TLV checking back in the generic function. * tools/generate_wire.py: make functions allocate the TLV. * wire: generate printwire_ routines for bolt12 CSV. * plugins/fetchinvoice: remove obsolete string-based API. * tools/generate-wire.py: make (and expose) individual TLV print functions. * decodemsg: if decode fails, status 1. * tools/generate-wire.py: have printwire routines do demarshal. * tools/generate-wire.py: simplify printwire routines, fix ... handling. * tools/generate_wire.py: allow generated C files to have includes added. * tools/generate_wire.py: tlvs should start with tlv_ * offers: neater response on malformed invoice. * onion_messages: remove obs2 support. * devtools: remove blindedpath tool. * pytest: test for compat code. * offers: import latest variant from draft, part 2. * offers: import latest variant from draft. * devtools/decodemsg: fix printing of wireaddr. * common/onion: always fill in failtlvtype and failtlvpos parameters. * test: avoid testing for secret_eq under CI. * Add more warnings to fundchannel_start * plugins/bcli: fix false memleak detection. * lightningd: allow 'style' 'legacy' for clboss compat (with deprecated_apis). * pytest: fix flake in test_pay.py::test_setchannel_state * cli: make the command line more user friendly. * cli: check if the pass and the confirmation pass match * pytest: fix bad gossip flake in test_multifunding_v1_v2_mixed * lightningd: free peer->uc when openingd fails. * pytest: fix flake in test_connection.py::test_opener_feerate_reconnect * pytest: fix test_connection.py::test_funding_close_upfront fake * gossipd: don't use connectd daemon_conn after it shuts down. * memleak: fix double-free if we timeout. * pytest: fix flake in test_funding_fail * pytest: add reconnect and restart to test_multichan. * connect: delay return until all subds ready. * lightningd: clean up peer connection handling a little. * pytest: test closing one multichannel works as expected. * lightningd: don't tell connectd to discard peer unless no subds left. * lightningd: use a better channel if available to next hop. * lightningd: remove checks which prevent us from opening multiple channels. * lightningd: remove (most) functions to search channels by status. * lightningd: don't assume a single channel per peer. * lightningd: add find_channel_by_scid * lightningd: make setchannelfee handle multiple channels per peer. * lightningd: associate connect commands with peer, not channel. * lightningd: rename activate_peers() to setup_peers(). * lightningd: remove some "single active channel" assumptions. * lightningd: use channel_id when a peer is activated. * connectd: key multiple subds by channel_id, use for lookup. * connectd: track the channel_id of each stream to/from peer. * connectd: tell lightningd the channel_id when we give it the active peer. * lightningd: always tell connectd the channel id. * lightningd: handle reestablish directly from connectd. * connectd: hold peer until we're interested. * lightningd: move notification of disconnect into when we hear from connectd. * connectd: tell lightningd when disconnect is complete. * connectd: handle connect vs closing race better. * openingd: disconnect from peer when an error occurs. * connectd: fix confusing names. * pytest: add test to check we notice remote disconnects. * lightningd: clean up connect code. * connectd: prepare for multiple subd connections. * connectd: clean up decrypted packet memory handling. * connectd: make sure we do IO logging on final_msg output. * lightningd: add explicit "connected" flag. * spender: free up vars to avoid transient false leak reports. * pyln: Remove two more occurences of non-None default args * py: Make the pyln dependencies editable * py: Disentangle pyln dependencies * pay: fix leak detect on shadow route. * setchannel: don't let them advertize htlc_maximum_msat larger than capacity. * setchannel: add minhtlc * lightningd: check htlc_maximum_msat of channels for routehints. * pytest: add checks that pay and getroute respect htlc_maximum_msat. * devtools/fp16: conversion tool for fp16 format. * lightningd: deprecate setchannelfee, use setchannel in tests. * lightningd: new setchannel command. * channeld: generalize specific_feerates mesage into config_channel. * listpeers: show maximum_htlc_out_msat. * lightningd: enforce htlc_maximum_msat. * channeld: get htlc_maximum_msat from lightningd. * lightningd: store htlc_maximum_msat for channel in the db. * connectd: use closefrom for faster forking, and ignore children * psbt: fix dual-funding memleak. * memleak: handle libwally allocations better. * hsmd: Add validate_commitment_tx * lnprototest: upgrade the lnprototest version * memleak: make sure we catch children which are also "notleak". * fix: Remove \ from help documentation * doc: update ubuntu install instructions * lightningd: remove support for legacy onion format. * pytest: convert test_sendonion_rpc to modern TLV onion. * pytest: Remove onion test vectors containing legacy onions. * lightningd: opt_var_onion is now a compulsory feature. * hsmd: Add fields to hsmd_sign_{,remote_}commitment_tx for validating signers * docker: Build the docker image with psql support * doc: fix sendonionmessage documentation. * cln-rpc: Fixed two minor compiler warnings * msggen: Add meta file to keep the field numbers stable * pyln: Change the default argument to `exclude` in `pay` * websocketd: fix random failures by blocking stdin reads * Goofy af tor.md systemd syntax update (#5059) * doc: update faq and docs for IP discovery * db: small code cleanup * lightningd: do not use remote_addr for always_use_proxy * chore: remove EXPERIMENTAL for rfc #917 remote_addr * test: connectd netaddress * pytest: IP address discovery updates node_announcement * gossipd: send updated node_annoucement remote_addr * lightningd: store recently reported remote_addr * wireaddr: adds wireaddr_eq_without_port and wireaddr_cmp_type * connectd: ignore private remote_addr on non-DEVELOPER builds * pyln-proto: fix port typo in example script * chore: fix typo announcable -> announceable * doc: remove ambiguity for functionality enabled after a release version * Update LICENSE * hsmd: Add wallet index metadata to existing messages * hsmd: Augment call to hsmd_sign_withdrawal_tx with wallet index metadata * hsmd: Add PSBT keypath utility functions * hsmd: Add redeemscripts to anchor outputs * cln-plugin: Ensure we cleanly shut down when we lose the master conn * cln-plugin: Make notification handlers asynchronous * cln-plugin: Make hooks asynchronous * cln-plugin: Make rpcmethod handlers async * cln-plugin: Add notification subscriptions and hooks to the plugins * cln-plugin: Add support for synchronous RPC methods * cln-plugin: Rework the plugin library using a Builder * pytest: Use valgrind target suppressions instead of skipping tests * pytest: Mark Rust-dependent tests as skipped with VALGRIND * cln-plugin: Populate the options when we get an `init` call * cln-plugin: Add options to the `getmanifest` call * pytest: Add a test for the cln-plugin logging integration * cln-plugin: Implement logging facade adapter for cln plugins * cln-plugin: Get started with the plugin interface * ci: introduce in ci the compilation testing on different os * dualopend: restore memleak calls. * dualopend: fix memleak reports. * lightningd: simplify memleak code. * docker: Clean up the dockerfile to use poetry * py: Update mrkd * pyln: Switch to binary psycopg2 * pyln: MIgrate the BOLT packages to poetry / PEP 517 * gci: Only test pushes if it's `master` * gci: Fix up the MacOS test with poetry * python: Exclude newly detected f-string flake * misc: Make the spell checker just a tiny bit less trigger happy * gci: Switch to python 3.7 * gci: Switch docs and CI builder to use `poetry` * gci: Do not generate JSON reports for test runs * pyln: Use poetry and PEP 517 locks for the root project * pyln: Migrate pyln-testing to PEP 517 (poetry) * pyln: Migrate pyln-client to PEP 517 (poetry) * pyln: Migrate pyln-bolt7 to PEP517 (poetry) * pyln: Fix the pyln-proto version and migrate to PEP 517 (poetry) * gossipd: allow faster updating, 2 per day, before ratelimiting. * gossipd: force a fresh node_announcement every 24 hours. * gossipd: always re-sent our own node_announcement on startup. * gossipd: make sure we set the `urgent` bit if we move our own node_announcement. * connectd: also fail without a scary backtrace when listen fails. * connectd: make sure we announce websocket addr which succeeded. * connectd: use listen_fd array directly, rather than returning binding arr. * connectd: hoist find_local_address so we can give more graceful Tor erros. * connectd: add is_websocket and wireaddr to struct listen_fd. * connectd: be more graceful when we an address is in use. * balance_snapshot: don't count unconfirmed utxos * coin_mvt: only chain moves have a blockheight * json: reverse parse a coin_mvt tag back into an enum * listpeers: include the `pushed_msat` amount * coin_mvt: report mutual close outputs also * tx_parts: as a convenience, copy over the value as the 'satoshi' * coin_mvt: mark coins destined for wallet w/ originating acct * coin_mvt: report the number of outputs on a channel close tx * coin_mvt: mark every event that already has a destination * coin_mvt: rm unncessary if statement * db: collapse the db_queries files into the generated ones * db: mark that column is unused/ignored * database: pull out database code into a new module * db: decouple `fatal` reliance, have as impl defined function * lightningd/Make: cleanup lightningd+wallet headers * plugins/bcli: neaten to fix spurious leak report. * bitcoin: Comment typo fix * test_close: sync l1 to the blockchain, so that the events are there * rust: Use $CARGO_OPTS when building the example plugin * cln-grpc: Add `listpeers` RPC method * cln-grpc: Allow fields to be mapped to None to ignore them * cln-grpc: Add conversion from pb Amount to json Amount * cln-grpc: Add conversion from pb Amount to json Amount * msggen: Support renaming methods in GRPC * msggen: Support enums in requests too * msggen: Handle some more types in request conversions * cln-grpc: Add glue to get all pieces to work together * cln-grpc: Generate server dispatcher * cln-grpc: Add generation of request conversion * cln-grpc: Add result conversion generator to `msggen` * cln-grpc: Generate grpc bindings from proto * cln-grpc: Add generation of grpc protobuf file from schema * rust: Actually check if we have cargo * testing: fix flake in test_ping_timeout * pyln-client: Plugin, improve background comment * pyln-testing: improve description of wait_for_logs * pyln-testing: in LightningNode.openchannel, make wait_for_announce more reliable * pyln-testing: in utils, update method fundbalancedchannel, now it also works with dualfund * plugin-funder: fix typos in option lease-fee-base-msat and funder-fuzz-percent * doc: add details about: log-level, experimental options and `features` field returned by listnodes, * lightningd: fix parsing of option dev-force-features * JSON-RPC: getinfo, add field `our_features` * testing: cleanup in test_plugin_shutdown * devtools: in decodemsg, add newline and fix decoding a node_announcement with tlvs * configure: restore -Og default on non-developer builds. * memleak: fix handling of excluded pointers. * memleak: fix handling of notleak pointers. * memleak: restore functionality accidentally removed. * plugins/funder: actually free the pending_opens. * libplugin: make memleak see current requests. * common/param: don't keep around the temporary array off cmd. * common/param: fix leak with opt_param_def. * onchaind: fix minor leaks. * plugins/pay: make memleak happy. * libplugin-pay: fix spurious leak reports. * common/bolt11: make decoded routes hang off routes arr, not b11. * plugins/keysend: prevent leak detecting getting upset. * hsmd: remove unnecessary memleak_remove_region. * lightningd: don't leak address for printing. * gossipd: move deferred_update list ptr to head to ease memleak checks. * plugins/pay: fix unnecessary code. * libbacktrace: update to latest. * pytest: Remove 3 stress-tests * hsmd: don't leak message buffers * gossipd: don't send updates in error messages for unannounced channels. * channeld: Comment update for channeld * pytest: remove test_htlc_rexmit_while_closing as too flaky. * Clean up outdated comments about gossipd * bolt11: mark when expiry is decoded * bolt11: mark when we decode min_final_cltv_expiry * json_add_invoice: fix crash if missing invstring * Add LIGHTNINGD_NETWORK env variable to Dockerfile for ARM * doc: reintroduce the fmt command to fmt the schema * misc: Add build targets for the tarball and debian packages * chore: use EXPERIMENTAL for BOLT1 remote_addr #917 * pytest: check for remote_addr * peer_exchange: set, read and log remote_addr * peer_wire: add remote_addr to init_tlv * wiregen: adds note about what <count> can be * bitcoind: importmulti fails (bitcoin master), use importdescriptors * pytest: fix test_statictor_onions * pytest: YA bitcoind API break. * bitcoin/tx: remove unused SEGREGATED_WITNESS_FLAG * cln-rpc: Add type for AmountOrAll and AmountOrAny * gci: Limit the RUST=1 config to test rust-related functionality * gci: Add rust configuration to Github actions * pytest: Test the rust bindings from cln-rpc * cln-rpc: Scaffolding for the cln-rpc crate * rust: Add rust detection to configure and a target to add binaries * json-rpc: Add request stubs for a couple of calls * msggen: Generate the cln-rpc Rust structs * msggen: Parse JSON-RPC schemas and build the in-memory model * pyln: Delete psql DBs after testing * Remove useless \ * Remove some formatting inconsistencies * doc/schemas: disallow unknown fields in listchannels. * Add short_channel_id to listchannels.schema.json * misc: clarifications from cdecker review. * pytest: note unfixable test_htlc_rexmit_while_closing test. * pytest: make test_mpp_adaptive more reliable * dual_funding: don't steal inflight field in update_channel_from_inflight. * lightningd: clean up subds before freeing HTLCs. * gossipd: fix longstanding logic error in gossip_generation. * pytest: protect against bad gossip messages from mining confirms too fast. * pytest: make test_gossip_no_empty_announcements more robust. * pytest: remove flake in test_upgrade_statickey_onchaind * gossip_store: handle compacted gossip_store correctly. * connectd: don't received useless peer fd if we're told to send final msg. * connectd: remove second gossip fd. * gossipd: don't hand out fds. * subdaemons: remove gossipd fd from per-peer daemons. * channeld: simply exit if hsmd vanishes. * connectd: handle custom messages. * common: make sure we hand through peer for io logging. * connectd: do io logging properly for msgs we make. * connectd: handle onion messages. * connectd: handle pings and pongs. * pytest: ignore pings when doing query_gossip. * gossipd: send all gossip msgs directly to connectd, not peer. * connectd: divert gossip messages directly to gossipd. * gossipd: add routines to send gossip messages to and from connectd. * connectd: temporarily have two fds to gossipd. * dualopend: tell lightningd about new channel as soon as it's locked in. * channeld: pause before sending initial channel_update. * channeld: send channel updates and announcements via lightningd. * channeld: keep local copy of latest channel_update for errors. * lightningd: use our cached channel_update for errors instead of asking gossipd. * lightningd: tell gossipd when we use the channel_update. * gossipd: feed lightningd the channel_updates as soon as we make them. * gossipd: infrastructure to tell lightningd about local channel updates. * lightningd: start gossipd after channels are loaded. * fix UnicodeDecodeError in doc/schemas/listfunds.schema.json * fix UnicodeDecodeError in doc/lightning-listfunds.7.md * db: enable SQLite extended result codes * lightningd/Make: fixup reference to wallet headers * doc: turns out these fields are optional * coin_mvt: tiny, dont import lightningd header * coin_moves: an htlc_fulfill claimed by them is a credit to "external" * listchannels: add funding_outnum to listchannels * wallet: was erroring out, saving to null field * coin_moves: remove unused method * pytest: update elements for new, more accurate feerate calc. * elements: fix gross weight differential. * closingd, lightningd: use bitcoin_tx_2of2_input_witness_weight * elements: unify overhead calculation. * bitcoin: implement bitcoin_tx_2of2_input_witness_weight * pytest: test that our closingd tx weight estimate is correct. * bitcoin: fix tx weight calculation when there are no witnesses, but will be. * coin moves: notify when we make deposits to external accounts * coin moves: external deposits should be *deposits* * doc nit: wrap rest of tags in backticks * coin_moves: add an 'originating_account' field * make: add ctags target * pytest: fix flake in test_upgrade_statickey_fail * gossipd: use tal_dup_talarr helper. * gossipd: fix unknown channel_update recovery logic. * subd: fix waitpid properly. * lightningd: fix backwards test in sigchld. * CI: actually check db statements. * common: remove stderr debug in is_valid_witnessprog. * common/socket_close: remove now only connectd talks to peer. * pytest: clean up test_channel_state_changed_unilateral * subds: remove "ignore error" from old LND nodes. * pytest: fix race when we mine blocks after pay(). * pytest: disable test_closing_different_fees for elements temporarily. * connectd: don't ignore requests to connect if we're shutting down. * connectd: make sure we io_log msgs doing to gossipd. * pytest: disable automatic reconnection. * pytest: disable tests/test_closing.py::test_onchain_all_dust * peer subds: ignore failed writes. * pytest: fix flake in test_reconnect_sender_add1 * patch dual-open-control-double-notify-fix.patch * connectd: flush queues before hanging up. * connectd: also do the shutdown()-close for final_msg sends. * connectd: don't just close to peer, but use shutdown(). * ccan: update to get io_sock_shutdown * msg_queue: don't allow magic MSG_PASS_FD message for peers. * connectd: implement @ correctly. * connectd: drop support (unused) for @ during handshake. * various: minor cleanups from Christian's review. * connectd: get addresses from lightningd, not gossipd. * connectd: put more stuff into struct gossip_state. * connectd: remove per_peer_state in favor of keeping gossip_fd directly. * patch lightningd-peer-fds.patch * subdaemons: don't stream gossip_store at all. * connectd: serve gossip_store file for the peer. * common: add routine for absolute timeouts (vs. relative). * connectd: do nagle by packet type. * connectd: do dev_disconnect logic. * per_peer_state: remove struct crypto_state * peer_io: replace crypto_sync in daemons, use normal wire messages. * connectd: do decryption for peers. * lightningd: make sure gossipd knows before we update blockheight. * gossipwith: create our own internal sync_crypto functions. * connectd: maintain connection with peer, shuffle data. * pytest: make test_channel_state_changed_unilateral more robust. * connectd: keep timeout timer around so we can disable it. * update mocks * cli: change the redefine logic to read from the stdin * fix dockerfile * Update lightning.py * LightningRpc should accept pathlib Path * options: only allow one DNS announcement * pytest: further deflake test_funding_push. * common/coin_mvt: make it clear we're using Lightning HRP not BIP 173. * common/coin_mvt: clean up API a little. * global: use tal_dup_or_null(). * ccan: update to get new helpers in ccan/tal * common/socket_close.c: remove obsolete comment. * connectd: rename struct peer in peer_exchange_initmsg to early_peer. * gossipd: make request handlers return void. * gossipd: rewrite update generation, remove local_chan. * gossipd: local_direction helper to generalize is_local_channel. * channeld: add dev-disable-commit-after instead of dev-disconnect -nocommit * dev_disconnect: remove @ marker. * pytest: disable dev-disconnect blackhole tests. * common/socket_close: don't wait forever. * common: generalize extract_channel_id(). * connectd: clean up lightningd connection handling. * common: is_msg_gossip_broadcast() for putting things in filter. * lightningd: handle channel cleanups more explicitly. * lightningd: close one more fd for subdaemons. * external: update lnprototest dependeces * balance-snaps: add a `balance_snapshot` event; fires after first catchup * test plugins/coin_moves: clean up/tidy coin moves plugin logic * balance snap: first pass * coin_moves: cleanup tags list, update docs, remove unused method * coin_mvt: log events for pushes/lease_fees for leased channels * coin_mvt: record fees for an outbound htlc * coin_mvt: record new 'fees' field on htlc channel moves * coin moves: turn 'tag' into 'tags' array, add OPENER tag * coin_moves: update some info about 'blockheight' * coin_moves: remove 'index' for moves; bump version * onchaind: remove 'is_replay' logic * coin_moves: de-dupe moves as they come in * coin_mvts: rewrite how onchain events are recorded, update tests * onchaind:move some stuff around so we can call it * coin_mvts: don't overwrite account names we add in onchaind * coin_mvt: add "output value" to coin movement * coin_moves: fix account name serialization * amount: helper for msat == 0 * coin-move: the txid is now optional but outpoints are not * coin-mvts: on close, record an 'end' of our channel balance * coin-mvt: remove all the chain fees tracking * coin-mvt: remove all the spend-tracking * coin-moves: remove penalty + chain fee types * Add pip requirements install to Mac instructions * Fix comment to reflect the code * doc: document the errors code * lightningd: support hsm error code * wire: add missing spec patch. * ccan: update, add graphql module. * pay: Disable channels that are not in normal state * pytest: restore gossip advertizement tests for default port. * doc: force mistune==0.8.4 during pip package resolution. * Removed FIXME to match bolts PR 918 * hsmd: Add hsmd_ready_channel * hsmd: Add hsmd_new_channel * wallet: make wallet_can_spend non-static because needed * bitcoin/chainparams.h: Split BIP173 name into onchain and Lightning HRPs. * Revert "bitcoin/chainparams.c: Change `signet` BIP173 name to `tbs`." * fix 'listincoming' field incoming_capacity_msat. * hsmd: Add hsmd_validate_revocation. * doc/Makefile: speed up schema check, only do on check-doc. * lightningd: cleanup obsolete plugins->shutdown flag * testing: remove test_htlc_accepted_hook_shutdown, move relevant parts into test_plugin_shutdown * testing: fix flake in test_fetchinvoice_3hop * common: cleanup unsused parameter in timer_expired() * plugins: simplify shutdown loop, simply close the db * Update .gitignore for tmp files * connectd: fix empty error message * Fix dockerfile for arm32/64 * pytest: fix test_pluginconnected_hook_chaining * Upgrade bitcoin-cli to 22.0 * doc: lightningd-config.5.md - default ports * Set default port according to network * make: remove generated files when running clean * gossipd: remove gossip_msg.[ch] * daemons: remove unused functions or make static. * common: remove unused functions or make static. * bitcoin: remove unused functions, or make static. * devtools: simple script to look for functions which don't seem to be used. * tools/check-includes.sh: make shellcheck happy. * common: remove unused io_lock.c * check: make sure all files outside contrib/ include "config.h" first. * pytest: fix occasional flake in test_announce_address * common/utils: introduce tmpdir_mkstemp; use it * connectd: don't advertize websocket support if we have no other (non-Tor) addrs. * options: fix handling of wildcard (allproto) address. * connectd: fix websocket binding when we're doing both IPv4 and IPv6 on same port. * pytest: test to show conflict between websocket and wildcard addresses. * status: do not log ping and pong packets * .github: Fix FreeBSD workflow * doc: add exclude arg doc for pay rpc * tests: add pay test for exclude arg * Add route exclusion payment modifier and pay rpc exclude arg * refactor: move exclude parsing to json_tok * doc/lightning-datastore.7.md: Fix typos. * wallet/db.c, wallet/wallet.c: Add a partial index to speed up startup. * pytest: fix flake in testing. * lightningd: fix leak report for self_id * blinding: check against two of the test vectors in the PR. * bolt12: remove deprecated `chains` fields * doc: clean up offers bolt quotes * features: don't announce onion_messages in bolt11 invoices. * test: make an onionmessage test vector. * common/test/run-blindedpath_enctlv: use modern functions to create test vector. * test: hack in test for (now-obsolete!) test vector Thomas emailed me. * pytest: re-enable modern/obsolete fetchonion tests. * gossipd: handle receipt of modern onion message. * lightningd: Send updated onion spec messages. * lightningd: return both obsolete and modern blindedpaths from "blindedpath" RPC. * spec: import latest onionmessage spec, based on routeblinding. * common: rename current onion message structures to obs2_. * common: remove support for pre v0.10.2 onionmessages. * plugins/Makefile: improve header dependencies. * doc: update shutdown notification, changelog * hsmtool: use flag TCSANOW when disabling tty ECHO, fixes rare hang in test_hsm* * lightingd: removal of sigchild_handler in shutdown now also closes its pipe * libplugin-pay: fix valgrind error * testing: fix test_upgrade_statickey_onchaind * testing: fix test_closing_higherfee * testing: test that RPC calls made in shutdown fail and receive error code -5 * JSON RPC: Calls made in shutdown loop receive error code -5: LIGHTNINGD_SHUTDOWN * JSON RPC: In the shutdown loop, ignore plugin responses to JSON RPC requests * testing: remove test_stop_pending_fundchannel * lightningd: cleanup, freeing jsonrpc in shutdown cannot trigger db write's anymore * lightningd: shutdown plugins after subdaemons and assert no write access to db * testing: test hook semantics is preserved in shutdown * chore: use EXPERIMENTAL for BOLT7 DNS #911 * pytest: test connecting to a DNS only announced node * connectd: resolve ADDR_TYPE_DNS * bolt7: allow announcement of ADDR_TYPE_DNS * wireaddr: adds helper is_ipaddr, is_toraddr and is_dnsaddr * options: fix respect --disable-dns when parsing wireaddress * chore: gitignore gdb_history * options: make --disable-dns an early arg * options: fix ld proposed_wireaddr proposed_listen_announce get out of sync * doc: Fixed grammars and add jq as dev dependences. * doc: Fixed doc check with the last options * doc: Document max-dust-htlc-exposure-msat the command line option * jsonschema: Formatting json schema and regenerate docs * tools: add a tool to formatting schema, and fixed (hack) the Makefile code checking. * ci: Introduce doc sanity check in Github action. * bitcoin/chainparams.c: Change `signet` BIP173 name to `tbs`. * pytest: don't valgrind plugins under CI if many nodes. * pytest: Reduce memory consumption by test_plugin_disable * backtrace: avoid duplicate backtrace objects. * memleak: make notleak use the tal name instead of a boutique struct. * log: don't leak log prefixes. * lightningd: remove some unnneded notleak(). * devtools/taldump-analyze.py: script to analyze tal_dump() output. * ccan: import udpated version (with better tal_dump() format). * doc/PLUGINS.md: Clarify that `db_write` hook only works on SQLITE3. * doc/BACKUP.md: Discourage litestream use. * wallet: fix sqlite3 column renaming since db backup merge. * lightningd: fix compilation error on OpenBSD * wallet/db_sqlite3.c: Support direct replication of SQLITE3 backends. * wallet: db column manipulation helpers. * db: turn generated queries array into a simple hash table. * wallet: remove db_column_ functions. * wallet: remove redundant field counters, ignored fields. * wallet: have db track what columns are accessed in DEVELOPER mode. * CCAN: import strset. * wallet: convert db internal routines to db_col_ * wallet: convert wallet.c to db_col_ APIs * wallet: use db_col_ accessors in wallet/invoices.c * db: improve db_col_* APIs. * db: db_col_ variants for accessing SELECT statements by name. * db: create simple hashtable of fields in SELECT. * tests: use common_setup/common_shutdown to avoid leaks. * common: clean up autodata in common_shutdown(). * pytest: allow bad gossip in pruning test. * contrib: fix other pyln-spec package requirements too. * pyln-bolt7: fix requirements.txt * paystatus: remove doubled amount_msat. * doc: update HACKING.md build dep instructions * doc: add missing dev deps, upgrade and don't use sudo for pip * gitignore: ignore eggs and pyln versions * pytest: don't checksum plugins on startup in VALGRIND developer mode. * pytest: don't create 5 nodes in test_fetchinvoice. * common: use ccan/base64 instead of libsodium. * common: test that our sodium-based base64 encoding is identitcal to ccan's. * wireaddr: clean up tor parsing. * lightningd: really do allow two Torv3 addresses. * doc/TOR.md: simplify, and don't cover autotor. * doc/TOR.md: Make it clear that Tor == Torv3. * torv2: remove support for advertizing and connecting. * custommsg hook: remove message field. * fundchannel_complete: remove deprecated txid/txout params. * pytest: remove test_shutdown_alternate_txid. * Makefile: Avoid overriding `PYTHONPATH`. * subd: clean up our fd shuffling logic. ------------------------------------------------------------------- Wed Nov 03 18:28:22 UTC 2021 - marec@detebe.org - Update to version 0.10.2: * doc: Update CHANGELOG.md for v0.10.2 release * doc: Update CHANGELOG.md for v0.10.2rc2 * gci: Add `-f` flag to git fetch * doc: Remove non-ASCII character from docs * peer: Wrap the HTLC out destructor in a DB transaction * wallet: Re-sync the payments.id sequence with postgresql * db: Allow DB-specific queries in the migrations * Fix typo in the error message. * Ignore all the deamon inside the git history * wallet/db_postgres.c: Fix obvious typo in error message. * websocket: switch to address type 6, since DNS will take 5. * meta: Add version 0.10.2rc1 changelog * pytest: Create less dust in `test_delpay_payment_split` * tests: raise dust limit on mpp test * dusty htlcs: don't fail the channel, make it error a whole bunch * channeld: add in RFC notes for max_htlc_dust_exposure_msat * tests: check for incoming + outgoing dust limits * dusty-htlcs: enforce limit on dusty htlcs * htlcs: add flag to 'fail immediately' * tests: bump the ceiling for dust, fixes failures for liquid * lightningd: new option for htlc dust limit * doc: Add M1 mac + homebrew fix * pay: Use log probability based bias in channel selection * libplugin-pay: bias towards larger channels. * common/dijkstra: hand channel direction to path_score callback. * lightningd/lightningd.c: Only impose fd limit if absolutely needed. * *: Use new closefrom module from ccan. * ccan: update to include closefrom * experimental-websocket-port: option to create a WebSocket port. * lightning_websocketd: simple proxy for websockets. * connectd: listen on ports for which we should spawn a proxy. * patch websocket-address-support.patch * CCAN: add base64 module. * gossipd: when we refresh channel, don't override pending updates. * common: reltimer_arg to access a timer argument. * Move log level from warning to info when creating a new ln directory * doc/BACKUP.md: Warn about version if using Litestream, `sqlite3` `.dump` is safe(r) now. * wallet/db_sqlite3.c: Increase busy timeout to 60 seconds. * CI: suppress postgres vaccuuming. * wallet: only hand onchaind the HTLCs it needs to know. * onchaind: queue any unexpected messages while waiting for htlcs. * onchaind: tell lightningd our commitment number, then get htlcs. * common: use bitcoin_outpoint. * common/type_to_string: formatting for bitcoin_outpoint. * lightningd: use bitcoin_outpoint in watch. * db: vacuum after a db upgrade. * db: clear our unneeded htlc fields in old terminated HTLCs. * wallet: remove unnecessary data from channel_htlcs when htlc is dead. * lightningd: populate min/max commit fields in db. * db: add min/max commitnum fields to channel_htlcs. * wallet: wrap htlc_state enum in db function. * utils: add max_unsigned/min_unsigned helpers. * fuzz: make it build again. * tests: Change currency from `msat` to `sat` in the regex string * Fixes msat and sat convention in the error message * offers: fix overzealous BROKEN log. * pay: listpays groups by payment_hash and groupid * pay: Stash and forward results for duplicate `pay` calls * pay: Mark completed payments as such by nullifying `cmd` * pytest: Adjust `test_sendpay` to the new semantics * libplugin: Add callbacks for successful and failed payments * pytest: Fix `test_onchain_timeout` to use `groupid` * pytest: Add `groupid` to `test_partial_payment_{timeout,restart}` * pay: Fail a `sendpay` or `sendonion` that'd produce a DB collision * pytest: Fix up `test_partial_payment` to use a single `groupid` * paycore: Default `groupid` to increment from last one * paycore: Prevent multiple concurrent payment groups * pay: Do not delete old sendpay attempts if we retry * pay: Make `pay` idempotent * jsonrpc: Add `groupid` to `waitsendpay` * doc: Add missing `amount_sent_msat` to `listpays` * pay: Call `listsendpays` to find prior attempts and abort if needed * jsonrpc: Add missing `partid` in `listsendpays` schema * db: Add `groupid` to HTLCs * wallet: Add function to retrieve the latest groupid for a payment * pay: Add `groupid` to the payment struct * pyln: Add groupid to `sendpay` and `sendonion` * jsonrpc: Add groupid to `sendpay` and `sendonion` * db: Add groupid to the payments table * pytest: Reproduce #4482 * closingd: ignore spurious pong messages. * pyln: Add missing requirement from pyln-client to pyln-proto * pytest: Use the same `TIMEOUT` in `wait_for_route` * db: Ensure sqlite3 library minimum version and major version match * doc: update ungenerated file. * pytest: fix flake in test_upgrade_statickey_onchaind * pytest: fix flake when test reconnects by itself. * channeld: restore ping command, but only for channeld. * channeld: perform regular keepalive pings. * channeld: fix halting gossip when a timer is active. * channeld: remove liveness logic pre-commitment. * gossipd: remove ping/pong handling * common/read_peer_msg: handle pings in handle_peer_gossip_or_error(). * Fix memory leak abort in debug mode * On Mac OS X a different error is raised on failed connection * db: Don't switch between text and blob fields * db: Fix a syntax error with the optional parameters * ci: Add missing envvar forward for the DB configuration * channeld: handle reestablish from previous release with EXPERIMENTAL_FEATURES. * channeld: import updated channel_upgrade spec. * features: EXPERIMENTAL_FEATURES: advertize option_quiesce * lnprototest: update to latest master. * BOLT 12: switch invoice_request/invoice to singular `chain` field. * bolt12: update to latest spec. * BOLT12: use point32 instead of pubkey32. * bolt12: upgrade to handle latest test vectors. * Makefile: be sure to build generated sources before tests. * plugins/libplugin.c: Allow freeing notification `struct command *`. * Length prefix bitcoin tx, to fix layer violation * ci: Add BSD configuration * docs: Update README to link to pyln-client * gha: Fix the trigger in the PyPI publication action * correcting: 'needs' instead of 'need' * common/json.c: Have `json_add_literal` *use* `len` argument. * pylightning: Make it clear pyln-client should be used instead * onion_message: dev options to ignore obsolete/modern onions. * offers: send a modern onion reply in response to a modern request. * offers: save replytok properly. * fetchinvoice: handle modern onion_message reply. * fetchinvoice: send modern as well as obsolete messages. * gossipd: handle modern onion messages. * lightningd: handle modern onion termination. * lightningd: fix use-after-free during shutdown. * Correcting typo * gci: Checkout tags in CI to get auto-versioning working * gci: Remove push trigger * gci: Use stripped down bitcoind for Mac OS * gci: Pin down a couple more dependencies * gci: Limit PyPI publication to pull request merged and version tags * pyln: Derive version from git for pyln-proto * pyln: Derive version from git for pyln-testing * gci: Checkout entire history in PyPI build to ensure tags are there * pyln: Add production publication on tag push * doc: Add the in-tree-build option to install docs * gci: Add missing tooling * pyln: Derive version from git for pyln-client * gci: Give PyPI packages a reasonable name * pyln: Materialize setup.py and requirements.txt in pyln-spec * gci: Add workflow to deploy PyPI artifacts * pytest: Stabilize `test_addgossip` * misc: ignored last autogenerated file from the github tree. * misc: Add generated files to .gitignore * gci: Add macOS configuration * Add missing headers for compiling on FreeBSD 13 * lightningd: provide 10 minutes for channel fee increases to propagate. * lightningd: refactor forward feecheck. * common/features: try to add all proposed feature names. * devtools/features: tool to convert feature bitmap to names. * pytest: fix flake in test_gossip.py::test_addgossip * Makefile: remove generated files. * Makefile: remove NO_PYTHON flag in anticipation of making it compulsory. * review 2/2: Use generic query to make the code more readable. * review 1/2: Fixed the doc missing details about formatting. * plugin: Adding status to the pay plugin * doc: Update doc with the new parameter supported * pyln-client: Adding parameter to the listpays rpc method. * wallet db: Support the query on database with status and payment hash. * rpc: Integrate the status flow in the listsendpays command * common: json_to_reply_path to helper to handle blindedpath/onion_message hook * lightningd: `blindedpath` helper to create a blinded path to ourselves. * lightningd: new sendonionmessage routine. * hsmd: derive an onion_reply secret. * common/blindedpath: enctlv unwrapping primitives. * common/blindedpath: enctlv creation primitives. * wire: import new onion message spec. * invoice: Fix mismatch between ">=" and "greated than" in error msg * bcli: Accept "already in chain" errors as success * doc: Add links to mandelbit's recovery walkthrough * jsonrpc: Simplify the state decision of `listpays` * pay: Fix wrong result being returned by pay * hsmd: Remove duplicate status message for incoming messages * fetchinvoice: include send in path, use pubkeys. * lightningd: rename sendonionmessage to sendobsonionmessage. * gossipd: rename onionmsg messages to lightningd to obs_onionmsg. * channeld: rename onion_message to obs_onion_message. * common/sphinx: add helper to prepend length to payload. * lightningd: remove sendonionmesage parity hack. * gossipd: fix blinded onion forwarding. * lightningd: fix memleak false positive. * type_to_string: detect type failure in DEVELOPER mode. * ccan: remove autodata. * common/autodata: use instead of ccan/autodata * common/autodata: autodata replacement which uses __attribute__((constructor)) * pytest: ensure hsmtool tests do complete writes. * wire: Fix generated files * Fix for Alpine Linux ond OpenBSD * cleanup: remove unneeded includes in C files. * cleanup: remove unneeded includes in header files. * devtools/reduce-includes.sh: brute-force script to remove includes. * tools/check-includes.sh: test that c files include their .h files. * closingd: add notifications for feerate ranges. * lightningd/closing_control.c: move json_close here from peer_control.c * tests: test that successive closes do update parameters (particularly feerate!). * lightningd: fix leak report when close called multiple times. * Remove lnprototest * openingd: clean up state properly when a hook says to reject opening. * common/channel_type: fix up bolt quotes now channel-types is merged. * closingd: fix BOLT quotes to match final spec version. * openingd: tell them channel_type if signature is bad. * lightningd: print out support for channel_types in --list-features. * openingd: allow channel_type to *downgrade* from default. * openingd: channel_type TLV no longer experimental. * openingd: EXPERIMENTAL_FEATURES provide and use channel_types. * openingd: tell lightningd what channel type we negotiated. * lightningd: use channel_type, pass to-and-from channeld. * common/initial_channel: use channel_type instead of individual option bools. * common/channel_type: wrapper for generated 'struct channel_type'. * tools: handle generating print templates for CSV without tlvs, messages * openingd: don't hand redundant feature flags. * resolve() appears broken on Python 3.7+ * plugin: register `shutdown` instead of `plugin_shutdown` in notifications list * close: spec is final, it's not experimental. * closing: add option to set closing range. * pytest: add test for closing feerate greater than final commit tx. * closingd: allow higher closing fee if anchor_outputs. * --experimental-quick-close to enable quick-close negotiation * Makefile: import latest spec, which includes quickclose. * Makefile: update bolt version to include option_anchors_zero_fee_htlc_tx. * Makefile: update spec version * channeld: allow large HTLCs if peer offers option_support_large_channel * Makefile: Update BOLT version to cover bolt11 test vectors with payment secret. * lightningd/chaintopology: stop callbacks rearming once topology stopped. * libplugin: make leaks log at LOG_BROKEN so they break CI. * plugins/libplugin: mark timers as not-a-leak. * libplugin: fix leak of struct command when we don't read it all at once. * plugins/topology: use memleak annotation instead of global_gossmap hammer. * pytest: annotate name_option against leak detection in test_libplugin * wire/tlvstream: don't leak in tlvstream_set_tu64/tu32/short_channel_id. * plugins/txprepare: annotate unreleased_txs list against false memleaks. * plugins/bcli: keep a list of current bitcoind requests. * plugins/spender: don't use global tal context, use take() instead. * spender/openchannel.c: don't leave psbt parts owned by NULL. * plugins/funder: don't use global owner parent; mark policy for memleak. * plugins/bcli: handle memleak. * libplugin: let plugins add annotations to memleak. * libplugin: if DEVELOPER set, do memleak detection on shutdown. * libplugin: allow take() for commands, notif_subs, hook_subs, notif_topics. * libplugin: allow stealing of feature sets, make keysend do that. * common/memleak: hoist strmap helper from out of lightningd/memdump.c * common/memleak: take over dump_memleak(), allow print pointer. * ccan: update to get version where tal_dump goes to stderr. * Critical funder flaw * pyln-client/gossmap: adds testcase for half channels * pyln-client: fix mypy warnings, fix and test deletion of a channel. * pyln-client/gossmap: init GossmapNode and Id also with hexstring * pyln-client/gossmap: make node and node_id comparable with __lt__ and __eq__ * pyln-client/gossmap: let half channel have source and destination * pyln-client/gossmap: start internal function names with underscore _ * pyln-client/gossmap: have channels link to their nodes instead of id * pyln-client/gossmap: make test gossip_store include channel_updates, deletions. * pyln-client/gossmap: more fixes, make mypy happier. * pyln-client/gossmap: Don't mix bytes and GossmapNodeId * pyln-client/gossmap: adds __repr__ functions * pyln-client/gossmap: extend testcase * pyln-client/gossmap: adds channel satoshi capacity * pyln-client/gossmap: adds helpers to get channels and nodes * pyln-client/gossmap: adds GossmapHalfchannel objects * pyln-client/gossmap: add NodeId class. * pyln-client/gossmap: use ShortChannelId class from pyln.proto, if available. * pyln-proto: expose ShortChannelId and PublicKey. * pyln-client/gossmap: add a little documentation. * pyln-client/gossmap: save deconstructed fields instead of raw msg bytes. * pyln.client: new functionality to access Gossmap. * pyln.client: don't try to use module inside setup.py. * pyln-spec: update to latest version of the spec. * Mac OS X generates these files automatically. * Removed submodule external/lnprototest * Shellcheck cleanup * doc: document shutdown plugin. * lightningd: call shutdown plugin when we dynamic shutdown a single one. * libplugin: plugin_exit helper which flushes stdout. * lightningd: call "shutdown" notification on plugins at shutdown. * lightningd: disable topology timers on shutdown. * lightningd: don't keep timer ptr inside struct chain_topology. * doc: fix GH/readthedocs rendering of manual pages. * psbt: set the witness script to null, also * dual-fund psbt: hoist up single use method so we can test it * db: simple index addition to speed listpeers. * docs/fetchinvoice: document payer_note * pytest: test new commitment_revocation hook values * doc: new values for commitment_revocation hook * hook: adds channel_id and commitnum to commitment_revocation * doc: Regenerated man-pages to fix hash mismatch. * doc: fix listnodes schema, doc. * Anchor outputs are enabled by dual-funding, so this was reported by users there (and reliably detected now). * configure: update from -fstack-protector to -fstack-protector-strong. * adding check for the deprecated api. * Fixed order parameter in the listforwards command * wire: fix patch file to match actual peer_wire.csv. * docs: Remove explicit markdown configuration for rtd * lightningd: don't reject attempts to forward 2^32 msat out of hand. * Don't limit channel_updates to 2^32 msat. * Skip following import and skip flask type to mypy code analysis. * Move to minimum required mypy dependeces * Added deprecated phase to rpc framework rules changes. * Required jsonrpc inside the method request. * pytest: don't copy db to query it. * wallet: set a timeout if the db is busy. * added Litestream Replication * pylightning: fix requirements to work with pyln-client * datastore: turn keys into arrays * datastore: add generation, simple atomicity. * datastore: allow replace/append. * datastore: allow strings. * datastore: docs, schemas, and a test. * JSON-RPC: add interfaces to access datastore. * wallet: add datastore access routines. * wallet: add datastore for plugins to use. * exact bk file has to be in path * missing --lightning-dir * gossmap: don't spew to stderr, include counter for callers. * pytest: remove only_one() duplicate * wallet-test: fix segfault due to uninitialized block * cleanup: rename use_proxy_always to always_use_proxy to match cfg * cleanup: nits * connectd: prefer non-TOR connections and filter wireaddr hint duplicate * fetchinvoice: add developer option to send raw invoice_request. * pytest: fix flake in test_channel_lease_unilat_closes * wallet-test: print better error message on failure * utxos: more comprehensive fix for csv check * lnprototest: update version to fix flake issues. * pytest: try to prevent spurious timeouts in test_channel_lease_unilat_closes * pytest: fix test_channel_lease_unilat_closes flake * pytest: fix bad gossip flake. * pyln: Fix up the mypy annotation for `init` self-disabling * pyln-client: document and test that init can self-disable. * common/setup: check libwally initialization * bitcoin/tx: check PSBT sanity after parsing * Build race fixes: create ccan/config.h atomically, make tools/test/test_gen.c depend on it * common/test/run-bolt12_period.c: fix warnings at -O3. * common/tests: these test are no longer experimental-only. * chaintopology: tell gossipd that channels no longer exist on reorg. * changelog: fix typo in release notes * new line for default value, makes it more visable ------------------------------------------------------------------- Tue Aug 10 09:09:55 UTC 2021 - marec@detebe.org - Update to version 0.10.1: * changelog: v0.10.1 release notes * wallet: correct CSV check in `deep_enough` * tests: check that we don't re-up the leased funds when we RBF * tests: promote method to utility file * funder: default to only funding leases * funder-rbf: correctly iniitalize lease params * df-tests: wait for disconnect before asserting unknown channel * df-tests: bump feerates to get over the 'insufficient fee' limit * pytest: update test_queryrates for API change. * dualopend: remove runtime memleak in favor of at-termination. * GitHub CI: Restore at least one DF CI flow. * offers: make `decode` command available even without experimental-offers. * CHANGELOG.md: note the broadening in fee_update accept behavior. * changelog: Updated the changelog for v0.10.1rc2 * pyln: Add E741 as excluded flake warnings * doc/MAKING-RELEASES.md: advice on updating python package versions. * pyln.{proto,client,testing}: bump version numbers to match release. * pyln.proto.message: add new BOLT12 fundamental types. * pyln.proto.message: export two more symbols, remove unused __version__ from message.py * common/bolt12_merkle.c: fix comments. * common/test: test the test vector JSON from the spec (if present). * common/test: Fix test vector generation. * pyln: Fix backward compat issue in `LightningRpc.close` * connectd: allow out-of-bounds fees unless they're actually getting *worse*. * channel lease: only send update_blockheight iff has channel lease * Moving dependencies to the minimum required * changelog: add two general notes and fix the typo * changelog: correct link for v0.10.0 release on github * changelog for v0.10.1 rc1 * diff -R to diff -r * common: fix up missing file which can cause parallel build issues. * lightningd: don't turn zero-length tlv fields into NULL. * lightningd: don't assume zero-length tlv fields will be NULL. * pytest: test for offer failing decode when empty description. * offerout: don't insist that invoice vendor match our vendor. * offer / offerout: return existing if its still active. * disableoffer: fix disabling of already-used offers. * fetchinvoice: don't consider "msat" a change if fetchinvoice specified it. * offers: make sure quantity limits are sane. * bolt12: import latest spec (timestamp -> created_at). * funderupdate: add documentation, update some params * printlogs for failing tests * common/lease_rates: more things const. * lease_rates: lease_rates_eq. * gossipd: de-duplicate code. * lease_rates, funder: use overflow helpers * overflows: helpers for integer assignment overflows. * reestablish_tlvs: don't lose them btw tmpctx cleanup * queryrates: make it dev-only * lease-channels: tests for the test gods * dualopend: hold onto open_tlv for longer than the tmpctx persists * channel-leases: add lease fee to accepter's funding earlier up * nit: better error messages on dualopend's check_balances failures * channel-lease: reject if we're not currently advertising liquidity * channel leases: add expected lease fee to opening PSBT * channel leases: pass expected lease rates around in compat form * lease rates: helpers for parsing, serializing + equating lease_rates * channel lease: update the blockheight on reconnect, also * close: param to force-close a leased channel * channel lease: fail leased channel if peer offline + blockheight behind * channeld: don't forget channels that we've put funds into * utxo: don't add outputs that aren't currently eligible to be spent * channel utxo: persist the 'csv' lock value to database * channel lease: set csv_lock for commitment transactions * chaintopology: new command 'parsefeerate' * liquidity-ad: wire in csv updates to commitment txs * channel: wire blockheight updates from channeld in to database * liquidity-ads: persist channel blockheight states to disk * lease_rates: persist channel's lease info * onchaind: add recovery scans for option-will-fund * lease_rates: pass in 'lease_expiry' and 'csv' to commitments/channel * lease_rates: add csv lock to modify anchor scripts * script: add csv_lock to scripts * lease-rates: add to accepter's side on accepter pathway * lease-rates: calculate the fee owed to peer for the funds lease * channel-lease: validate accepter's sig on lease terms * rpc: add queryrates * dualopend, openchannel_init: pull out feerate parsing logic * liquidity-ad: fill in acceptance response * liquidity-ad: pipe `request_amt` all the way out to fundchannel * liquidity-ad: request amount, pass through to dualopend * liquidity-ads: pipe through request to funder, update policy * hsmd: method to sign liquidity ad offer * amount: add 'is_zero' helper * tests: add test for setting/updating node announce * contrib: offer a liquidity-ad lease in developer mode * funder-liqudity-ads: support options + setliquidity ad * funder_policy: set default to 0 * funder: make policy a pointer, most places * gossipd: turn off lease offers if the rates are empty * libplugin: add u16_option parsing * option_will_fund: print out lease rates in listnodes * setleaserates: new RPC to pass in lease rates * gossipd: signal whether node_ann is updated, are we starting up? * gossip/liquidity-ad: node_ann comparison, optional TLV check * lease_rates: parse them out of node_announcements * liquidity-ads: import from spec * dual-funding: update fee_step to be a feerate * change fees: more accurate rounding for change amount * Github actions: reduce lnprototest. * chore: add 3 gpg pubkeys of maintainers to the repository * docs: fix typo and wording * added minimum word list needed * improve hsm_secret doc * pytest: rescan restarts plugin on update * plugin: rescan restarts plugin on update * pytest: Add a `keysend` test that includes `routehints` * keysend: Add `routehints` argument to the `keysend` command * jsonrpc: Add parsers for routehint-arrays * opts: Add option to register extra TLV types to accept * sendcustommsg: promote to non-dev * route: assume modern TLV support if we don't see a node_announcement. * lightningd: insist on payment_secret if bit compulsory on invoices. * lightningd: make invoices insist on payment_secret. * pytest: make test_sendonion_rpc use a TLV onion for final hop. * plugins/keysend: add the payment_secret to the payload. * pytest: always provide payment_secret when making payments. * invoice: output payment_secret in JSON. * pytest: listchannels filter by destination * doc: listchannels filter by destination * pyln: rpc support for listchannels by destination * topology: listchannels by destination * plugins/offers: fix crash when we report unpaid prev invoice. * doc: fix up bolt12 quotes. * bolt12_merkle: change offer merkle to latest spec. * DEVELOPER: allow fetchinvoice to force they payer_secret. * offers: don't crash in decode if there's no signature on the offer. * patch remove-payer-backcompat.patch * tests: update coin movements for dual-funded mutual closes * tests: update deprecated 'funding_msat' for new output format * schema: fixup for dual-funding things * schema: add 'funding_outnum' to openchannel_update schema * lightningd: don't complain about unable to estimate fees if not mainnet. * lightningd: add force-feerates option. * channeld: don't ever use 0 as a feerate. * listinvoices: support listing by local offer_id. * invoices: display the payer note if it's for local offer, allow in fetchinvoice. * offers: include the payer_note field in the invoice if it's in the request. * BOLT 12: update to include replace_invoice options. * offers: signatures are now optional. * features: renumber from 102/103 to 38/39. * libplugin: create debug message for notifications * pytest: test for autoconnect in different circumstances. * pytest: test that we try direct connection even for public nodes. * fetchinvoice: try to connect to note if we can't find a path for messages. * listinvoice: populate local_offer_id when iterating whole db. * listinvoice: don't crash if listinvoice given a non-bolt11 string. * fix: remove spammy debug statements from getroute * lightningd: print out what errno we got if unexpected in sigchild. * closingd: use a more accurate fee for closing fee negotiation. * common: log at debug level when we update billboard. * lightningd: fix typo in debug statements. * pytest: fix weird case in test_closing_different_fees * doc: fix * sort alphabetically * add binaries to gitignore * pytest: Add test for keysend extra-tlvs * pyln: Add wrapper for the keysend command * keysend: Accept a payment with unknown even TLV types * keysend: Add `extra-tlvs` option to keysend * tlv: Add sorting of the tlvstream * jsonrpc: Add parsing for parameter of extra parameters * plugin: Add extra TLVs to the invoice_payment hook * htlc: Keep the parsed TLV payload around with the htlc_in * htlc: Wire extra TLV types into the HTLC onion decoding * tlv: Allow passing some extra types to accept when parsing the stream * opts: Add option to register extra TLV types to accept * pytest: Add test that we can redo closing negotiation even after onchain. * channeld: send shutdown_complete even if reestablish_only. * openingd: tell lightningd if we get a reestablish. * channeld: add a message to read if we only want to reestablish. * lightningd: add channel_closed helper, expose find_channel_by_id(). * common/read_peer_msg: don't try to handle reestablish/reopen. * pytest: add test (failing) which allows talking about closed channels. * lightningd: always pass closing connections through channeld. * pytest: test for closing needing retranmission bug. * dev_disconnect: new option to stop using connection, but don't close. * tools/check-spelling: Fix grep error on macOS * tools/mockup: Fix macOS bash 3 syntax error * Update at latest version of lnprototest with the support of channel_type * Update lnprototest submodule * doc: schema for listconfigs, compare with documentation. * lightningd: clean up listconfigs output. * doc: schemas for everything else. * doc: listpeers schema. * tools/fromschema.py: handle deprecated null field, don't create empty lists. * doc/schemas: listfunds, listinvoices, listnodes, listoffers * doc/schemas: listfunds * doc/schemas: getsharedsecret, help, invoice, listchannels, listforwards * doc: document how to try to write JSON schemas. * doc/schemas: tighten requirements. * plugins: add `command` field to subcommand output. * listtransactions: don't create a msat field called "satoshis". * listpeers: clean up JSON output. * pyln.proto: fix receiving unknown fields in TLVs. * pyln.proto: fix handling of subtypes in TLVs. * EXPERIMENTAL_FEATURES: don't offer to downgrade anchor outputs! * cleanup: remove unused pyln-spec symlinks. * replace freenode with libera * pay: Fix another instance of a crash when we can't find ourselves * invoice: allow creation of giant invoices. * pyln-testing: check if process died after having read all logs * hsm_encryption: merge getline() logic between stdin and non-stdin * qa: test hsm encryption password provided from stdin * hsmtool: this is 'success', not 'succes' * pyln: adds a help text to start plugins manually * plugins/bcli: fix uninitialized variable. * connectd: fix advertizing for Tor ports. * common/test: test wireaddr parsing. * lightningd: wait for gossipd to finish initalizing before starting plugins. * gossipd: remove routing, listchannels and listnodes infrastructure. * gossipd/test: remove routing tests. * pyln-client: revert old/new comparisons for getroute, listchannels, listnodes. * signmessage: use listnodes instead of gossipd_getnodes_request. * invoice: overhaul routehints to use topology.listincoming, cleanup. * plugins/topology: listincoming function to help invoices. * pyln-client: hack in test that listnodes returns the same old/new. * plugins/topology: add listnodes command. * common/wireaddr: fromwire_wireaddr_array helper. * pyln-client: hack in test that listchannels returns the same old/new. * plugins/topology: make listchannels mark disconnected local channels active=false. * plugins/topology: listchannels command. * pyln-client: hack in test that getroute returns the same old/new. * topology: plugin to implement getroute command. * common: note that command_fail doesn't return NULL. * ccan: update to get RETURNS_NONNULL macro. * pytest: fix flake in test_misc.py::test_funding_reorg_* xxremote_lags * wire/Makefile: always generate non-exp files. * Makefile: `pytest` target should depend on ALL_TEST_PROGRAMS. * close: check that destination is going to be accepted. * tests: test future segwit versions for withdraw / close. * common/json_tok: allow non-v0 segwit addresses. * devtools/encodeaddr: tool to create regtest bech32m vectors. * common/bech32: update to bech32m reference. * lightning/pay: don't crash when trying to send payment down unconfirmed channel. * chore: fix compile issue discovered by gcc * keysend plugin: make cltv_expiry for keysends slightly more conservative * plugin: Add debug log entries when calling and returning from hooks * plugin: Add a status field to htlcs in listpeers * lightning: Add status field to htlc_in * plugin: Add the plugin we're serializing for in the serializer * pay: Do not attempt to filter routehints if we can't find outselves * pytest: speed up test_opening.py::test_funder_contribution_limits when !DEVELOPER * pytest: test that both sides refuse upgrade if not quiescent. * pytest: test onchaind handles unilateral and penalty txs on either side of upgrade. * channeld: set desired_type to upgrade `option_static_remotekey` if not already. * channeld: handle upgrade match. * onchaind: handle static_remotekey thresholds. * wallet: save thresholds for option_static_remotekey. * channeld: send upgradable types, add logging (EXPERIMENTAL_FEATURES) * channeld: send current features (EXPERIMENTAL_FEATURES) * channeld: send next_to_send if EXPERIMENTAL_FEATURES * channel: import upgrade spec. * common/features: helper to pretty-print feature bits. * channeld: tweak function to allow testing for pending *uncommitted* changes. * onchaind: limp along if we cheat. * onchaind: don't hand redundant commit_num to handle_unknown_commitment. * onchaind: don't rely on knowing option_static_remotekey for unknown commitments. * gen/impl_template: fix generation of singleton varsize elements. * stop requiring channel and direction on `sendonion` since they're not used. * hsm_encryption.c: fix source include order * hsm_encryption.c: remove whitespace from line end * lightningd: remove duplicate temp term creation * hsm_encryption: read from STDIN if not in a TTY * pay: Skip the presplitter mod if it'd exhaust our HTLC budget * pay: Compute the number of HTLCs needed after presplit correctly * plugin/pay: prevent presplitter from dividing if max already tiny. * pay: Fix use-after-free issue with routehints in shortlived payments * pytest: show problem with pay when not enough HTLCs available. * openingd/dualopend: remove send_msg functionality. * lightningd: use connectd to send the final error message, not openingd/dualopend * connectd: new command to make us send message and close. * remove checking for unused "direction" property in sendpay. * openingd: return more informative error message for v2 opens * tests: cleanup any unfinished/hanging channel opens * Revert "pytest: Skip hsm encryption test if we don't have a TTY" * reqs: Remove wallycore, no longer needed for CI tests * libwally: update to release_0.8.3 * EXPERIMENTAL: dev-quiesce to initiate (and test) quiescence. * EXPERIMENTAL: handle receiving quiescence request. * channeld: implement pending_updates() * EXPERIMENTAL: import spec for quiescence. * doc: document ordering of listpays/listsendpays. * pytest: test pay ordering. * plugins/pay: sort output payments in listpays * wallet: sort listsendpays by ID * Suggested code cleanup by TODO comment * libhsmd: Do not use the message as the peer ID... * libhsmd: Add a simple tx signature test * libhsmd: Always use gcc to configure and compile * libhsmd: Correctly wrap with `tmpctx` management in libhsmd-python * lightningd: check closing tx signature Changelog-None * plugins/test/Makefile: fix typo causing build race. * doc/schemas: disableoffer, disconnect, feerates, fetchinvoice, fundchannel, fundchannel_cancel, fundchannel_complete, fundchannel_start, fundpsbt, getinfo, getlog, getroute. * doc/schemas: delexpiredinvoice, delinvoice, delpay. * doc/schemas: decode, decodepay * doc/schemas: schema for connect. * doc/schemas: create close schema. * doc/schemas: add schema for autoclean. * doc/schemas: add some simple schemas and generate manpages from them * doc/schemas: generate manpage section from schema. * tools/fromschema.py: tool to replace start/end markers in markdown with schema. * pytest: add schema support for JSON responses. * decode: always return "valid" field. * close: add "unopened" type if we simply discard channel. * libplugin: remove command_success_str function. * autoclean: don't return a raw string as result. * pytest: don't use command_success_str in test_libplugin.c * doc: fix up mangled nroff from mkrd. * Makefile: allow postfixes to SHA256STAMP. * Makefile: fix $(FORCE) in sub-Makefiles * Makefile: update to latest spec. * channeld: update test vectors for msat differentiation test. * lightningd: option_shutdown_anysegwit is no longer experimental. * lightningd: respect anysegwit on dual-funding opens too. * fundchannel, multifundchannel: reserve inputs for two weeks, not 12 hours. * sendpsbt, fundpsbt, utxopsbt, reserveinputs, unreserveinputs: allow custom number of blocks to reserve. * pytest: test double-spending an opening tx input. * wallet: add "reserved_to_block" field to listfunds. * common: Check chain hash in gossip_timestamp_filter * pay: fix corner case where we eliminate our own hint. * This should hopefully alleviate the pain of figuring out how to properly quote utxos in fundchannel command :) It took me a while to figure it out and I would be lost without @cdecker s help on IRC. * CI: Disable dual-funded tests while they're hanging. * pytest: add message check to test_blockheight_disagreement * pay: don't wait forever if we're already past deadline. * pytest: fix gossip flake in test_restart_many_payments * CI: give nicer names to Ci runners. * doc/TOR: remove references to deprecated v2 hidden services * options: deprecate v2 mode for autotor * Refuse to parse v2 onion addresses without deprecated_apis * wireaddr: fix indentation / line breaks for autotor parsing * pytest: check nonstandard onion generation. * devtools/onion: handle onions which are not 1300 bytes long. * "createonion" to accept an optional custom onion_size. * readme: fix ubuntu installation instructions * Clearer RETURN VALUE * added FAQ on getting a PSBT * dual-open: save our now updated info about broadcast state * rbf-test: confirm that rbf'd inflight is opened and closed as expected * nit: add another debug log * rbf: when a peer is activated, also keep track of all of its inflights * rbf: when a channel is open, remove all the inflights * rbf: update the channel's funding_txid to match what's mined * inflights: relax assertion channel funding_txid is last inflight txid * open-rbf: we broadcast all the commitments for a channel when closed * rbf-tests: check every inflight has signed commitment produced * inflights: save the whole psbt to the database * dev-sign-last-tx: include inflight signed txs * listpeers: always show all the inflights * listpeers: show the inflight's 'commitment tx' txid * rbf: on close, drop every inflight transaction's commitment * df-rbf: test that, with enough bumps, we can eventually add to mempool * listpeers: add inflights info * inflights: add checks that there's actually an inflight * df-rbf: order inflights by funding_feerate * df-tests: add better explainer for test logic * df-test: test for a failed rbf attempt, currently crashes * df-tests: re-write test_funding_cancel_race for v2 commands * df-tests: re-write the funding wallet corners test for v2 * df-bugs: only include the `funding` if we're the opener * df: differentiate error message between * CI: Re-enable experimental build. * funder: print reason that we don't contribute funds * funder: test for adding utxos, calculating total available * funder: subtract the fee of the utxo from the available amount * fundpsbt: dont add utxos that are a net-loss * funder: add count of utxos we're signing to the logs * pytest: disable mpp test which is blocking merges. * dijkstra: add chan pointer argument to path scoring. * common/dijkstra: remove dijkstra_amount(). * common/route: route_from_dijkstra returns route_hop array. * gossipd / plugin: clean up names in struct route_hop. * common/gossip_store: avoid fd pass for new store, use end marker. * common/gossmap: reload once we hit end marker. * gossip_store: create end marker for EOF. * gossmap: fix gossmap_chan_get_announce() and gossmap_node_get_announce(). * gossmap: function to get channel features bitmap. * gossmap: save the offset where the channel updates are, function for details. * gossmap: add gossmap_chan_get_capacity() helper. * gossmap: remembers when a channel is private. * common/json_tok: hoist param_short_channel_id from inside lightningd/ * plugins/test/run-route-overlong: modern overlong routing test. * plugins/libplugin: make headers update-mocks friendly. * plugins/libplugin-pay: extreact route function so we can write unit tests. * common/test/run-route and run-route-specific: modern gossmap tests * fetchinvoice: don't produce invalid JSON on sent_inv timeout. * update-mocks: prefer to find function defs from local includes. * Makefile: don't let update-mocks run unless DEVELOPER and EXPERIMENTAL_FEATURES. * update-mocks: make sure we cover all test programs. * update-mocks: don't try to regenerate files. * lightningd: attach HTLC timeout to htlc itself, fix gratuitous disconnect bug. * pytest: add test for HTLC timeout on failed htlcs. * reorganize .gitignore entries across subdirs. * gitignore more things. * pytest: fix flake in test_reconnect_no_update * make: Do not scan contrib for discouraged functions * make: Add CPPCHECK_OPTS to allow excluding files from the check * libhsmd: Clean up the build descriptors for libhsmd_python * libhsmd: Add missing implementation of status functions * libhsmd: First working build script for libhsmd-python * plugin notifications: minor cleanups. * df-test: make flakey test not as flakey * df-bugs: if we've already got the funding_locked, dont renotify * df-tests: test_coin_movement_notices, make work? * df: rework closing logic * tests: mark test as requiring developer * df-test: v2 has reconnects * df-callbacks: dont log as broken, just reconnect * df-bug: avoid referencing null channel->owner * df: patch for valgrind error * df-tests: patch for state == AWAITING_UNILATERAL problem * df-tests: Turn on v2 tests on CI * tests: explicitly mark tests to run both as v1+v2 * tests: mark as v1 or v2 openchannel required * tests: add marker for v1/v2 channel opens * tests: not DEVELOPER -> mark.developer * df: anchor outputs are on if EXP_DF * tests: move EXP_DF into the testing utils * df-tests: remove df_accepter plugin * psbt: really hacky patch over libwally's 0-input txs parse fail * dual-fund: reconnections were borked, this fixes them * funder, startupregtest: fixup default config to be 100 * df-bugs: rm duplicate call to channeld * !fixup * bcli: become less aggressive with onchain fee levels. * plugins/bcli: make feerate calls more changeable. * lightningd: implement --log-timestamps=false. * libhsmd: Prefix check_client_capabilities * libhsmd: Cleanup the bip32_key_version * libhsmd: Finalize encapsulation of hsmd data in libhsmd * libhsmd: Migrate bolt12 initialization into hsmd_init * libhsmd: Start migrating initialization to hsmd_init * libhsmd: Clean up includes in hsmd.c * libhsmd: Migrate handle_sign_delayer_payment_to_us * libhsmd: Migrate handle_sign_remote_htlc_to_us * libhsmd: handle_sign_commitment_tx * libhsmd: Migrate handle_sign_penalty_to_us * libhsmd: Migrate handle_sign_remote_commitment_tx * libhsmd: Migrate handle_sign_remote_htlc_tx * libhsmd: Migrate handle_sign_local_htlc_tx * libhsmd: Migrate handle_sign_mutual_tx * libhsmd: Migrate handle_sign_withdrawal_tx * libhsmd: Migrate handle_get_per_commitment_point * libhsmd: Migrate handle_channel_update_sig * libhsmd: Migrate handle_sign_node_announcement * libhsmd: Migrate handle_cannouncement_sig * libhsmd: Migrate handle_get_output_scriptpubkey * libhsmd: Migrate handle_check_future_secret * libhsmd: Migrate handle_ecdh * libhsmd: Migrate handle_get_channel_basepoints * libhsmd: Migrate handle_sign_invoice * libhsmd: Migrate handle_sign_bolt12 * libhsmd: Add dispatcher function * libhsmd: Add status functions to report to whoever is listening * libhsmd: Migrate check_client_capabilities to libhsmd * libhsmd: Add scaffolding to start separating hsmd from io logic * plugins/bcli: strip trailing whitespace appropriately * plugins/funder: fix up compilation. * docs: Document the custom plugin notifications * pay: Add pay_failure notification * pay: Add notification for pay_success * plugin: Make unannounced notification topics no longer fatal * libplugin: Add functions to start and end notifications * libplugin: Add notification topics to the manifest response * libplugin: Add notification topics to plugin_main * cleanup: Make lnprototest run only with DEVELOPER=1 * plugin: Restrict plugin notifications only to announced topics * plugin: Wrap custom notifications in a dict with additional origin * plugin: Remember the shortname for a plugin * plugin: Move list of notification topics to each plugin * plugin: Prevent plugins from registering native notification topics * plugin: Implement custom notification dispatch for plugins * plugin: Store the notification topics announced by the plugins * plugin: Move the notification subscription check into a second phase * plugin: Add a list of notification topics registered by plugin * tests: add test for funder options * pyln-testing: use provided outnum instead of trying to find it * funder: handle RBF callback * rbf_channel hook: add channel_max_msat parameter * funder: `funderupdate` command to see/set configs * funder: sanitize inputs * contrib: if you're in dev mode, use dual-funding (with matching) * funder-test: tests for our policy configurations * funder: use listfunds to fetch utxo data * funder: add a plugin, `funder`. policies for dual-funding * amount: `amount_sat_scale` method * openchannel: add missing string args to format string * dualfund: set the locktime for the user-provided PSBT * openchannel2/rbf hooks: reject if response malformed * psbt-open: method to quickly check if has our input * openchannel2: add channel_max_msat to openchannel2 hook payload * dualopend: don't use final channel_id for accepter_start2 * plugin-tests: cleanup outdated openchannel2 hook fields * tests: remove EXPERIMENTAL_FEATURE flag from openchannel2 hooks * libplugin: add no-op command complete function * test/run-commit_tx: test static remotekey. * test/run-commit_tx: test HTLC differentiation. * tests: make run-commit_tx output match the BOLT test vectors more closely * channeld: fix test code for tx fee calculation (inside `#ifdef PRINT_ACTUAL_FEE`) * bitcoin: allow developer override of signature grinding. * pytest: Fix racy `test_unreachable_routehint` * docs: Add documentation of --experimental-dual-fund option * channeld: check if we want to change fees after sending revocation. * channeld: increment fee states properly. * channeld: don't ever send back-to-back feerate changes. * pytest: fix race in test_penalty_htlc_tx_timeout * pytest: make it work with latest bitcoind master branch. * lightningd: don't send other messages until we've received version. * gossipd: don't start listening to connectd until we get init message. * lightningd: don't abort on incorrect versions, but try to re-exec. * subd: send versions at startup. * status: new message for subdaemons to tell us their versions. * CI: make sure we do a fuzzing build. * tests/fuzz: update for fmt_amount changes, create pseudorand for fuzzing. * dual-funding: rework how funding feerates are passed around * dual-funding: bugfix, swapped commitment/funding feerates * dual-fund: on witness failure, route through dualopend * psbt: temporarily patch over failure to parse the empty tx * closingd: handle custommessages * dual-fund: update state before checking for 'doneness' * peer_control: fix leak false positive. * pytest: fix flake in test_v2_open_sigs_restart_while_dead * pytest: handle case where funding tx is not tx #1. * pytest: fix flake in test_pay_disconnect * pytest: fix spurious failure in test_local_basepoints_cache. * lnprototests: update to latest, which adds wait for test_query_channel_range * ci: turn on proto-tests for the CI builds * pyln: Pin mypy to version 0.790 since later has import path issues * gci: Remove already included requirements.txt files * pyln-spec: Clean up the setup.py files * pyln: Change the setup.py file not to import the package * make: Setup the PYTHONPATH to include the pyln-spec packages * python: Use relative URLs for pyln-{client,proto,testing} * make: add lnprototests to check!! * gossip: allow us to only look at 1 block * make: include lnprototest in pythonpath * tests: add submodule for lnprototests (points at branch right now) * plugins: remove now-unused single-hook infrastructure. * newaddr: don't include "address" field. * plugin: always send allow-deprecated-apis in getmanifest. * lightningd: remove unused `original_directory` field. * chaintopology: remove deprecated urgent/normal/slow feerate display. * devtools: add experimental section. * pyln: remove deprecated fundchannel/fundchannel_start variants. * pyln: remove deprecated pay/sendpay variants. * pyln: remove deprecated close variant. * close: activate notifications even with deprecated-apis. * Makefile: make it easier to check bolt text. * doc: remove suffix for included-in-master BOLTs. * Makefile: update to latest BOLT versions. * pay: Use gossmap for reachability check * wire: clean up patch revision files in clean ------------------------------------------------------------------- Wed Mar 31 10:39:57 UTC 2021 - marec@detebe.org - Update to version 0.10.0: * CHANGELOG.md: update for v0.10.0 * gossipd: assert we're not blatting the version header. * gossip_store: don't make bogus assumption that writes are atomic wrt readers. * pyln-proto: make an error message more detailed * pyln-proto: write out length of arrays of subtypes to wire * dual-funding: introduce racy state * v0.10.0-rc2 * CHANGELOG.md: update for recent changes. * lightningd: set "direction" correctly for connect which is already connected. * lightningd: get connection direction from connectd. * gossipd: fix rolling corruption. * df-tests: update to take 'no channel_id' into account; use exp-df flag * pytest: fix up rbf tests which use dev-disconnect. * features: match up feature names to feature bit * dual-funding: convert to runtime flag, --experimental-dual-fund * dual-funding: use correct feature bit number!! * dual-funding: implies anchor outputs. * connectd: clearly differentiate incoming and outgoing paths. * test_v2_open_sigs_restart: dont mine blocks until funding tx in mempool * df-tests: make bad tests a bit better, maybe * common: fix MacOS compile error. * ccan: import latest * doc: Remove leftover conflict marker in the docs * 0.10.0-rc1. * CI: Disable EXPERIMENTAL_FEATURES temporarily. * plugin: Call invoice_payment hook before the matching notification * keysend: Add error when trying to send to self * keysend: Fix a use-after-free error * channel: initialize inflight->tx_broadcast (EXPERIMENTAL_FEATURES) * mfc: turn "UNKNOWN TYPE channel_id" logs into actually the channel id * df: more reconnection tests * df: actually save the peer's sigs transmission to disk * dual-funding: don't not update the state! log the issue and move on with * df: don't ignore their sigs, even if we've already seen them * df: move from warning to unusual * df-broadcasts: use an impermanent marker to make sure we've sent things * channel: why were these commas in the first place * connect: return address we actually connected to. * channeld: allow remote node to exceed their own HTLC count limits. * remove duplicated erring_node from paystatus. * pytest: fix now-obsolete use of fundchannel_complete via txid. * lightningd: don't complain about bad funding PSBT for elements. * lightningd/opening_control: deprecate old fundchannel_complete args. * txprepare, withdraw: return the PSBT as well as raw tx. * python: handle psbt-version of fundchannel_complete, deprecate old. * lightningd/opening_control: allow single-arg fundchannel_complete with PSBT * lightningd/opening_control: store funding scriptpubkey. * lightningd/opening_control: d_o_n_t a_d_d e_x_t_r_a u_n_d_e_r_s_c_o_r_e_s * onchaind: see closes when wrong_funding shutdowns are used. * close: accept wrong_funding outpoint arg if we negotiated the feature. * closingd: add support for handling wrong_funding. * channeld: code to send wrong_funding if lightningd says to. * channeld: accept the 'wrong_funding' shutdown TLV. * bitcoin: add fromwire/towire routines for bitcoin_outpoint. * wallet: out "wrong_funding" information in the db for persistence. * protocol: add TLV for shutdown message, use 100 as "wrong_funding" outpoint. * Experimental option option_shutdown_wrong_funding: help me, I screwed up! * typo: information is an uncountable mass noun * added the default bitcoin path for macos to startup_regtest.sh * contrib: add the dev- options to the config iff DEVELOPER=1 configured * mfc: use consolidated error reporting, reduce reliance on json-str * mfc: check that we can retry when things go haywire * mfc: have error messages for redos be more robust (include why) * mfc: add convenience method for determining 'parity' * df: add openchannel_abort command * df: nit, update to use proper helper json function for channel_id * Fix wallet creation in contrib/startup_regtest.sh * df: update the openchannel2 parameter 'accepter' -> 'our' * df: add missing check for already set scriptpubkey * docs: add notes on v2 openchannel plugin hooks * nit: change spacing on PLUGIN doc * df: add doc for `channel_open_failed` notification * df-doc: add docs for openchannel_bump, more checks for valid psbt * df-doc: update error codes, make sure they're correct * CHANGELOG: add title and namer to last release. * mfc: move input validation down to input acceptance * mfc: final inversion! * mfc: move fundchannel_starts down * mfc: move connection beneath funding * mfc: invert some more * mfc: begin inversion, start at bottom as well as re-try * mfc: rm call chain inversions + declarations * mfc: invert call-chain, remove declarations * mfc: rm declarations; invert call chain * mfc: rm declarations, invert ordering * mfc: cleanup static declarations/invert call chain * mfc: uninvert/remove declarations for mfc cleanups * pay: Add functions to tell bolt12 string apart * pay: Do not clobber error messages if we fail decoding a bolt11 * pay: Abort the payment if destination is unreachable * pay: Remove routehints with unreachable entrypoints * pay: Filter out routehints whose entrypoint is unknown * keysend: Set an error message if we abort due to missing TLV * libplugin: Introduce `payment_abort` to terminate a payment * pay: Do not mark in-flight parts as failed * pay: Keysend tries to send even if featurebit 55 isn't set * pyln: Fix listfunds call defaulting to spent=False * configure: Pick up the correct pytest executable * plugin: Add details about which plugin caused a clash in RPC methods * plugin: Abort early if we have a misconfiguration in the plugins * pyln: Plugins can be nice if invoked from CLI too * gci: Disable test for NO_PYTHON * wallet: Do not generate basepoints in the hsmd * wallet: Load local basepoints from the database alongside the hsmd * wallet: Store local channel basepoints when a new channel is created * db: Add migration for the local basepoints and the funding pubkey * db: Add the migration_context to hold info required by migrations * wallet: Add migration to cache local basepoints from the HSMd * df-spec: limit allowable inputs/outputs to 252 * df: fail channel if peer sends witnesses that aren't paid for * bitcoin: method for finding weight of 'wally_tx' * psbt: mark psbt arg as const for `psbt_compute_fee` * df-spec: use an empty bit-set as the basepoint for chan-id at start * df-spec: consolidate dual-funding patches, update feerate protocol * dual-fund: remove all references to PODLEs * hsmd: Ensure the hsmd is initialized before anything else is called * plugin: Do not send the internal framed message over the wire * connectd: clean up the channel stuffs when we get a reconnect * df: add (over zealous?) note about the usage of `psbt_has_req_fields` * df-memleak: expose memleak error and fix * Revert "df: turn off integration tests that depend on v2 funding" * df-tests: check that rbf includes overlapping inputs * df-tests: check that multiple RBFs and tx-sigs errors during RBF work * df-rbf: test for interrupts during tx construction * df-rbf: reconnection tests (init_rbf + ack_rbf) * df-tests: happy path rbf test * df: cleanup error handling on lightningd side * df-rbf: update rejection criteria for tx_sigs * listpeers: include feerate info for RBF-candidate channels * channel: move inflight + feerate methods to channel.c * listpeers: include info on channels that are in-process * df: handle funding_locked + tx_signatures messages out of order * df: move methods around so we can call them elsewhere * rbf: clean up channel on error * rbf: consolidate failure paths, use "warnings" * df-rbf: separate first-time channel save from update (RBF) * df-rbf: only permit the channel initiator to init RBF * df: update reserve calculation * df-rbf: handle rbf failures in lightningd * df-rbf: init an rbf for real, using openchannel_bumpfee * listpeers: add latest feerate and actual last fee amount. * Makefile: fix parallel build error on clean tree. * wire/Makefile: make sure we regen exp files if patches change. * df-feerates: send back original + last known feerates * df: revert channel state in dualopend * df: allow for RBF_INIT in sigs states * dualopen: check that peer is connected for json calls * df: update channel_id when opening; closing; disconnecting * memleak,df: patchup memleak code for v2 channels * df, channel: if a channel reconnects, but it's "unsaved" still * df: check inflights when deciding whether or not to allow command * channel: skip unsaved channels * connectd: check for re-connected unsaved channels * subd: remove "swap" methods * subd: remove ctype (channel_type) * df: start of RPC for bumping the fee! * df: do the right thing, now that we have a cancellation routine * df: move to inflights for sigs/openchannel_signed parts of flow * df: move initialization over to unsaved channel * df: get rid of unnecessary struct `commit_rcvd` * df: helper to find the last (most recent) inflight for a channel * df: consolidate callbacks for opener/accepter, move to open_attempt * df: method for disconnecting an *unsaved* channel * df: add helper for creating an "unsaved" channel * df: migrate from "uncommitted_channel" to "open_attempt" * df-rbf: hook for rbf_init attempt, called "rbf_channel" * channel: remove 'uncommitted_channel' from channel lookup * channel: remove `psbt` and `remote_tx_sigs` * channel: only delete saved channels from database * dual-funding, rbf: reject RBFs that will fail the total-fee rule * df-rbf: validate that the candidate funding transaction is valid * df-rbf: break out opener commitment codepath, use for rbf * df-rbf: opener setup for "interactive" protocol portion * df: move out logic for adding funding output to PSBT * df-rbf: split out "commitment txs" portion of accepter * df-reinit: pass back channel_flags * df: put channel_flags onto state * df-rbf: start of handling an incoming RBF attempt from peer * df-rbf: add method to fail an RBF * df-rbf: update wires with ack and fail rbf * df-rbf: refactor state for opening attempts * rbf: check all inflights for matching transaction data * df-rbf: update channel data on depth reached * inflight: add a 'channel-inflight' concept, in-progress channel stuffs * df: turn off integration tests that depend on v2 funding * nit: make comments 80 chars * Updated listforwards doc with new parameters * Added listforwards test * Added index to forwarded_payments.state to speedup lookups using this column * Added in_channel,out_channel,state params to listforwards * json: fix oom when adding a long string * pytest: test rpc_command hook chain * doc: make rpc_command hook chainable * plugins: make rpc_command hook chainable * pytest: reduce test to edgecases for CI. * common: support opt_shutdown_anysegwit checks (EXPERIMENTAL_FEATURES). * bitcoin/script: use script opcode constants from wally_script.h * openingd: check upfront shutdown script. * common/shutdown_scriptpubkey: extract shutdown scriptpubkey test. * plugin: Fix the custommsg hook not to include the internal prefix * pytest: test fee estimates which start working after channel establishment. * chaintopology: fix notification first time fee estimate works. * plugins/bcli: fake minimum fee if we're in regtest mode. * test: Fix memory leak in unit test run-gossmap_local * doc: Fix minor markup issues in the newaddr manpage * plugin: log self-disable at debug level, other manifest fails at unusual. * plugin: add log-level to plugin_kill, make it take format string. * lightningd: reap zombies (particularly plugins). * lightningd: differentiate cases of plugin death correctly. * Minor wording * pytest: fix "bad gossip" flake in test_forward_event_notification * offers: handle re-fetching the same invoice twice. * offers: make 'used' flag more useful. * doc: add mpp warnings to lightning-invoice(7) * doc: add disableoffer and listoffers manpages. * doc: fix up incorrect punctuation in lightning-fundchannel page. * contrib: make startup_regtest.sh work for modern bitcoind. * gossipd: use modern 'sync_complete' field. * doc: update BOLT quotes (to include links for anchor outputs). * doc: update BOLT quotes (to include same-transmission-order rule). * doc: update BOLT quotes (to include preimage leakage). * doc: update BOLT quotes (to include LOW-S-standard rule). * gen: Commit derived files for PR #4378 * db: Set the failonion to the default value on creation * db: Set the failonion on channel_htlcs if is is not set * df: more descriptive error message * psbt: method to calculate fee paid on PSBT * df-tests: send payment over opened channel * df: patch hole where we wouldn't notify peer of failure * df: use simplified version to print json * psbt: method to clone a PSBT onto a context * tests: test_no_fee_estimate robustify * gitignore: ignore libwally-core external directory * gitignore: ignore experimental unit test build files * Fix typo in doc/BACKUP.md * pytest: fix flake in test_no_fee_estimate() * pytest: fix pay flake. * readme: Replace outdated Travis-CI badge with Github Actions badge * make: Add target to check generated/derived files were checked in * Regenerate derived files * wallet: Fix a potential memory leak when loading state changes * pytest: Stabilize test_onchain_feechange * pytest: Stabilize test_forward_event_notification * pytest: Stabilize funding cancel race by killing in parallel * gci: Split test configurations into groups * gci: Move the setup and build steps into their own scripts * pay: Check for self-payments before attempting to pay * plugin: Check that we have a non-empty route in libplugin * pytest: Reproduce #4345 * Regtest does not exist to the CLI * Added regtest, example, k and null explanation * doc: Update the recommonmark extension for RTD compilation * cleanup: Update generated files * df-tests: use `excess_as_change`, test two-sided channels better * reservations: add weight of fee-output to weight calculation * fund/utxopsbt: flag 'excess_as_change' to add a change output for excess * pytest: detect warnings, too. * common: disallow NULL channel_id to peer_failed_err. * common: remove peer_failed in favor of peer_failed_warn/peer_failed_err * common: infrastructure to construct warning messages. * common: treat all "all-channels" errors as if they were warnings. * lightningd: implement receiving warnings. * pytest: fix false bad gossip issue in test_forward * gossmap: change local API. * common: support transitory local map additions to gossmap. * common: fix gossmap min/max rounding. * common: extract fp16 routines into their own file. * check-includes: allow redundant "config.h" * pyln-testing: add `scids` param to `wait_for_htlcs` * pyln-testing: improve `wait_for` a bit * docs: document addgossip API. * lightningd: no longer forward failures to gossipd, let caller do it. * pay: inject channel updates from errors ourselves. * lightningd: addgossip API to inject gossip messages. * gossipd: add an internal flag to force a channel update * connectd: don't crash if connect() fails immediately. * lightningd: disallow --daemon without --log-file. * bitcoind: remove v0.9.0-compat for rejecting sendrawtransaction arg. * listsendpays: remove deprecated "null" amount_msat. * chore: cleanup some nits * pytest: peer_connected chainable tests * feat: make peer_connected hook chainable * doc: openchannel note close_to can only be set once * doc: document peer_connected hook chainable * chore: fix typo in openchannel hook log * pytest: improve test_openchannel_hook_chaining * pytest: custommsg chainable tests * doc: document custommsg hook now chainable * feat: make custommsg hook chainable * pyln: Catch OSError when cleaning up test directories * travis: Goodbye Travis, hello github actions * gci: Stabilize test_forward_event_notification * gci: Add the JSON report plugin to the ci configuration * pyln: Pretty print RPC calls in the testing framework * gci: Give each configuration an CFG value to identify them later * gci: Pin mypy to version 0.790 since 0.800 gives strange errors * gci: Upload the junit.xml report * gci: Switch to the flaky plugin * pytest: Stabilize test_forward_stats * pytest: Stabilize test_routing_gossip * pytest: Give each run of the hsmtool its own pty * pytest: Stabilize test_funding_close_upfront * pytest: Stabilize test_closing_negotiation_reconnect * pytest: Stabilize test_bad_onion * pytest: Stabilize test_multiple_channels * make: Remove hardcoded timeout to pytest * gci: Add pytest.ini in order to randomize the groups * gci: Format the build script * pyln: Adjust maximum load allowed by the throttler * pytest: Parameterize process waits for hsmtool calls * pyln: Use a fair FS lock to throttle node startups * pytest: Stabilize test_channel_{spendable,receivable} * pytest: Stabilize the negotiation tests * pytest: Stabilize test_funding_external_wallet_corners * pytest: Disable test_v2_open if not developer * pytest: Stabilize test_channel_state_changed_bilateral * pytest: Stabilize test_setchannelfee_state * pytest: Stabilize test_gossip_persistence * pytest: Stabilize test_onchain_timeout * pyln: Update dependencies for all pyln packages * pytest: Stabilize test_penalty_htlc_tx_timeout * pytest: Simplify and stabilize test_reconnect_no_update * pytest: Skip hsm encryption test if we don't have a TTY * gci: Expand matrix to include all CI configurations * gci: Add a tester Dockerfile * gci: Add basic configuration for Github Actions as CI * pyln: Bump pyln version to 0.9.3 before publishing on PyPI * pyln: Pin the mypy dependency to 0.790 since 0.800 has a regression * git: Ignore some more generated files and artifacts * doc: Make the release process a bit more readable * make: Explicitly set PYTHON_VERSION=3 for libwally * tools: Pretty print the credits * tools: Allow versions to either have a v prefix or not * doc: Update release process document * devtools: Make the raw changelog more readable * Fix dockerfiles * nit df: correct spelling mistake * wallet nit: consolidate stmt free into one line * wallet: cleanup fee_states alloc on error ------------------------------------------------------------------- Fri Jan 22 16:17:31 UTC 2021 - marec@detebe.org - Update to version 0.9.3: * release: Update the changelog for v0.9.3 * hsmtool: don't streq() on NULL * wallet/db.c: Speed up deletion of single peers. * Added link to https://lightning.readthedocs.io/ * changelog: Add v0.9.3rc1..v0.9.3rc2 changes * doc/PLUGINS.md: Document the onion_message and onion_message_blinded hooks. * lightningd: don't expose (undocumented) experimental-dual-fund. * doc: note where experimental-offers is required or affects commands. * doc: Manual page for sendonionmessage. * lightningd: fix double-free when forking subdaemon fails. * fetchinvoice: don't abort() if we try to fetch an invoice from ourselves. * travis: Bump number of valgrind configs to 12 * offers: make it a runtime option. * common/bolt12: include correct bolt12 wiregen header. * tools/generate-wire.py: include digits in #ifndef idempotent header. * libplugin: make init return a string. * plugins: allow plugins to disable themselves at startup. * lightningd: control onion messages by `experimental-onion-messages` option. * wire: Apply onion mesages spec patch even when not experimental. * wire: add bolt12_wire.csv * wire/Makefile: refactor to allow non-experimental spec patches. * doc: Add changelog for version v0.9.3rc1 * autodata: requests new line number, please * df-tests: simultaneous openchannel_init (while in progress) * df-tests: test_lightningd_still_loading * df-tests: test_disconnect_half_signed + test_reconnect_signed * df-tests: test_reconnect_openingd * df-tests: test_reconnect_sender_add1, others * df-tests: test_reconnect_receiver_add * df-tests: test_peerinfo * df-tests: test_channel_persistence * df-tests: have the df_accepter plugin keep track of attempts * df-tests: test_disconnect_opener * df-tests: test_forward_local_failed_stats * df-tests: if we're using fundbalancechannel, set up plugin * df-tests: add `setacceptmaxfunding` to df_accepter plugin * pyln-tests: tweak `fundbalancedchannel` to assume peer will match * df-tests: test_payment_success|failed_persistence * df-tests: make memleak happy by cleaning up `psbt` * df-tests: test_io_logging * df-tests: test_channel_state_changed_unilateral * df-tests: test_channel_state_changed_bilateral * df-tests: test_coin_movement_notices * df-tests: test_channel_opened_notification * df-tests: test_feature_set * df-tests: test_plugin_feature_announce * df-tests: test_channel_state_change_history * df-tests: test_openchannel_hook_error_handling * df-tests: test_openchannel_hook_chaining * df-tests: test_openchannel_hook * df-tests: test_funding_reorg_private * df-tests: test_node_reannounce correct features * df-tests: test_fail_unconfirmed, duplicate with correct wires * df-tests: test_shutdown_awaiting_lockin check correct state * df-tests: adjust logsearch/features for test_dataloss_protection * df-tests: fixup expected features for dual-funding channels * df-tests: test_multifunding_v2_v1_mixed update to use flag * df-tests: updates for plugin hook_accepter helper * tests: don't enforce daemon name on opening logline * df: add flag to signal to run v2 channel open tests * pyln-testing: add 'config' method to a node * experimental-dual-fund: add config level option for enabling dual-fund * df: use dev-env flagged upfront shutdown script * df: free psbt on negotiation failed * df: add a new 'channel_open_failed' notification * df: include channel_id in openchannel2 hook * df: don't update local shutdown scriptpubkey from dualopend * df: if 'openchannel2' is "rejected" we should send that error * df: move callback up * df-reconnects: allow tx-sigs in channeld iff we're reconnecting * df: state, clean more things up * df: actually send a temp-id to the peer in open_channel2 WIP * df: assume this is a 'negotiation failure' * df: actually run 'reconnect' code * df: handle locked in etc in dualopend * df: reinitialize dual-funding * df: bunch of driveby formatting fixes * df-feerates: mimic old behavior if there's no plugin handling * df: there's only one state that we should be in for a `dualopend` * channel: save the fact that funding_locked was rcvd to disk * channel_fallen_behind: break out functionality into two methods * db: add flag to database for recving remote's sigs * temp_channel_id: move to common * dual-fund: use correct 'last_tx' * df: handle shutdown (before lockedin) in dualopend * df: handle fails * subd: keep track of 'channel's type * setchannelfee: allow dualopend's "awaiting" state to be valid also * feerates: mimic behavior of original opening if feerates not specified * billboard: add in billboard updates from dualopend * billboard: break out common method for billboard updates * dual-open: handle sigs and channel lockin in dualopend, not channeld * dual_open: move two functions * dual-fund: don't kill dualopend after commitments exchanged * channel_control: when we reach depth, for v2 channels, tell dualopend * channel-state: add two new states, for dualopend * channel_control: rename record_channel_open + expose * channel_control: break out function for on_funding_locked * dual_open_control: correct name of hook * open_command: keep them local to channel * autodata: tweak where createinvoice is * peer_control: remove PSBT from peer_start_channeld * whitespace changes, 80 chars etc * channeld: remove peer->psbt * subd: swap out the channel + error callback * df nit: capitialize acronym * df: open_channel -> openchannel * df: make sure to end wally alloc * df: move tx_sigs from channeld to dualopend * fetchinvoice: allow amounts to be specified. * offers: convert currency when they request an invoice. * offer: allow offers in other currencies if we can convert. * common/amount: make fmt_amount_sat / fmt_amount_msat etc take copy. * onion_message: support variable-length onion messages. * sendinvoice: don't allow separate invoice and command timeouts. * doc: fix waitinvoice update. * doc: update and fix listinvoices output in man page. * sendinvoice: call waitinvoice directly to on invoice payment. * lightningd: fix temporary onion_message leak: * decode: new generic API to decode bolt11 and bolt12. * common/bolt11_json: extract bolt11->json code. * offers: split offer send_invoice generation into new JSON command * doc: document experimental commands. * fetchinvoice: try direct peers if we can't route. * lightning/onion_message: hack to allow x-only guesses for first hop. * gossipd, channeld: handle onion messages in gossipd so we don't need a channel. * fetchinvoice: check we're in the period before attempting to fetch. * fetchinvoice: implement timeout. * plugins/fetchinvoice: handle sendinvoice timeout, error or payment. * plugins/fetchinvoice: implement `sendinvoice` command. * lightningd: new api payersign to sign bolt12 message with a payer_key. * plugins/offer: handle receiving an invoice in an onion_message. * plugin/offers: hoist send_onion_reply into core. * plugins/offer: allow `offer` to create `send_invoice`, `refund` and `single_use` offers. * sendonion: remove underscores from JSON arg name. * pytest: Mark `test_lockup_drain` as disabled * doc: Document existence of `erring_direction` in `sendpay` result * configure: Prefer pytest over pytest3 * tests: wait for drop to chain test_onchain_middleman_their_unilateral_in * pyln: Add ammag key to onion keyset * invoice: add ctlv option. * pay: Fix a crash when waitblockheight times out * pytest: Reproduce waitblockheight timeout issue #4309 * json-rpc: Give `waitblockheight` an error code for timeouts * Revert "wallet: Remove in-memory utxoset filter" * common/json: make json_scan return an error string. * lightningd/bitcoind: use json_scan. * chaintopology: fix crash if bcli doesn't include errmsg. * common: remove now-unused json_delve. * libplugin: replace rpc_delve with rpc_scan. * plugins: use json_scan. * plugins/bcli: convert to json_scan. * common: enhance json_scan with simple array helpers. * common/json: new helper to scan JSON: json_scan. * common: add length-limited versions of json_streq and json_get_member. * json_to_psbt: fix API to match other allocating json_to functions. * pytest: Test LightningRpc and plugin command notification support * pyln: Add notifications support to LightningRpc * pyln: Add command notification support for plugins * pytest: Add a test for `listinvoices` * docs: Document `bolt11` and `payment_hash` arguments of listinvoices * jsonrpc: Allow filtering via bolt11 or payment_hash in listinvoices * channeld: stop taken() leak. * df-mfc: remove guard-rails for failures * mfc: avoid off-by-one rounding errors when finding change fee * mfc-df: if the psbt we've got has inputs from our peer, remove them * mfc-df: if there's an error with a openchannel_signed, fail * tests/fuzz: add a fuzz target for hsm_encryption * libfuzz: add a NULL-termination in to_string * hsmd: cleanup encrypted hsm_secret detection * hsmd: regroup hsm_secret decryption logic * hsmd: group hsm_secret encryption * lightningd: regroup hsm_secret password input logic * lightningd: group hsm_secret encryption key derivation * lightning: confirm password on hsm_secret encryption * hsmtool: use errx() instead of err() everywhere * pyln-testing: suppress mypy on FileLock * df-plugin: add `signonly` to signing in df_accepter * dual_open: neaten spacing for macro * df: update an error message to give more info * json: allow helper to take/free psbt * pyln-client: to_whole_satoshi returns the rounded up satoshi value * pyln-client: add missing `min_witness_weight` to fundpsbt/utxopsbt * hsmtool: only write 32 bytes to hsm_secret on generatehsm * pytest: reproduce issue #4302 * hsmtool: reuse read_stdin_pass for `generatehsm` passphrase * hsmtool: input encryption password from stdin * pytest: Really only check DB statements if the env-var is "1" * pytest: Disable test_funding_cancel_race on slow machines * pyln: Do not rstrip() the return value of .append() * pytest: Stabilize test_routing_gossip * pyln: Check that the stderr line is not none in pyln-testing tail * pytest: Add throttler to limit load on the test system * pyln: Add logging to bitcoind RPC calls * pyln: Log mempool when we generate blocks in BitcoinD fixture * fix memset for route_hop json parsing when no channel_id exists. * bcli: print error message in debug log for broadcast failures * bcli: only print error if exitcode is non-zero * bcli: allocate temp string off of short-lived cmd * bcli: print sendrawtx every time (even on failures) * plugins/offer and plugins/fetchinvoice: send and recv errors. * fetchinvoice: return the next period for recurring offers. * invoice: hack in merkle of invoice as "payment_secret" (EXPERIMENTAL_FEATURES) * pytest: finally test fetchinvoice send, and offer plugin receive. * plugins/fetchinvoice: catch invoice_request reply. * plugins/fetchinvoice: plugin to send an invoice_request for a given offer * lightningd: low-level createinvoicerequest API (EXPERIMENTAL_FEATURES) * hsmd: code to sign bolt12 messages with a tweaked key. * plugins/offers: add code to catch invoice_request onion messages. * JSON: offers plugin `offer` command. * plugin/offers: skeleton for offers plugin (EXPERIMENTAL_FEATURES) * common/bolt12: add calculations for offer periods. * wire/Makefile: make generated wire files depend on Makefile. * Added listfunds test * Added spent option to listfunds in pyln-client * Added spent option to listfunds * setchannelfee: fix crash when channel is not in valid state. * add a note to "unknown parameter" error about -k autodetect failure. * pyln: add RpcException for finer method failure control. * doc/BACKUP.md: Minor typo due to rearranged text. * common: change default network to mainnet for all * doc/BACKUP.md: Document backup strategies for `lightningd`. * cleanups: feedback from Christian Decker review. * common/blindedpath: fix type of enctlv. * plugins/pay: allow bolt12 invoices (EXPERIMENTAL_FEATURES). * pay, wallet: rename internal `bolt11` vars to `invstring`. * invoice: handle bolt12 strings if EXPERIMENTAL_FEATURES. * lightningd: createinvoice low-level invoice creation command. * bolt11: expose routine to parse bolt11 string except signature. * lightningd: add low-level offer interfaces. * sendpay: optional argument to link local offer. * invoice: add an optional local_offer_id. * db: offer table. * common/iso4217: make find_iso4217 a little more usable. * Plugin: support extra args to "start". * plugins: support concatenation of multiple args. * plugins: remove deprecated string plugin options. * common/private_channel_announcement: don't include wire/peer_wiregen directly * pytest: check millisatoshi by float raises * pytest: test mul and div units for Millisatoshi * pyln: fix Millisatoshi div with msat itself * pytest: Debug test_lockup_drain * tests: Move ecdh stub out of autogenerated stubs * tests: Update mocks after breaking them via interleaved merges * common/sphinx: remove assertion that onions we parse must be 1366 bytes. * common/sphinx: make onionpacket.routinginfo a dynamic member. * common/sphinx: make TOTAL_PACKET_SIZE a macro. * common/sphinx: eliminate temporary buffer for prefill. * common/sphinx: don't use temporary to xor in cipher stream. * common/sphinx: don't make copy to compute packet hmac. * pytest: Add test for keysend feature support * keysend: Check that the destination supports keysend upon init * make: have `git submodule update` be recursive * pyln: Add TOR and SOCKS5 support in pyln.proto.wire.connect * wallet: remove debugging fprintf. * hsmd: code to sign a bolt12 merkle root. * common/blindedpath: EXPERIMENTAL creation of blinded paths. * devtools/bolt12-cli: cheap ripoff of bolt11-cli tool. * common/iso4217: define the currency codes. * common/bolt12: encode/decode for bolt12 offer, invoice_request and invoice * common/bech32_util: simple routines for checksumless bech32 charset usage. * common/bolt12_merkle.h: experimental bolt 12 implementation. * channeld: hand up onionmessage fields. * sendonionmessage: add invoice, invoice_request and invoice_error fields. * wire.c: import bolt 12 offers CSV. * utf8: handle UTF-8 arrays. * gossmap: helper to try to map x-only nodeid into 33-byte pubkey id. * bitcoin/pubkey: add pubkey32 primitive for xonly pubkey types. * common/json_tok: minor cleanup to bitcoin_outpoint array parsing. * bitcoin/tx: fix type of outpoint (n is a u32), simplify json_to_outpoint * cli: Add a JSON parsing sanity check before sending request * jsonrpc: Forward errors on malformed requests to cli * pytest: Add a test to reproduce #4258 * txprepare: Verify that outputs arg is an array * cleanup: Remove stdio header from param.c * txprepare: Use param_outpoint_arr helper to validate input * json: Add two param functions to parse string arrs and outpoint arrs * pyln: Remove deprecated txprepare variant * plyn: use math.floor for msat mul and div * pyln: fix msat float multiplication * pyln: extend msat floating testcases * channeld: Fix the shutdown_sent billboard direction * pytest: Disable flaky test_fundchannel_start_alternate * connectd: add annotation and fix up formatting on connectd/netaddress.c * connectd: fix valgrind complaint on FreeBSD * doc: fees on listpeers * pytest: tests fee_base and fee_ppm on listpeers * feat: adds local halfchan fees to listpeers * onchaind: cap max_possible_feerate using commitment transaction. * onchaind: speed up HTLC matching by skipping identical CLTVs. * bitcoin_tx_check: don't rely on tmpctx. * onchaind: test stressful input. * common/daemon: remove notifiers on root at cleanup. * wire/tlvstream: add tlv_make_fields helper to populate ->fields array. * tools/generate-wire.py: use helpers. * tools/generate-wire: don't use void * pointers for tlv fromwire. * plugins/libplugin-pay: expose gossmap for plugin special effects. * lightningd: split onion_message hook. * channeld: hand input blinding to lightningd. * libplugin-pay: don't expose bolt11 details. * pyln: Replace undecodeable symbols when tailing logs * pytest: Use datetime.strptime instead of datetime.fromisoformat * pyln: Set the prefix in TailableProc * plugins: add txprepare to .gitignore * peer_htlcs.c: close taken() leak. * pytest: recreate wallet on bitcoind restart. * channeld: fix gcc warning on gcc 8.3.0 (Debian 8.3.0-6) * invoice: fix potential race where invoice is paid/expired while we're calling hook. * lightningd/log.c: Fix up handling of SIGHUP. * tests/test_misc.py: Add a test for issue #4240. * df-mfc: set the minimum input weight to 110 iif v2 * df-mfc: set protocol type when we know it * fundpsbt/utxopsbt: new `min_witness_weight` param * bitcoin: break out method to calculate input weight * df-tests: use multifundchannel pathways instead of custom plugin * df: rename the dual_open wires to match other daemon wires * invoice: make invoice_payment hook a multi-user hook. * unittest: use common_setup / common_shutdown almost everywhere. * common: check for outstanding taken() pointers in common_shutdown. * ccan: update ccan/utf-8 to reject NULs embedded in strings. * common/json: json_add_stringn helper. * libplugin: add command_hook_success helper. * json: add json_tok_endswith and json_tok_startswith helpers. * gossipd: fix memleak. * Makefile: separate check-python targets. * JSON-RPC: don't allow any strings which aren't valid UTF-8. * common/bolt11: reject bad UTF-8 strings. * common/utils: simple utf8 helpers. * pyln: failing test msat from float str * pyln: fix msat from float str * pytest: use default fundamount on two tests * pyln: def amount 10**6 for fund- and openchannel * external/Makefile: fix parallel build with missing secp headers. * lightningd/plugin_hook.c: Make `db_write` a chained hook. * lightningd/plugin.c: Make plugin-exclusive loop support multiple plugins. * wally: remove wally casts that are no longer needed * tx: Remove wally workaround that is no longer needed * libwally: update to latest which uses libsecp-zkp submodule * df-spec: fail the open if they send us too many tx-collab msgs * mfc-psbt: mark all of our inputs as "ours", then only sign those * df: pass back 'close_to' for completed/commitment secured channels * mfc-df: add 'happy path' tests for the v1+v2 things * df-accepter plugin: temporarily dont pass in signpsbt * mfc-df: after openchannel_signed is finished, we call finished * mfc-df: after psbt signed, send to openchannel_signed if v2s * mfc-df: after sigs are collected, go sign the psbt * mfc-df: once we've gotten the PSBT finalized, we wait for peer sigs * mfc-df: only add outputs for v1 outs; go to openchannel_update if v2s * mfc-df: rework how openchannel_update works, callbacks * mfc-df: merge openchannel_init/fundchannel_start results * mfc-df: have both paths use redo_multifundchannel * mfc: consolidate to a single FAILED state * mfc-df: track destination's openchannel version * mfc: add a 'fail_destination' helper * mfc: add happy path-way for v2 in multifundchannel * nit,df test plugin: change up how feerate is formatted * doc/TOR.md: Add missing instructions to add user to Tor group. * htlc.h: move NUM_SIDES to define, not enum member * lightningd: remove unused pid_fd member in struct lightningd * doc/PLUGINS.md: Fix typo. ------------------------------------------------------------------- Sat Nov 21 10:29:52 UTC 2020 - marec@detebe.org - Update to version 0.9.2: * Update to v0.9.2 * release: bump to v0.9.2rc2 * Revert accidental experimental libwally update from f56266c1c1e135a29124448b822b511ce7f3120b * plugins: undeprecate old form of hooks. * pay: Cleanup the route applicability checks for channel hints * pay: Retry the route computation if we could not apply the chanhints * pay: Add a pre-apply check to channel_hint updates * pay: Move the chanhint update up to payment_getroute * connectd: properly cleanup 'competing' outgoing connections * tests: make connection details correct * connectd: Update connection list with new address * pytest: use the 'contrib' plugin set, not the 'test' set * chore: reduce CI load merging a testcase * pytest: fix a probable flake on test_channel_opener * pytest: tests closer on unilateral close * fix: closer set to remote for onchain settlement * fix: hangup in plugin rescan * v0.9.2rc1 - Changelog * htlc_accepted: log BROKEN if both 'failure_msg' + 'failure_onion' * refactor htlc_accepted_hook_deserialize. * failure_onion: a test that only ensures lightningd doesn't crash. * optional "failure_onion" in reply to htlc_accepted hook. * pytest: silence pytest warnings about unregistered 'slow_test' marker * devtools/decodemsg: handle truncated TLVs properly. * gossipd: fix response for large replies. * doc: document output descriptors in hsmtool * pytest: test onchain wallet descriptor import on bitcoind * pytest: create a wallet for bitcoind if none is present * hsmtool: add a 'testnet' option to 'dumponchaindescriptors' * tools: add a new 'dumponchaindescriptors' command to hsmtool * fuzz: add a fuzz target for common/descriptor_checksum * fuzz: add a 'to_string' function to libfuzz * common: add a descriptor checksum computation module * pyln-testing: require bitcoin v0.20 for PSBT handling, and create wallet for v0.21 * `hsm_secret` generation from a seed-phrase * gossipd: minor cleanups. * gossipd: simplify large reply code. * gossipd: new struct to hold scids and timestamps together. * gossipd: use straight counter instead of bitmap for query_channel_range replies. * pytest: make test_hook_dep tests more stable. * pytest: simplify test_openchannel_hook_chaining now order is deterministic. * plugin: sort topological candidates by specified order. * pytest: test that we maintain load order unless hook deps require a change. * plugins: check order once all plugins have returned from getmanifest. * pytest: test that we don't complain about missing dependencies at startup. * plugins: deprecate old form of hooks. * libplugin: add support for before and after deps on hooks. * pyln: add support for dependent hooks. * lightningd: actually order the hooks. * plugins: allow 'before' and 'after' arrays for hooks. * common: allow tal_arr_expand() to have an `n` argument. * lightningd/opening_control.c: Remove predeclaration. * openingd/: Fail `fundchannel_start` if we already are, or will become, the fundee. * tests/test_connection.py: Test for #4108. * doc: update channel_state_changed notification * pytest: state_change history * feat: adds state_changes to listpeers output * db: adds a state_change history to database * feat: adds timestamp to state_change notification * rpc: adds json_add_timeiso helper * pytest: test opener and closer of listpeers * rpc: adds opener and closer to listpeers channels * db: persist channel closer and state change cause * pytest: channel state change cause and message * feat: adds state change cause and message * lightningd: fix similar race in `stop`. * lightningd: fix crash if we abort after enabling notifications. * doc: move "next_onion" and "shared_secret" to inside "onion". * dual_open: correct silent enum conversions * df: move tx_role functions (used in next patch) * bitcoin/signature: wrap libwally call. * chore: fixes a cppcheck false positive * wire: remove declaration of towire_tlvs since its implementation is already removed * travis: Add a pytest run with EXPERIMENTAL_FEATURES=1 * pyln: Bump cryptography dependency to 3.2 due to upstream bug * wallet: Remove orphan statements.po file * pyln-proto: Add compactsize alias for varint_{encode,decode} * nit: lightningd.c: fix "Richard Stevens' advice" * doc: STYLE.md fix "parameters get" * lightningd: note whether payment secret is used or not. * gossmap: fix reutrn of gossmap_xxx_has_feature, rename. * df: rename 'feerate_per_kw' to 'feerate_per_kw_commitment' * df: Pass new feerate options through to plugin, set reasonable bounds * df: incorporate a few spec changes -- serial_id is now 64-bits * nit: lightningd.c: Fix a couple of SATTS comments * dev-force-features: adds a second valid format for forcing features * features: add 'feature_bit_sub', which will subtract/unset bits * features: have clear_feature_bit correctly resize bitfield * features: reorg, going to use elsewhere soon * Makefile: fix bashism in NO_PYTHON * Makefile: fix typo and wrong path in external/Makefile to properly remove objects with "make clean" * Makefile: add status_levels.c to $(COMMON_SRC_NOGEN) * Makefile: add $(LIGHTNINGD_SRC_NOHDR) to $(ALL_C_SOURCES) for "make clean" to work properly * chore: optimize postgres binary lookup * dualopend: handle dev-memleaks; fix placement of return on DEV options * dualopend: re-order 'handle_dev_memleak' occurrence * df: finalize redeemscript at the same time as witness stack * script: break out redeemscript->scriptsig function * doc: update HACKING.md * doc: typo in lightning-setchannelfee.7 * close: add notification for slow closes. * lightningd: infrastructure for internal notifications. * pytest: add notifications to tests. * lightning-cli: print notifications (with '# ' prefix) if received. * pyln: handle (ignore) notifications, and add notify_msg to send them. * libplugin: ignore incoming notifications. * libplugin: support for sending notifications. * lightningd: forward notifications from plugins if enabled. * JSON-RPC: notifications command. * common/json_stream: add generic double-cr helper. * ccan: update to latest version, get json_out_finished update. * common: add routines for log level names. * lightningd: clean up close code now force is always true. * pyln: Adds type annotations to zbase32 functions * pyln: Moves old zbase32 test from primitives * pyln: Adds tests to zbase32 * pyln: Adds one-liner docs to zbase32 functions * pyln: Improve zbase32 encoding / decoding * Makefile: fix dependencies for check-pyln-% * Makefile: make check-includes check all the non-generated files. * Makefile: unify generated files definition. * doc: add a document about our fuzz testing integration * fuzz: add a runner Python script for fuzz targets * configure: allow to run with UB sanitizer * fuzz: add a fuzz target for initial_channel * contrib: remove ASAN suppressions * fuzz: add a fuzz target for the close tx * fuzz: add a fuzz target for channel_id derivation * fuzz: add a fuzz target for common/bip32 * fuzz: add a fuzz target for common/bigsize * fuzz: add a target for base32/64 encoding * common: remove b64_decode * fuzz: add a target for common/bech32 * amount: correctly parse amount strings we generate * fuzz: add a fuzz target for amounts parsing * build: introduce a fuzzing mode * mfc: blackbox test for commitment vs funding tx feerates * tx,bugfix: correct signature length estimate * mfc: use declared constants * mfc: Add a `commitment_feerate` optional parameter to multifundchannel * opening: use correct dust_limit for reserve floor * df, channeld: cleanup how psbt signalling works * df, bugfix: wait til after we've saved the channel to do this * dual-open,openchannel_update: include the index of the funding output * psbt: save the index of the change on the 'parent' * df: add notification for receiving peer's funding tx sigs * dual-fund: rework where we send our tx-sigs message, allow peers in * channel.psbt: make non-const * channeld-df: only send our sigs if we've got them * channeld-df: actually check serial_id of input when setting sigs * common: cleanups suggested by Christian Decker's review. * plugins/libplugin-pay: use gossmap. * libplugin-pay: incorporate gossip store. * plugins/pay: fix leak on failed new payments. * common/gossmap: digest private channel information too. * gossip_store: make private channels more similar to channel_announcement * pytest: test that route can see private channels. * gossmap: implement feature tests * common/gossmap: fix gossmap_node_get_announce() on unannounced nodes. * gossip_store: don't copy old delete markers on startup compact. * route: return NULL if destination is unreachable. * dijkstra: fix heap ordering. * df, bugfix: use tal_wally around an allocation * df, nit: make this error message a little bit more informative * df, bugfix: set the reserve correctly on the channel * df, bugfix: dont free the `uc` yet * df: Pass the serial_id of the funding output to openchannel_init caller * df: simplify `check_balances`, add spec quotes * dualopend, nit: move some lines to be within 80chars * dualfund: rearrange things so that the wire-dependent calls are separate * nit: move changeset_get_next to inside EXPERIMENTAL_FEATURES flag * feerate: remove duplicate method * psbt: hoist up `psbt_add_serials`, so we can use it elsewhere * psbt-finalized: hoist method to common * json nit: use const for json_add_psbt * json: add channel_id helper * psbt: have the unknown map 'add' be a 'set' instead * elementsd: disable dual-funding test; don't add non-witness utxo * df-open: use channel_id for openchannel_update and openchannel_signed * dualopen: use separate wire for passing updated PSBTs back to dualopend * dualopend: reverse polarity of send_next * dual-fund: test (turned off) and plugin for executing an openchannel * psbt: new methods for generating serial_ids for an input/output * channel: save funding_psbt to database * df-open: send the tx-sigs from channeld, check that we send it at the * df-channeld: broadcast funding tx, once we get tx_sigs from peer * peer_control: move open_command up to where channeld can get it, * dual-open: use tx_roles, not side, as switch * psbt: method for extracting witness stacks * df-open: preliminary handling for tx_sigs message * df-open: commands to update a PSBT or submit a signed PSBT * df-open: add a 'open_commands' list to stash pending opens around in * peer_channeld: pass over PSBT, remove second message * df-open: pathway for getting a commit back from peer * dualfund: Pass in expected remote's serial parity * dualfund: opener, openchannel_init command (1/3) * tx_roles: pull up roles, rename * dualopend: use status_failed, not peer_failed * psbt: add the full transaction for the utxo in a psbt * dual-fund: remove flag for option_anchor_outputs * dual-fund: check the max feerate also, not just min feerate * psbt: add helper to set the non-witness utxo for an input * psbt_open: method for adding a witness_stack to a psbt input * wires: towire/fromwire for wally_tx * db: add helper for db_col_psbt * common: pull up `param_psbt` * removed unwanted lines and added punctuation marks * chore: adds env NO_PYTHON check to Makefiles * Add OpenBSD instructions to INSTALL.md (#4136) * test: simplify test for close_to * tests,fundchannel: return the result from the fundchannel rpc call * test, fundchannel: pass kwargs down to fundchannel command * fundchannel: take a 'close_to' address * mfc: allow a 'close_to' address to be passed in * openingd: patch test_opening_tiny_channel under EXP_FEAT * doc: reorder --mainnet before --testnet * doc: Add missing signet to --help and man * Update configure * channeld: order htlcs by id before retransmission. * pytest: add test for HTLC ordering. * channeld: log broken message if we receive HTLCs out of order. * lightningd: fix crash when we try to send fail_htlc msg to onchaind. * pyln: utils:BitcoinD: add -wallet="test" option * options: remove unused 'commit-fee-min/max' options. * plugins: use "slow" feerate for mutual close negotiation. * pyln: Plugin methods and hooks refuse to set results twice * pyln: Add logging handler that passes records to lightningd * pytest: fix overzealous removal of directories on failure. * travis: Specify pytest-rerunfailures==9.1 to avoid regression * travis: Consolidate pytest options to cli options * travis: Allow environment to specify $PYTEST_PAR * make: Include the $PYTEST_OPTS when running pytest * pyln: Remove any logging handlers at teardown to avoid logging error * pytest: Check for connection close and reset errors in timeout test * pytest: fix flaky test if one side hasn't processed close yet. * travis: Enable TEST_CHECK_DBSTMTS=1 mode for one of the configs * db: Fix statement expansion bugs found through dblog mode * pytest: Verification mode of expanded stmts for the db_write hook * fix: broken SQL statement in wallet db_set_utxo * pytest: fix flaky test. * pyln-client: listpayments -> listpays * pytest: fix experimental test. * pytest: Don't give up on the first psql connection error * pytest: Stabilize test_penalty_htlc_tx_timeout * pyln: Decode process output once before storing it * pytest: Stabilize test_setchannelfee_{restart,zero} * travis: Add blinker dependency * Fix pytest major version * pyln: Add pytest to type ignores * pytest: Make LightningNode.fund_channel more resilient * pytest: Cleanup test_gossip_ratelimit * pytest: Don't start 3 nodes in test_gossip_ratelimit * pyln: Add type-annotations to pyln-proto * make: Enable mypy checks on pyln-testing * pyln: Add mypy typing exceptions for external dependencies * make: Set $PYTHONPATH once * pyln: Use pyln-client Makefile for checks * pyln: Parametrize and unify Makefiles for pyln package * chore: reduce log buffer from 100MB to 10MB * travis: Re-enable the postgres configuration again * pytest: Reduce the db_provider fixture scope to function * pyln: Allow the base test directory to contain some files * pyln: Start each postgres DB in its own sub-directory * travis: Reduce reruns to 2 and early failure on test failure * pytest: Rename all fund_channel to fundchannel * plugins: generate list of plugins more atomically, respect V=1 and --quiet. * build: run update-mocks. * configure: allow to run clang with ASAN * pytest: Add test for failcode conversion * fix: openchannel_hook log BROKEN on dup close_to * pyln: Migrate from binascii.hexlify to bytes.hex * pyln: Add code to unwrap an encrypted onion at the intended node * pyln: Add a warning that pyln-proto is not safe for production use * pyln: Implement sphinx onion packet generation in python * nit: doc/lightningd-config.5: Remove redundancy * Update comment regarding signet using tb now * Updating signet chainparams for the latest PoW fix * bolt11: have caller supply preferred chain. * bitcoin/chainparams: signet uses the same P2PKH and P2SH version bytes as testnet. * utils: use a cleaner pattern to capture wally allocations. * bitcoin/: neaten use of libwally base58 and bas64 routines. * Makefile: work with shasum -a as well as sha256sum. * pytest: Add test for key-value style postgresql DSNs * db: Add support for key-value pair DSNs in postgresql * Remove varint typedef for bigsize * lightningd: don't hand around partially-assembled channel_info. * lightningd: separate fee_states out of struct channel_info. * dual_open: don't release uc here, it gets freed elsewhere * dual-fund: call daemon_shutdown only * dual_open: alignment nit * dual_open: memleak problem with fee_states * dual_open_control: nit, reword comment * dual_open: split logic for deserializing a changed vs signed PSBT * psbt_txid: it's possible a psbt may already have the finalized scriptsig * dualopend: tiny billboard capitalization change * dualopend: check that we're not exceeding max allowable capacity * dualopend: use status failed instead of peer failed * pyln: Rename LightningNode.fund_channel to fundchannel * pyln: Add type annotations to lightning.py * pyln: Add type-annotations to plugin.py * pyln: Fix two formatting string not matching parameters * pyln-client: Add support to monkey patch the JSONEncoder * pyln: Add stubs to generate documentation for pyln-client * Travis: disable postgres test. * topo: Do not keep txids in memory indefinitely * common/memleak: simplify and document API. * pytest: don't test for memleaks under valgrind. * common: don't suppress leak detection for libwally allocations. * memleak: make "_notleak" names less powerful. * common: enforce that we have collected all wally allocations. * bitcoin: use tal_gather_wally() so we don't leave unattached allocations. * common: add tal_gather_wally() function to reparent libwally objs. * bitcoin/psbt: psbt_input_add_unknown/psbt_output_add_unknown needs a tal ctx. * bitcoin/psbt: psbt_finalize needs a tal ctx. * bitcoin/psbt: wallt_tx_output needs a tal ctx. * bitcoin/psbt: psbt_txid needs a tal ctx. * bitcoin/tx: trivial cleanups. * bitcoin/psbt: attach destructors to wally allocations to avoid leaks. * bitcoin/psbt: more const pointers. * dual-openingd: msg_type should be dualopend_wire not u8 * db: Fix size mismatch on postgres in a migration * configure: clean up tmpfile if we fail. * configure: refuse to create config.vars without any database. * lightningd: really remove relative plugins dirs. * lightningd: remove deprecated result for `plugin stop`. * lightningd: remove deprecated rpc_command hook return. * lightningd: remove deprecated return for `db_write` hook. * lightningd: remove deprecated per_hop_v0 in htlc hook. * lightningd: remove deprecated plugin entries. * update SHA256STAMPs using sorted dependencies * Makefile: sort input filenames to SHA256STAMP * listpeers: show correct amounts for starting funding allocation * contrib: make bitcoin polling and gossip propagation snappier * doc: correct weight numbers for fundpsbt * memleak: 'generify' the memleak calling to code for openingd/dualopend * repro: Add missing Dockerfiles * repro: Allow dashes in the version number * docker: Bump OS version to debian buster ------------------------------------------------------------------- Sun Oct 11 09:32:52 UTC 2020 - marec@detebe.org - Update to version 0.9.1: * Ready for release! * common/amount: fix OpenBSD compile warning. * lightningd: don't declare local vars stdin and stdout. * pyln: Sets remote_pubkey for LightningConnection if node is not the initiator * CHANGELOG.md: 0.9.1rc2 * doc: note default proxy port. * pytest: test connection timeout. * connectd: implement connection timeout (60 seconds). * plugins/pay: handle case where amount_msat is 'null'. * Travis: spread the load a little more. ------------------------------------------------------------------- Sun Oct 11 09:25:07 UTC 2020 - marec <marec@detebe.org> - initial package build
Locations
Projects
Search
Status Monitor
Help
OpenBuildService.org
Documentation
API Documentation
Code of Conduct
Contact
Support
@OBShq
Terms
openSUSE Build Service is sponsored by
The Open Build Service is an
openSUSE project
.
Sign Up
Log In
Places
Places
All Projects
Status Monitor