Skip to content

Swap NetworkGraph PublicKeys for [u8; 33] #960

Closed

Description

@TheBlueMatt

Deserializing the network graph takes quite a while, of which my initial testing shows 80+% of the time is spent deserializing the publickeys from compressed 33-byte form into secp 64-byte form (and presumably verifying they are on the curve). There is relatively little reason to be verifying all the public keys, let alone using them in 64 byte format everywhere in the network graph in memory. We should instead simply use [u8; 33] for them, checking their correctness only after we've built a route with them and need to construct a Route object. Note that I'd think this may also represent a material speedup for route finding as pubkey comparison can be inlined instead of needing to call out to C code due to lack of cross-language LTO in most builds.

Activity

  1. dunxen commented on Aug 8, 2021

    @dunxen
    Contributor

    I'm going to take a look at this one. Will reach out for help / guidance.

  2. dunxen commented on Aug 9, 2021

    @dunxen
    Contributor

    I've migrated the NetworkGraph side of things mostly. Just busy with the router stuff. Should have a PR up within the next couple days if I don't get too caught up in work.

  3. TheBlueMatt commented on Aug 23, 2021

    @TheBlueMatt
    CollaboratorAuthor

    Glad to hear someone is working on this! If its not too late, it may make sense to base this work off of #1043, as otherwise it may cause some disruptive conflicts.

  4. dunxen commented on Aug 23, 2021

    @dunxen
    Contributor

    @TheBlueMatt thanks, it's definitely not too late. Had to pause for a bit because of work but getting back to it today 😄

  5. Kixunil commented on Aug 23, 2021

    @Kixunil
    Contributor

    Randomly going by, perhaps struct NodeId([u8; 33]);?

  6. dunxen commented on Oct 3, 2021

    @dunxen
    Contributor

    I have this basically complete on my fork. Just making some tweaks and cleaning up before I open the PR.

  7. TheBlueMatt commented on Oct 8, 2021

    @TheBlueMatt
    CollaboratorAuthor

    Results of #1107 are, roughly (from Github CI):

    test ln::channelmanager::bench::bench_sends               ... bench:   8,679,569 ns/iter (+/- 1,160,867)
    test routing::network_graph::benches::read_network_graph  ... bench: 2,495,846,755 ns/iter (+/- 145,851,981)
    test routing::network_graph::benches::write_network_graph ... bench: 141,748,993 ns/iter (+/- 6,248,343)
    test routing::router::benches::generate_mpp_routes        ... bench:  85,042,605 ns/iter (+/- 55,020,220)
    test routing::router::benches::generate_routes            ... bench:  81,382,609 ns/iter (+/- 58,966,139)
    test bench::bench_sends ... bench: 181,266,525 ns/iter (+/- 153,994,734)
    

    to

    test ln::channelmanager::bench::bench_sends               ... bench:   8,300,767 ns/iter (+/- 1,520,325)
    test routing::network_graph::benches::read_network_graph  ... bench: 1,501,082,125 ns/iter (+/- 79,991,352)
    test routing::network_graph::benches::write_network_graph ... bench: 132,349,099 ns/iter (+/- 8,865,711)
    test routing::router::benches::generate_mpp_routes        ... bench:  57,506,312 ns/iter (+/- 38,795,099)
    test routing::router::benches::generate_routes            ... bench:  58,337,687 ns/iter (+/- 44,755,523)
    test bench::bench_sends ... bench: 200,507,491 ns/iter (+/- 158,389,150)
    

    In other words, a slight regression in sending in a loop (maybe, it may just be actions being weird, most of that time is in I/O so I dunno why it would slow down), but otherwise a pretty nice win for routing and a huge win deserializing the network graph itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions