Skip to content

perf: search sorted children through one shared probe - #10452

Merged
robert3005 merged 10 commits into
developfrom
ji/adoring-brown-55sw7y
Nov 15, 2026
Merged

robert3005 merged 10 commits into
developfrom
ji/adoring-brown-55sw7y

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Several reads and slices binary search a sorted child, such as patch indices or run ends. Some of them built a new searcher, and so a new probe, for every search. RunEnd's probe lookup didn't use the search-sorted API at all: it hand-rolled a binary search that read the ends one scalar at a time. This PR changes only those call sites. Every sorted search now goes through the search-sorted API with one shared probe. That holds even for a single search inside a scalar read: the search reuses the caller's probe instead of building its own.

Decoding the sorted child once per repeated probe is split out into #11473, which is stacked on this PR.

Instructions per iteration (valgrind/cachegrind) on the new patched_search benchmark: 1M rows, one BitPacked patch or run end every 64 rows.

Benchmark develop This PR Ratio
run_end_read_once (256 reads) 12.16M 5.60M 0.46x
run_end_read_repeated (256 reads) 12.24M 5.46M 0.45x
run_end_slice (64 slices) 7.05M 3.24M 0.46x
patches_slice (64 slices) 2.02M 1.99M 0.99x
sparse_read_once (256 reads) 3.41M 3.44M 1.01x
sparse_read_repeated (256 reads) 3.42M 3.49M 1.02x

Wall-clock medians agree. For example, run_end_read_repeated goes from 1.47 ms to 645 µs, and run_end_slice from 868 µs to 395 µs.

Changes

  • New constructor: SearchSortedPrimitiveArray::with_probe searches through an ArrayProbe the caller already holds, such as a ProbeState child slot. A canonical primitive child still reads its buffer directly.
  • RunEnd lookups: a RunEnd probe lookup now searches its ends slot through with_probe instead of the hand-rolled loop.
  • RunEnd slicing: finds both bounds with one searcher, through find_physical_range.
  • Patches::slice: finds both of its bounds through search_indices, which shares the chunk-offsets probe and the indices searcher.
  • Patches::probe_patched: for repeated reads of Sparse, BitPacked, ALP and ALP-RD, it searches the indices slot and reads the values slot through the probe. One-off reads keep get_patched, which narrows the search by chunk offsets.
  • Error handling: an out-of-range patch range now returns an error instead of panicking.
  • Benchmark: adds encodings/fastlanes/benches/patched_search.rs.

API Changes

All additive:

  • SearchSortedPrimitiveArray::with_probe
  • Patches::probe_patched
  • RunEndArrayExt::find_physical_range

https://claude.ai/code/session_01Jck72aD351kMboqN5g8Ccs

🤖 Generated with Claude Code

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants