Repository navigation
perf: search sorted children through one shared probe - #10452
Merged
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Several reads and slices binary search a sorted child, such as patch indices or run ends. Some of them built a new searcher, and so a new probe, for every search. RunEnd's probe lookup didn't use the search-sorted API at all: it hand-rolled a binary search that read the ends one scalar at a time. This PR changes only those call sites. Every sorted search now goes through the search-sorted API with one shared probe. That holds even for a single search inside a scalar read: the search reuses the caller's probe instead of building its own.
Decoding the sorted child once per repeated probe is split out into #11473, which is stacked on this PR.
Instructions per iteration (valgrind/cachegrind) on the new
patched_searchbenchmark: 1M rows, one BitPacked patch or run end every 64 rows.developrun_end_read_once(256 reads)run_end_read_repeated(256 reads)run_end_slice(64 slices)patches_slice(64 slices)sparse_read_once(256 reads)sparse_read_repeated(256 reads)Wall-clock medians agree. For example,
run_end_read_repeatedgoes from 1.47 ms to 645 µs, andrun_end_slicefrom 868 µs to 395 µs.Changes
SearchSortedPrimitiveArray::with_probesearches through anArrayProbethe caller already holds, such as aProbeStatechild slot. A canonical primitive child still reads its buffer directly.with_probeinstead of the hand-rolled loop.find_physical_range.Patches::slice: finds both of its bounds throughsearch_indices, which shares the chunk-offsets probe and the indices searcher.Patches::probe_patched: for repeated reads of Sparse, BitPacked, ALP and ALP-RD, it searches the indices slot and reads the values slot through the probe. One-off reads keepget_patched, which narrows the search by chunk offsets.encodings/fastlanes/benches/patched_search.rs.API Changes
All additive:
SearchSortedPrimitiveArray::with_probePatches::probe_patchedRunEndArrayExt::find_physical_rangehttps://claude.ai/code/session_01Jck72aD351kMboqN5g8Ccs
🤖 Generated with Claude Code