Skip to content

Shrink FSST and OnPair codes when compression leaves large spare capacity - #11434

Open
shu-chen-dot-ca wants to merge 2 commits into
vortex-data:developfrom
shu-chen-dot-ca:fix-11420-fsst-codes-capacity
Open

shu-chen-dot-ca wants to merge 2 commits into
vortex-data:developfrom
shu-chen-dot-ca:fix-11420-fsst-codes-capacity

Conversation

@shu-chen-dot-ca

@shu-chen-dot-ca shu-chen-dot-ca commented Nov 12, 2026 •

Copy link
Copy Markdown

Summary

fsst_compress builds the codes buffer by appending one batch at a time, and BufferMut growth at least doubles the capacity. freeze keeps that allocation, so a compressed FSST array could hold up to about 1.9x its nbytes(), depending on where the last batch landed relative to the last doubling. OnPair has the same gap: onpair::compress_rows grows the codes Vec from empty, and Buffer::from(Vec) keeps its whole allocation.

Changes

  • FSST: at the end of FsstSink::finish, when the codes buffer's spare capacity is more than a quarter of its length, copy the codes into an exactly sized buffer (same alignment, the context's allocator) before freezing. Otherwise the buffer is frozen as before.
  • OnPair: in onpair_compress, call shrink_to_fit on the codes under the same condition, before they become a Buffer. The dictionary bytes were already copied into an exactly sized buffer.
  • Add a codes_buffer_has_little_spare_capacity rstest regression to each crate, over 8,192 random hex strings of several lengths. Each one unwraps the uniquely owned codes buffer with try_into_mut and checks that the spare capacity is at most a quarter of the length. Both fail on develop and pass with this change.

The issue's reproducer (live heap left behind by compression, divided by nbytes()), pointed at this branch:

Input FSST on develop FSST with this change OnPair on develop OnPair with this change
8,192 x 64 chars 1.01 1.01 1.84 1.00
8,192 x 80 chars 1.51 1.01 1.52 1.00
8,192 x 128 chars 1.90 1.01 1.00 1.00
8,192 x 160 chars 1.55 1.01 1.56 1.00
100,000 x 48 chars 1.57 1.00 1.63 1.00

Checks run:

  • cargo nextest run -p vortex-fsst -p vortex-onpair --all-features (158 passed, 1 skipped) and cargo test --doc -p vortex-fsst -p vortex-onpair --all-features
  • cargo clippy -p vortex-fsst -p vortex-onpair --all-targets --all-features -- -D warnings
  • cargo +nightly-2026-11-13 fmt -p vortex-fsst -p vortex-onpair -- --check

Benchmarks were not run locally; the extra copy or shrink happens at most once per compression call, on the compressed output.

AI assistance: this change was prepared with help from an AI coding agent, and reviewed and submitted by me.

@joseph-isaacs

Copy link
Copy Markdown
Contributor

do we need this for onpair?

can you rebase we have added new file size checks

@codspeed

codspeed Bot commented Nov 13, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 10.58%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 1 improved benchmark
✅ 1920 untouched benchmarks
⏩ 50 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Walltime scalar_subtract_neon 13.8 µs 12.5 µs +10.58%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing shu-chen-dot-ca:fix-11420-fsst-codes-capacity (9ef884b) with develop (d81bd4c)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

…city

Signed-off-by: Shu Chen <338845574+shu-chen-dot-ca@users.noreply.github.com>
Signed-off-by: Shu Chen <338845574+shu-chen-dot-ca@users.noreply.github.com>
@shu-chen-dot-ca

Copy link
Copy Markdown
Author

Rebased onto the latest develop (815e344), no conflicts.

Yes, OnPair has the same problem. onpair::compress_rows grows the codes Vec from empty, and Buffer::from(Vec) keeps its whole allocation. The same reproducer against onpair_compress gave held/nbytes of 1.84 / 1.52 / 1.56 for 8,192 strings of 64 / 80 / 160 characters, and 1.63 for 100,000 strings of 48 characters. I added a second commit that calls shrink_to_fit on the codes when more than a quarter of their capacity is spare, using the same threshold as FSST. It also adds the same kind of rstest regression in encodings/onpair/src/tests.rs, which fails on develop and passes with the fix. Held/nbytes is now 1.00 to 1.01 in all of those cases. The dictionary bytes are already copied into an exactly sized buffer, so the codes were the only oversized buffer.

Checks on the rebased branch: cargo nextest run -p vortex-fsst -p vortex-onpair --all-features (158 passed), doc tests, cargo clippy -p vortex-fsst -p vortex-onpair --all-targets --all-features -- -D warnings, and cargo +nightly-2026-11-13 fmt --check for both crates.

@shu-chen-dot-ca
shu-chen-dot-ca force-pushed the fix-11420-fsst-codes-capacity branch from 9ef884b to cdfcca4 Compare November 13, 2026 16:07
@shu-chen-dot-ca shu-chen-dot-ca changed the title Shrink the FSST codes buffer when compression leaves large spare capacity Shrink FSST and OnPair codes when compression leaves large spare capacity Nov 13, 2026

Copy link
Copy Markdown
Contributor

Thanks for tracking this down! Rather than copying into an exactly sized buffer, could you use the new BufferMut::shrink_to_fit from #11454 once it lands? It asks the allocator to shrink the block in place (a realloc), so with glibc the codes are not copied at all and the old and new allocations are never live together, and it keeps the buffer's alignment if the allocator moves the block.

The change in FsstSink::finish would become something like:

self.codes.shrink_to_fit();
let codes_bytes = self.codes.freeze();

shrink_to_fit already leaves a buffer with little slack unchanged, so the capacity - len > len / 4 check is optional. Your regression test should keep working as is.


Generated by Claude Code

@shu-chen-dot-ca

Copy link
Copy Markdown
Author

Makes sense, no reason to copy when realloc can shrink in place. Once #11454 lands I'll rebase onto it, switch FsstSink::finish to self.codes.shrink_to_fit() and drop the len / 4 check. The OnPair side already uses Vec::shrink_to_fit, which is a realloc as well, so I'll drop its threshold too to keep the two the same.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants