Skip to content

[core] Preserve append order during full compaction - #11189

Open
zhangyue19921010 wants to merge 1 commit into
apache:masterfrom
zhangyue19921010:contrib/order-full-append-compaction-20261116
Open

zhangyue19921010 wants to merge 1 commit into
apache:masterfrom
zhangyue19921010:contrib/order-full-append-compaction-20261116

Conversation

@zhangyue19921010

Copy link
Copy Markdown
Contributor

Purpose

Fixes #11188.

Full compaction copies candidates from a priority queue whose iterator does not preserve sequence order. After an automatic compaction requeues a small result, a bucket that previously reads [0, 1, 2, 3, 4] can be rewritten as [0, 1, 2, 4, 3] despite bucket-append-ordered = true.

Sort full-compaction candidates by minimum sequence number before filtering and rewriting them.

Tests

Added a Java table regression that writes five real files, commits automatic compaction, performs full compaction, and reads both states through an initial streaming scan. It fails on unchanged upstream f1ffd1369 with [0, 1, 2, 4, 3] and passes with the change. No file metadata is fabricated.

11 selected tests passed on JDK 11.0.18, covering full compaction, automatic candidate selection, ordered split/read behavior and repeated compaction cleanup. Maven verify ran the selected suite twice; one existing disabled candidate-selection test was skipped in each run. Checkstyle, Spotless, Enforcer and RAT passed without fast-build.

Prepared with TRAE CLI.

Co-authored-by: TRAE CLI <traecli@bytedance.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Full compaction changes the row order of a bucketed append table

1 participant