Repository navigation
Core: Reject new equality deletes in v4 tables - #18993
Open
fmorillo7694 wants to merge 3 commits into
Open
fmorillo7694 wants to merge 3 commits into
fmorillo7694 wants to merge 3 commits into
Conversation
The v4 spec forbids adding equality delete files. Snapshot producers now reject them when the table is v4, including deletes staged against an earlier version and committed after an upgrade. Equality deletes already in a table upgraded to v4 are still applied and can still be removed. Tests that add equality deletes now skip their v4 runs. Generated-by: Kiro (Kiro Crew)
Core now rejects new equality deletes in v4 tables, so the delta writer and committer tests that commit equality deletes no longer apply to v4. Generated-by: Kiro (Kiro Crew)
Core rejects new equality deletes in v4 tables, so the shared delete read tests, the Spark reader and action tests, and the Flink upsert, delta writer and committer tests that commit equality deletes now run on v2 and v3 only. Generated-by: Kiro (Kiro Crew)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The v4 spec forbids adding equality deletes (#17783, vote), but
MergingSnapshotProducerstill accepts them on v4. They are now rejected, including deletes staged on v3 and committed after an upgrade. Existing equality deletes stay readable and removable. Tests that write equality deletes skip their v4 runs, dropping that coverage; reviewers may prefer DV fixtures.Test plan: four new
TestRowDeltacases. Theiceberg-coreandiceberg-datasuites pass. Spark 3.5 to 4.2 and Flink 1.20 to 2.3 tests that add equality deletes now skip v4; locally, the affected Spark 4.2 and Flink classes pass.AI Disclosure