Skip to content

Core: Reject new equality deletes in v4 tables - #18993

Open
fmorillo7694 wants to merge 3 commits into
apache:mainfrom
fmorillo7694:core-v4-reject-equality-deletes
Open

fmorillo7694 wants to merge 3 commits into
apache:mainfrom
fmorillo7694:core-v4-reject-equality-deletes

Conversation

@fmorillo7694

@fmorillo7694 fmorillo7694 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

The v4 spec forbids adding equality deletes (#17783, vote), but MergingSnapshotProducer still accepts them on v4. They are now rejected, including deletes staged on v3 and committed after an upgrade. Existing equality deletes stay readable and removable. Tests that write equality deletes skip their v4 runs, dropping that coverage; reviewers may prefer DV fixtures.

Test plan: four new TestRowDelta cases. The iceberg-core and iceberg-data suites pass. Spark 3.5 to 4.2 and Flink 1.20 to 2.3 tests that add equality deletes now skip v4; locally, the affected Spark 4.2 and Flink classes pass.


AI Disclosure

  • Model: Claude (Anthropic)
  • Platform/Tool: Kiro (Kiro Crew)
  • Human Oversight: partially reviewed
  • Prompt Summary: Reject new equality delete files on v4 tables in core, keeping existing ones readable and removable. The validation change and its tests are generated.

The v4 spec forbids adding equality delete files. Snapshot producers now
reject them when the table is v4, including deletes staged against an
earlier version and committed after an upgrade. Equality deletes already
in a table upgraded to v4 are still applied and can still be removed.

Tests that add equality deletes now skip their v4 runs.

Generated-by: Kiro (Kiro Crew)
Core now rejects new equality deletes in v4 tables, so the delta writer
and committer tests that commit equality deletes no longer apply to v4.

Generated-by: Kiro (Kiro Crew)
Core rejects new equality deletes in v4 tables, so the shared delete read
tests, the Spark reader and action tests, and the Flink upsert, delta
writer and committer tests that commit equality deletes now run on v2
and v3 only.

Generated-by: Kiro (Kiro Crew)
@github-actions github-actions Bot added the spark label Nov 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant