Skip to content

API: Prevent intermediate overflow in integer truncation - #18112

Open
laserninja wants to merge 3 commits into
apache:mainfrom
laserninja:fix/integer-truncate-overflow
Open

laserninja wants to merge 3 commits into
apache:mainfrom
laserninja:fix/integer-truncate-overflow

Conversation

@laserninja

@laserninja laserninja commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #18111.

Truncating Integer.MAX_VALUE - 1 with width Integer.MAX_VALUE currently produces a negative partition key. A scan filtered by id >= 0 then omits the matching file.

Use Math.floorMod in the int and narrow-integer helpers to avoid overflowing the intermediate remainder-plus-width addition. This preserves public APIs and existing behavior when the final mathematical result is outside the target type's range.

Validation on JDK 17:

  • Full :iceberg-api:test suite and targeted core truncation and table scan regressions passed. The new regressions failed before the fix.
  • :iceberg-api:spotlessCheck :iceberg-core:spotlessCheck passed.
  • :iceberg-api:revapi :iceberg-core:revapi passed.

Spark SQL regression (Spark 4.1): create an integer table partitioned by truncate(2147483647, id), insert 2147483646, and query WHERE id >= 0. On the original implementation the query returns no rows in Parquet, Avro, and ORC; with the fix it returns the inserted row in all three formats. The new Spark test, formatting, and Spark test checkstyle pass locally on JDK 17.

Additional local validation at 28aff7bfb: the full CI all-module build command (./gradlew -DallModules build -x test -x javadoc -x integrationTest) passed on JDK 17, including formatting and style checks across the engine versions. Apache RAT (dev/check-license), ./gradlew checkAllRuntimeDeps -q -DallModules=true, and ./gradlew -Pquick=true javadoc also passed. This build excludes tests; the Spark regression results above are from a separate test run.

Review follow-up: the same Spark SQL regression now covers Spark 3.5 (Scala 2.12), 4.0, and 4.2 (Scala 2.13), in addition to 4.1. All nine added version/format cases passed locally on JDK 17, together with spotlessCheck and checkstyleTest for the three changed Spark modules.


AI Disclosure

  • Model: GPT-6
  • Platform/Tool: Codex
  • Human Oversight: reviewed
  • Prompt Summary: Reproduce and fix integer truncation overflow, add regression tests, and create an issue and PR.

Use a nonnegative remainder without overflowing the width addition so matching partitions are not incorrectly pruned.

Fixes apache#18111

Generated-by: Codex
Verify that SQL inserts remain visible to range filters across Parquet, Avro, and ORC.

Generated-by: Codex

@Fokko Fokko left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a bit of an edge case, but I believe it is a bug. Thanks @laserninja for fixing this. Can you follow up on my comment?

}

@TestTemplate
void scanWithLargeIntegerTruncateWidth() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add this one for the other Spark versions as well?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the regression test for Spark 3.5, 4.0, and 4.2. All nine added cases passed across Parquet, Avro, etc. Updated validation notes too.

Add the scan regression to Spark 3.5, 4.0, and 4.2 so each supported engine version verifies matching rows survive partition pruning.

Generated-by: Codex
@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Oct 17, 2026
@github-actions

Copy link
Copy Markdown

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@wgtmac wgtmac left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a similar fix at apache/iceberg-cpp#1042 but I think they are still partial fixes and need more clarification on the spec side.

WDYT? @Fokko @nastra


public static byte truncateByte(int width, byte value) {
return (byte) (value - (((value % width) + width) % width));
return (byte) (value - Math.floorMod(value, width));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. The remainder, `v % W`, must be positive. For languages where `%` can produce negative values, the correct truncate function is: `v - (((v % W) + W) % W)`

This fixes the implementation, but the spec above still doesn’t say the arithmetic must use unbounded integers. Could we clarify the spec as r = floorMod(v, W) and v - r computed with unbounded arithmetic? Otherwise other implementations can still produce different partition values.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another issue is that truncateInt(10, Integer.MIN_VALUE) still returns Integer.MAX_VALUE - 1 after this PR: the final mathematical result is outside int, so wrapping breaks ordering and can cause incorrect partition pruning as reported in #9767 and #13105.

Do we need a spec change for clarification?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to updating the spec accordingly.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API: Integer truncate overflow can prune matching data files

5 participants