Repository navigation
fix(spec): format timestamp and binary partition values like iceberg-java - #3352
JeonDaehong wants to merge 2 commits into
Conversation
|
Hello. Thank you |
7da837a to
2cee4b7
Compare
|
Converted to draft to stay within the new limit of 3 open PRs (#3465). The change is still ready for review; I will mark it ready again when one of my other PRs lands. |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
Still relevant. It's a draft only because of the open PR limit (#3465), and I'll mark it ready once one of my open PRs lands. |
Which issue does this PR close?
What changes are included in this PR?
Datum::to_human_string, whichpartition_to_pathuses throughTransform::to_human_string, now matches iceberg-java for:timestamp/timestamp_ns:yyyy-MM-ddTHH:mm:ss, with trailing zeros of the fraction trimmed (.120000->.12, no fraction for whole seconds)timestamptz/timestamptz_ns: the same, followed by+00:00binary/fixed: standard Base64 with paddingDisplay for Datumis unchanged, so this only affects partition paths and thepartitions.<path>keys in the snapshot summary. Reads are not affected, since they take partition values from manifests. Tables already written by iceberg-rust with these partition types will get the new path format for new files.Are these changes tested?
Yes.
transform.rscover timestamps (micro and nano, with and without zone, whole seconds, trimmed fractions, pre-epoch values) and binary/fixed (empty, padding cases).partition.rstest checks a full path.Transform.toHumanString/PartitionSpec.partitionToPathonmain.cargo test -p iceberg --lib(1845 passed),cargo clippy -p iceberg --all-targetsandcargo fmt --checkare clean.AI Disclosure
I used Claude Code to help trace the Rust code path and draft the implementation and tests. I generated the expected values by running iceberg-java myself, and I reviewed the code and ran the tests and lints above.