You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Buffer length: Rust requires exactly rows × width bytes; Java only rejects a buffer that is too short.
Validity child: on a non-nullable node it is an error in Rust (fixed_width::deserialize_validity); Java silently ignores it.
Value range on legacy nodes: when the stored width is not the canonical width of the precision, Rust scans the valid values (WidenArray::from_stored_decimal → smallest_decimal_dtype) and fails if one is outside the declared precision. Canonical-width nodes are trusted and not scanned.
Alignment: the buffer must be aligned to the value width (DecimalData::validate).
Plan (one PR at a time, straight to main)
Bump vortex-jni 0.90.0 → 0.93.0 on its own, and fix or record whatever interop tests it breaks.
Make DecimalEncodingDecoder match the four rules above, with vortex-jni cross-checks: hand-built malformed nodes, plus legacy narrow-width files that must still read.
Keep this issue open for vortex.widen / DecimalByteParts v2 and follow upstream's preview edition.
Activity
dfa1 commentedon Nov 8, 2026
Status and plan (2026-11-08)
Checked against upstream #10701 / #10733 and the Rust
developsource.Already at parity
DecimalEncodingEncoderwrites every decimal at the canonical width of its precision (DType.Decimal#byteWidth), the rule upstream is moving to.DecimalEncodingDecoderalready rejects an unknownvalues_type.DecimalBytePartsEncodingEncodermatches Rust's current compressor (narrowed_decimal: the narrowest signed integer that holds the values).Blocked on upstream
vortex.widenandDecimalBytePartsv2 (upstream CD5, CD7 and WR1–WR3 in Tracking Issue: AddWidenencoding for integers and decimals vortex-data/vortex#10733): not merged, and no edition allows them yet.Doable now: Rust 0.92+
vortex.decimalread rules (vortex-data/vortex#10995,decimal/vtable/plugin/stored.rs)Java differs in four places:
rows × widthbytes; Java only rejects a buffer that is too short.fixed_width::deserialize_validity); Java silently ignores it.WidenArray::from_stored_decimal→smallest_decimal_dtype) and fails if one is outside the declared precision. Canonical-width nodes are trusted and not scanned.DecimalData::validate).Plan (one PR at a time, straight to
main)DecimalEncodingDecodermatch the four rules above, with vortex-jni cross-checks: hand-built malformed nodes, plus legacy narrow-width files that must still read.vortex.widen/DecimalBytePartsv2 and follow upstream's preview edition.