You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Decode the decimal scalar of a Rust-written flat node's min/max stats #710
Split from #583. ArrayStats.decodeScalar decodes a node's stats scalar without the column dtype, so a decimal bound arrives as text. #704 stops that from failing a filtered scan (the bound is ignored), but Java then never prunes on it.
To do: find the exact ScalarValue Rust writes for a decimal stat (look it up in the Rust source, not by probing bytes), pass the column dtype to ArrayStats.fromFbs, decode it to BigDecimal, and cross-check with a file whose decimal column has no zone map.
Affects only pruning of decimal columns without a zone map. vortex-jni writes a zone map even for 100 rows, so the path was not reachable from the small file in DecimalBytePartsInteropIntegrationTest; a non-zoned layout is needed to test it.
Fixed in #740. Rust writes a decimal bound as BytesValue: the little-endian two's-complement bytes of the unscaled value, 1/2/4/8/16/32 bytes (ScalarValue::to_proto / bytes_from_proto, vortex 0.93.0); Java now decodes it against the column dtype.
The cross-check this ticket asked for isn't possible with vortex-jni 0.93: a probe on a 100-row decimal column (byte-parts shapes, and p20/p38 plain vortex.decimal) found no min/max on the root node of any file it wrote, so the path is covered by unit tests of Rust's encoding instead. Left as is: the inspector still shows these bounds as text (it has no per-node dtype), and nested decimal leaves have no column dtype to decode against.
Activity
Decode the decimal min/max of a flat node's stats
dfa1 commentedon Nov 12, 2026
Fixed in #740. Rust writes a decimal bound as
BytesValue: the little-endian two's-complement bytes of the unscaled value, 1/2/4/8/16/32 bytes (ScalarValue::to_proto/bytes_from_proto, vortex 0.93.0); Java now decodes it against the column dtype.The cross-check this ticket asked for isn't possible with vortex-jni 0.93: a probe on a 100-row decimal column (byte-parts shapes, and p20/p38 plain
vortex.decimal) found no min/max on the root node of any file it wrote, so the path is covered by unit tests of Rust's encoding instead. Left as is: the inspector still shows these bounds as text (it has no per-node dtype), and nested decimal leaves have no column dtype to decode against.Decode the decimal min/max of a flat node's stats