You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Nullable ALP flat columns drop row validity — nulls export as invented values (silent corruption) #210
Found by the Raincloud conformance harness (#205) on palmer-archipelago-antarctica-penguin-data once #206 unblocked the scan.
bill_length_mm / bill_depth_mm are nullable F64 flat columns encoded as vortex.alp (no vortex.masked wrapper in the tree — layout shows vortex.stats → flat with encodings [vortex.struct, vortex.alp]). The file genuinely carries the nulls: the Python bindings (vortex-data==0.69.0) read rows 3 and 271 back as None. vortex-java exports 32.1 / 13.1 for those rows — the decoded ALP values at the null positions — so validity is dropped somewhere between the array node and the scan output when it is not expressed via vortex.masked.
Every prior ok dataset (uci-iris, uci-seeds, …) is null-free, so this path was never exercised: nullable + ALP + real nulls is the trigger. Likely broader than ALP — any nullable flat column whose validity rides on the array node rather than a masked wrapper. Worth checking how the 0.69 writer serializes validity (array-node validity child?) against what our decoders consume.
Repro: hydrate the slug via Raincloud, vortex-cli export …vortex -, rows 4/272 of the CSV vs the parquet sibling.
Activity
test: ground-truth RLE F64 interop + review nits (#209)
fix: lazy dict decode for I8/U8/I16/U16 value columns (#206)
fix: propagate row validity through wrapper decoders; NullArray CSV e…
test: ground-truth RLE F64 interop + review nits (#209)
fix: lazy dict decode for I8/U8/I16/U16 value columns (#206)
fix: propagate row validity through wrapper decoders; NullArray CSV e…
test: deterministic coverage for validity propagation (#210 review)
fix: lazy dict decode for I8/U8/I16/U16 value columns (#206)
fix: propagate row validity through wrapper decoders; NullArray CSV e…
test: deterministic coverage for validity propagation (#210 review)
test: triage round 4 — 10 more corpus slugs (38 ok total)
fix: propagate validity in RunEnd and Sparse lazy decoders (#225, #226)
refactor(reader): share MaskedArray unwrap/rewrap across 9 decoders