Skip to content

fix(writer): a nullable text column without a null is written without validity (#650) - #692

Merged
dfa1 merged 1 commit into
mainfrom
fix/650-nullable-text-dict
Nov 7, 2026
Merged

dfa1 merged 1 commit into
mainfrom
fix/650-nullable-text-dict

Conversation

@dfa1

@dfa1 dfa1 commented Nov 7, 2026

Copy link
Copy Markdown
Owner

First slice of #650 (does not close it).

Rust imports a nullable string array with no null as all-valid, so it has no validity child. Java wrapped it in vortex.masked with a constant validity. It now goes to the compressor unwrapped, also under a ColumnEncoding override; at cascade depth 0 it is still masked.

  • NullableTextInteropIntegrationTest: vortex-jni vs Java encoding tree (no-null unique shape matches; it failed before), and vortex-jni reads every Java file, full, filtered and ranged, compact included.
  • NullableTextMaskTest: masked only when there is validity to store.

Still open on #650: nullable text with nulls (validity inside FSST/OnPair/VarBin/Dict), FSST's lengths child (sequence vs primitive), the string probe. Struct fields and list elements of nullable text keep their previous path (no cross-check yet).

🤖 Generated with Claude Code

https://claude.ai/code/session_011fRbYsrGtpKHp9APhU8qDV

… validity (#650)

Rust imports a nullable string array whose null count is zero as all-valid, with no
validity child, so the cascade never wraps it. Java wrapped it in vortex.masked with a
constant validity. Route it to the compressor, as a nullable primitive without nulls.

NullableTextInteropIntegrationTest compares encoding trees with vortex-jni (the
no-null unique shape matches now; the other shapes still differ in the text scheme
itself, tracked in #650) and has vortex-jni read every Java file, filtered and ranged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011fRbYsrGtpKHp9APhU8qDV
@dfa1
dfa1 merged commit 8c577ba into main Nov 7, 2026
6 checks passed
@dfa1
dfa1 deleted the fix/650-nullable-text-dict branch November 7, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant