Skip to content

ScanIterator: nested struct column fails chunk planning ("0 flats" vs widest column) #207

Closed

Description

@dfa1

Found by the Raincloud conformance PoC (#205). countries-of-the-world (written by vortex-data==0.69.0) has a top-level struct column data that is itself a struct — its layout subtree is vortex.struct(262 rows) → 4× flat. ScanIterator.buildChunks doesn't treat the nested struct layout as a chunk source and fails:

VortexException: scan: column 'data' has 0 flats but the widest column has 1;
mixed per-column chunking beyond 1-vs-N is not supported
    at ScanIterator.buildChunks(ScanIterator.java:144)

Full layout tree of the file:

struct (262 rows)
  region:       vortex.stats → 2× [vortex.dict → 2× flat]
  country_code: vortex.stats → 2× flat
  data:         vortex.struct(262 rows) → 4× flat

schema/count/inspect all handle the file; only the scan path fails. Repro: python -m scripts.pipeline.build countries-of-the-world in Raincloud, then vortex-cli export countries-of-the-world.vortex.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions