You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Avoid casting unreachable child values in nested List casts #24506
cast_list_column and cast_list_view_column preserve parent offsets/sizes and recursively cast the full backing child array. A sliced or limited List can therefore fail because an unreachable child value cannot be cast.
Example:
CREATETABLEltASSELECT*FROM (VALUES
(1, [struct('1')]), (2, [struct('2')]), (3, [struct('bad')])
) AS t(i, l);
SELECT arrow_cast(l, 'List(Struct("c0": Int32))') FROM lt LIMIT2;
-- currently fails while casting the unreachable `bad` value
Invariant / desired behavior
Recursive nested casts must process only child values reachable from visible, non-null parent rows. Hidden backing children must not produce cast errors.
Proposed direction
Establish one consistent child-selection strategy for List, LargeList, and ListView casts, analogous to Map entry compaction. Preserve visible offsets, nulls, and schema while avoiding recursive casts of unreachable children.
Scope
In
List, LargeList, and ListView recursive cast paths in datafusion/common/src/nested_struct.rs
Slice and null-parent regression coverage
Out
Map behavior (already compacts entries)
FixedSizeList behavior, unless the shared solution naturally applies
Acceptance criteria
A cast succeeds when invalid nested child data is reachable only through sliced-out or null parent rows.
Visible invalid child data still returns a cast error.
Output offsets, nulls, and data type are preserved.
Tests / verification
Unit regressions for List, LargeList, and ListView sliced/null-parent inputs.
Activity
goutamadwant commentedon Aug 22, 2026
take
goutamadwant commentedon Aug 23, 2026
@kosiew I have a fix with #24590
Let me know if it looks good or in case you have any suggestions. thanks!
fix: Avoid casting unreachable nested List values (apache#24590)