Skip to content

Give each inline V2 scan root its own IO session - #11226

Merged
robert3005 merged 2 commits into
developfrom
ji/eager-curie-0xhukx
Nov 6, 2026
Merged

robert3005 merged 2 commits into
developfrom
ji/eager-curie-0xhukx

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

When the DataFusion V2 scan driver runs chunks inline (concurrency × available parallelism <= 1), it admits each spawned chunk to the scan's own Run as another root. Every root was given the same IoSource session. The run clears a root's session when that root retires, so once the first chunk finished, its clear() cancelled the reads of the others, and they waited forever for completions that never came.

  • vortex-datafusion/src/persistent/scan_v2.rs: RunState holds the scan's IoService instead of a single session, and the driver calls io.session() for every root it admits: the first root and each inline child. Each root's session is now cleared on its own. All sessions still read from the same SegmentSource.

joseph-isaacs and others added 2 commits November 4, 2026 16:42
Inline roots shared one session, and the run clears a root's session when the root retires. Once the first root finished, the remaining roots waited for reads that were never issued.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MZ3d6HFyGGeUGy8brEr6BM
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
A scan concurrency of 0 runs every chunk on the scan's own run as another root. With one shared
IO session, the first chunk to retire cleared the reads of the others and the scan hung.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBCw9H5HzcnWiPpiQjH
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@robert3005 robert3005 added the changelog/fix A bug fix label Nov 6, 2026
@robert3005
robert3005 merged commit 5852636 into develop Nov 6, 2026
125 of 127 checks passed
@robert3005
robert3005 deleted the ji/eager-curie-0xhukx branch November 6, 2026 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants