Skip to content

fix: publish a CollectLeft join's dynamic filter on the first build report - #26847

Open
lakshmie6x wants to merge 1 commit into
apache:mainfrom
lakshmie6x:collect-left-filter-first-report
Open

lakshmie6x wants to merge 1 commit into
apache:mainfrom
lakshmie6x:collect-left-filter-first-report

Conversation

@lakshmie6x

Copy link
Copy Markdown

Every probe partition of a CollectLeft hash join reports the same shared build to SharedBuildAccumulator, yet the filter is only finalized after all of them have reported, so each partition waits for every other one before probing. That build is already complete at the first report. A scheduler that runs only some of a plan's partitions at a time deadlocks on this wait: the running partitions wait for reports from partitions that never get to run.

The first CollectLeft report now finalizes the filter. No probe runs with a partial filter, unlike #21641, because the first report already has the whole build. Partitioned joins still wait for every build partition. The new test runs one probe partition of two; it hangs without the change.

@lakshmie6x

Copy link
Copy Markdown
Author

run benchmark hj

@adriangbot

Copy link
Copy Markdown

@lakshmie6x

Copy link
Copy Markdown
Author

run benchmark tpch

@adriangbot

Copy link
Copy Markdown

@github-actions github-actions Bot added the physical-plan Changes to the physical-plan crate label Oct 28, 2026
…eport

Every probe partition of a CollectLeft join reports the one shared build, yet
each waited for all of them before probing. A scheduler that runs only some of
a stage's partitions at a time then deadlocks: the running partitions wait for
reports from partitions that never get a slot, and hold every slot.

The first report already has the whole build, so it now finalizes the filter.
Partitioned joins still wait for every build partition.
@lakshmie6x
lakshmie6x force-pushed the collect-left-filter-first-report branch from 3c47013 to c668e2a Compare October 29, 2026 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants