Skip to content

fix: Allow segment pruners to indicate what segment details they need to effectively prune + fixup MSQ table slicer to honor such requests - #20892

Open
capistrant wants to merge 2 commits into
apache:masterfrom
capistrant:msqtableslicer-honorprunerdesires
Open

capistrant wants to merge 2 commits into
apache:masterfrom
capistrant:msqtableslicer-honorprunerdesires

Conversation

@capistrant

Copy link
Copy Markdown
Contributor

Description

FilterSegmentPruner can prune segments based on cluster groups for segments that are built using them since #20149. This should allow msq-task engine queries to prune segments when cluster groups are being filtered on. However, that is not the case because #20006 cut back on the segment details that the RetrieveUsedSegmentsAction grabs.

In the effort to actually support full pruning capabilities for FilterSegmentPruner (and create a path forward for future pruners), this PR proposes the SegmentPruner interface gains a new method getRequiredSegmentDetails which returns segment details it needs to make pruning decisions. The MSQ module takes advantage of this new method in the table input slicer by submitting the RetrieveUsedSegmentsAction request with the details needed by the pruner, if any exist. This allows msq-task queries to prune on cluster groups when there are there.

Release note

Close a functionality gap in segment pruning by msq-task queries by providing them the information about the input segments that they need to prune on cluster groups if the query is filtering on columns that are cluster group column in the underlying data. Enhancing the impact of #20149


Key changed/added classes in this PR
  • SegmentPruner
  • FilterSegmentPruner
  • IndexerTableInputSpecSlicer

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@github-actions github-actions Bot added Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Nov 12, 2026

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No actionable issues found. The required segment details are correctly propagated through the MSQ slicer and action clients, with coverage for cluster-group pruning and detail requests.

Reviewed 9 of 9 changed files.

Validation: git diff --check passed. No tests or builds were run.


This is an automated review by Codex GPT-5.6-Luna(max)

@FrankChen021 FrankChen021 added the Approval Recommended Used when AI completes a review and reports no findings label Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Approval Recommended Used when AI completes a review and reports no findings Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants