Skip to content

ci: run PR CI in one workflow that ends in a quality-gate job - #20878

Merged
amaechler merged 6 commits into
apache:masterfrom
amaechler:ci-quality-gate
Nov 13, 2026
Merged

amaechler merged 6 commits into
apache:masterfrom
amaechler:ci-quality-gate

Conversation

@amaechler

@amaechler amaechler commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Description

This PR merges Static Checks CI into Unit & Integration tests CI, so PR CI is one workflow that ends in a quality-gate job. quality-gate fails unless every other job passed, so a follow-up .asf.yaml change can require that single check to merge (which is the main reason I started this).

Today no check is required. This means an approved PR shows a green merge button even when CI never ran: #19884 merged with no check runs during a CI outage, and #20413 merged after its only unit-test run ended in startup_failure.

This PR changes workflows only. Nothing becomes required yet.

What changes

  • Gate: the trigger workflow calls ci.yml as one job, ci. quality-gate needs only that call and fails unless it succeeded, so every job in ci.yml is gated with no needs list to maintain. On a cancelled run it reports "cancelled", which does not satisfy a required check.
  • Path filter: the wildcard paths-ignore patterns move from the trigger into a changes job (dorny/paths-filter, pinned to a SHA on the ASF allowlist). A filter on the trigger starts no run for a docs-only PR, so a required check would never report and would block that PR (GitHub docs). Now the workflow always starts, and the Java jobs skip when only ignored paths changed.
  • Static checks and Docker tests: packaging-check and web-checks move from static-checks.yml into ci.yml, which now also calls docker-tests.yml. The jobs themselves and worker.yml are unchanged.
  • OpenRewrite: the dry run moves to the end of static-checks-maven.sh. This saves one job and one mvn install per push; static-checks-maven grows to about 33 min.

Two costs: check names gain a ci / prefix, and the Java jobs wait for changes to get a runner before they can queue (so one extra wait).

Follow-ups

To make the check required:

  • .asf.yaml: a Require CI ruleset that requires quality-gate on master. Committers can still merge a red PR by bypassing.
  • Auto-merge: allow_auto_merge, once the gate has proven itself.

Split out of this PR during review, each as its own PR:

  • Read-only token: a permissions block for the CI workflows, which have none on master today (discussion, discussion).
  • Job names: clearer names through each job's key (discussion).
  • Embed* tests: their own unit-test job (discussion).
  • Test-failure summaries: publish the JUnit report's summary for failing jobs (discussion).
  • Workflow names: ci.yml as the entry point with the jobs in ci-jobs.yml, if we agree on it (discussion).
Testing

I ran this branch on my fork with a push and three test PRs:

Case Result
Push to a release-pattern branch Every job green; coverage-jacoco skipped, as intended on push; quality-gate green
Non-docs change All 22 jobs green, including coverage-jacoco and quality-gate
Docs-only change The six Java jobs skipped, quality-gate green
Failing unit test That shard and quality-gate red
Cancelled run quality-gate reported "cancelled"
"Re-run failed jobs" quality-gate ran again
A job GitHub cancelled because no runner picked it up quality-gate red

This PR has:

  • been self-reviewed.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.

Merge Static Checks CI and Unit & Integration tests CI into ci.yml. A
changes job replaces the workflow-level paths-ignore, and quality-gate
fails unless every job it needs passed, so it can become the one
required check.

- Rename the jobs after what they run: unit-tests (classes C*),
  sql-query-tests (part N of 4), error-prone, static-analysis,
  distribution-build, docs-and-web-console, docker-image-tests.
- Run the OpenRewrite dry run inside static-analysis.
- Run actions-timeline inside quality-gate.
- worker.yml writes a job summary, and coverage-jacoco reads the PR's
  current labels for jacoco:skip, so a re-run after labeling works.
- Grant the CI token only contents, pull-requests and actions read.
GitHub appends matrix values to a job name that does not use them, which
showed the job as distribution-build (25).
paths-filter ignores base on pull requests and warns when it is set.
- quality-gate accepts a skipped coverage-jacoco only on push.
- changes checks out the repository only on push; pull requests are
  read through the API.
- Drop the workflow-level SEGMENT_DOWNLOAD_TIMEOUT_MINS (worker.yml sets
  it), the docker job's own summary step, and job names that repeat the
  job id.
- The job summary labels the script "Command" instead of promising a
  local run.
@amaechler amaechler self-assigned this Nov 10, 2026
@amaechler amaechler added Area - Dev For items related to the project itself, like dev docs and checklists, but not CI Area - GitHub Actions Any GitHub Action change labels Nov 10, 2026
@github-actions github-actions Bot added the GHA label Nov 10, 2026

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No actionable issues found in this review. The consolidated workflow preserves the intended CI coverage, the path filter and quality gate agree on when Java jobs may skip, and the current head's GitHub CI run completed successfully.

Reviewed 8 of 8 changed files.

Validation: inspected the current head's GitHub check rollup and quality-gate log; all reported checks passed. git diff --check also passed.


This is an automated review by Codex GPT-5.6 Luna(Max)

@FrankChen021 FrankChen021 added the Approval Recommended Used when AI completes a review and reports no findings label Nov 11, 2026
Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml
name: "unit tests(main)"
# Path filtering lives here, not in a workflow-level paths-ignore: a skipped workflow
# never reports quality-gate, so docs-only PRs could not merge once it is required.
changes:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I doubt the value of this changes stuff a bit - but my measurements showed that it has benefits...later this step will become compile and it will also produce the artifacts; so I think the pattern stuff will eventually be part of a script

based on the changeset later we could do selective testing in PRs which will be faster for leaf modules.

fun fact: I did experimented with similar changes on a branch for a different reason...I haven't opened a PR because I was considering altearnates instead of this :)

Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/worker.yml Resolved
Comment thread .github/workflows/ci.yml Outdated
- name: "QTest 3/4"
args: "-Dtest=QTest -Dquidem.split=3/4"
- { part: 1, split: "0/4" }
- { part: 2, split: "1/4" }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we extract Embed* into a separate split?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to do that as a follow-up, so this PR stays about the gate. Two things I'd like your view on first. Embed* matches 14 test classes, about 7 of the 23 minutes of test time in the E*,G*,F* shard, and three other shards are slower (29 to 30 min), so the run would not get shorter. And the embedded-tests module has 160 test classes, of which only 10 start with Embed. Is the goal to isolate the embedded-cluster tests, or to rebalance the shards?

Comment thread .github/workflows/ci.yml Outdated
Comment on lines +40 to +41
# Path filtering lives here, not in a workflow-level paths-ignore: a skipped workflow
# never reports quality-gate, so docs-only PRs could not merge once it is required.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI context leak of history - the value of such comment is nonexistent; and it could also confuse future agents with the nonsense they start to haul over time

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I personally also (almost always ;) hate the history part.

Comment thread .github/workflows/ci.yml
pull-requests: read # paths-filter lists a PR's files; coverage-jacoco reads its labels
actions: read # actions-timeline reads the run's jobs

jobs:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: ideally this file should not contain runs-on: things...only calls to other workflows so that this is the core orchestrating logic and execution is not blurred into it.

of course we can deviate....its just a concept :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. For now, I just kept it in changes (which has no separate workflow yet) and the only reason to use ubuntu-slim is that those runners (based on Claude suggestion and running some experiments) are more readily available.

Comment thread .github/workflows/worker.yml Outdated
jobs:
execute:
name: "${{ inputs.key }}"
name: "jdk ${{ inputs.jdk }}"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restore the usage of the key ; this job is not about jdk 25 which is invoked like like 20 times...if the jdk is important - then the caller should place that into the key

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed, and removed the whole the renaming portion for now. We can do it separately.

Comment thread .github/workflows/ci.yml Outdated
quality-gate:
name: "quality-gate"
if: always()
needs: [changes, distribution-build, docs-and-web-console, static-analysis, error-prone, unit-tests, sql-query-tests, coverage-jacoco, docker-image-tests]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that I see how this looks like on the UI not sure if we want to sunset the trigger job part or not....
change it for sure - but maybe utilize it for the quality-gate only

  • it separates these things
  • makes it less likely that it gets altered by mistake
  • the github UI renders in a better form that way

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's see.... With the last commit, he trigger workflow stays as a thin entry point.

On "change it for sure": the file and the workflow are still named after unit and integration tests, although they now run everything. What do you think about swapping the names so they match the roles, something like this:

File Role Workflow name
ci.yml Entry point: triggers, the call, quality-gate, actions-timeline CI
ci-jobs.yml The jobs, called by ci.yml (today's ci.yml) CI jobs

Pure move with no change in behaviour. Maybe I can do that in a follow-up PR if that makes sense to you as well.

Comment thread .github/workflows/worker.yml Outdated
echo '```'
echo "$SCRIPT"
echo '```'
} >> "$GITHUB_STEP_SUMMARY"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kinda wanted to add job summaries less so to just describe the job but to more easily see what failed. In iow I have this junit reporter where you can quickly see which unit tests failed without reading through thousands of lines of code. But the summary the way it is is not very useful agreed, I'll defer. Maybe the summary stuff is not that useful anymore anyway with the help of agents.

Address review feedback on the single-workflow layout:

- unit-and-integration-tests-unified.yml stays the entry point. It calls
  ci.yml as one job and holds quality-gate, which fails unless that call
  succeeded, so every job in ci.yml is gated without a needs list.
  actions-timeline is its own job again.
- ci.yml stays a reusable workflow. It gains the changes job, takes over
  packaging-check and web-checks from static-checks.yml, and calls
  docker-tests.yml instead of inlining it.
- worker.yml, the job names, the QTest matrix and the jacoco:skip
  condition are back to their state on master.
- The path filter keeps the wildcard patterns only.
- The explicit token permissions are dropped here.

@amaechler amaechler left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed review Zoltan! I tried to cut down on the noise a bit on this iteration and undo a bunch of changes based on your suggestions (some of it can follow later).

Comment thread .github/workflows/ci.yml Outdated
quality-gate:
name: "quality-gate"
if: always()
needs: [changes, distribution-build, docs-and-web-console, static-analysis, error-prone, unit-tests, sql-query-tests, coverage-jacoco, docker-image-tests]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's see.... With the last commit, he trigger workflow stays as a thin entry point.

On "change it for sure": the file and the workflow are still named after unit and integration tests, although they now run everything. What do you think about swapping the names so they match the roles, something like this:

File Role Workflow name
ci.yml Entry point: triggers, the call, quality-gate, actions-timeline CI
ci-jobs.yml The jobs, called by ci.yml (today's ci.yml) CI jobs

Pure move with no change in behaviour. Maybe I can do that in a follow-up PR if that makes sense to you as well.

Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml Outdated

# The one check to require in .asf.yaml. A job skipped by a condition reports success,
# so this job always runs and fails itself unless every job it needs passed.
quality-gate:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I undid the timeline change. With the quality-gate back in unit test workflow, I didn't use an aggregate job for now and it's simply:

  quality-gate:
    needs: [ci]
    runs-on: ubuntu-slim
    if: ${{ !cancelled() }}
    steps:
      - name: Fail unless CI passed
        if: ${{ needs.ci.result != 'success' }}
        run: exit 1

Comment thread .github/workflows/ci.yml
pull-requests: read # paths-filter lists a PR's files; coverage-jacoco reads its labels
actions: read # actions-timeline reads the run's jobs

jobs:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. For now, I just kept it in changes (which has no separate workflow yet) and the only reason to use ubuntu-slim is that those runners (based on Claude suggestion and running some experiments) are more readily available.

Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/worker.yml Outdated
echo '```'
} >> "$GITHUB_STEP_SUMMARY"

- name: 🏷️ skip label

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml Outdated
- name: "QTest 3/4"
args: "-Dtest=QTest -Dquidem.split=3/4"
- { part: 1, split: "0/4" }
- { part: 2, split: "1/4" }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to do that as a follow-up, so this PR stays about the gate. Two things I'd like your view on first. Embed* matches 14 test classes, about 7 of the 23 minutes of test time in the E*,G*,F* shard, and three other shards are slower (29 to 30 min), so the run would not get shorter. And the embedded-tests module has 160 test classes, of which only 10 start with Embed. Is the goal to isolate the embedded-cluster tests, or to rebalance the shards?

Comment thread .github/workflows/ci.yml Outdated
Comment on lines +36 to +37
pull-requests: read # paths-filter lists a PR's files; coverage-jacoco reads its labels
actions: read # actions-timeline reads the run's jobs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed for now and we don't really need it. It was a bot suggestion because apparently when merging a PR and pushing to master, the run has write access to everything (contents, actions, packages, pull requests and more), and every Maven build, npm install and third-party action in these jobs runs with write permissions. I might suggest a read-only token in a separate PR and we can discuss more separately. The only benefit I really see is the "third-party action" piece.

Comment thread .github/workflows/ci.yml Outdated
Comment on lines +40 to +41
# Path filtering lives here, not in a workflow-level paths-ignore: a skipped workflow
# never reports quality-gate, so docs-only PRs could not merge once it is required.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I personally also (almost always ;) hate the history part.

@FrankChen021 FrankChen021 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The current workflow refactor correctly consolidates the CI jobs behind the reusable workflow and the current head's GitHub check run completed successfully. Before making this gate the basis for merge protection, restore explicit least-privilege token permissions: the current layout lets repository defaults determine the token available to PR-controlled build scripts and third-party actions, so a write-default configuration creates a repository mutation risk.

Reviewed 6 of 6 changed files.

Validation: inspected the current head's GitHub check rollup with gh pr checks https://github.com/apache/druid/pull/20878; all reported checks completed successfully. No local build or test commands were run.

Severity Findings
P0 0
P1 1
P2 0
P3 0
Total 1

This is an automated review by Codex GPT-5.6 Luna(Max)

After addressing the findings or replying to the comments, you can request another review from me to trigger a new automated review.

Comment thread .github/workflows/ci.yml
- id: filter
# Third-party actions must be pinned to a SHA on the ASF allowlist
# https://github.com/apache/infrastructure-actions/blob/main/approved_patterns.yml
uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Restore least-privilege workflow-token permissions

Finding: The refactor removes the former read-only permissions declaration from the CI workflow and leaves the new caller workflow without one as well. On same-repository pull requests and pushes where repository defaults grant write access, the jobs execute PR-controlled Maven/npm scripts and third-party actions with that write-capable GITHUB_TOKEN, so a compromised build step can modify repository or pull-request state.

Suggestion: Declare least-privilege permissions at both the caller and reusable-workflow boundaries, granting only the contents, pull-requests, and actions read access required by checkout, paths-filter, coverage, and actions-timeline.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that it is worth having. master has no permissions block in these workflows today, so this PR does not change what the token can do. The first revision added one, and I took it out again after this discussion to keep this PR about the gate. I'll open a separate PR for a read-only token once this one is in.

@FrankChen021 FrankChen021 removed the Approval Recommended Used when AI completes a review and reports no findings label Nov 13, 2026
@Fly-Style
Fly-Style self-requested a review November 13, 2026 07:20

@kgyrtkirk kgyrtkirk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool!
Thank you for the updates!

@@ -14,49 +14,14 @@
# limitations under the License.

name: "Unit & Integration tests CI"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: we could probably rename this to something shorter like CI

@amaechler amaechler Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 3729896: the trigger workflow is now CI, and the called one is CI jobs. I'll rename the files in a follow-up.

@amaechler
amaechler merged commit 32384e0 into apache:master Nov 13, 2026
28 checks passed
@github-actions github-actions Bot added this to the 40.0.0 milestone Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area - Dev For items related to the project itself, like dev docs and checklists, but not CI Area - GitHub Actions Any GitHub Action change GHA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants