Repository navigation
ci: run PR CI in one workflow that ends in a quality-gate job - #20878
Conversation
Merge Static Checks CI and Unit & Integration tests CI into ci.yml. A changes job replaces the workflow-level paths-ignore, and quality-gate fails unless every job it needs passed, so it can become the one required check. - Rename the jobs after what they run: unit-tests (classes C*), sql-query-tests (part N of 4), error-prone, static-analysis, distribution-build, docs-and-web-console, docker-image-tests. - Run the OpenRewrite dry run inside static-analysis. - Run actions-timeline inside quality-gate. - worker.yml writes a job summary, and coverage-jacoco reads the PR's current labels for jacoco:skip, so a re-run after labeling works. - Grant the CI token only contents, pull-requests and actions read.
GitHub appends matrix values to a job name that does not use them, which showed the job as distribution-build (25).
paths-filter ignores base on pull requests and warns when it is set.
- quality-gate accepts a skipped coverage-jacoco only on push. - changes checks out the repository only on push; pull requests are read through the API. - Drop the workflow-level SEGMENT_DOWNLOAD_TIMEOUT_MINS (worker.yml sets it), the docker job's own summary step, and job names that repeat the job id. - The job summary labels the script "Command" instead of promising a local run.
FrankChen021
left a comment
There was a problem hiding this comment.
🟢 Approval recommended
No actionable issues found in this review. The consolidated workflow preserves the intended CI coverage, the path filter and quality gate agree on when Java jobs may skip, and the current head's GitHub CI run completed successfully.
Reviewed 8 of 8 changed files.
Validation: inspected the current head's GitHub check rollup and quality-gate log; all reported checks passed. git diff --check also passed.
This is an automated review by Codex GPT-5.6 Luna(Max)
| name: "unit tests(main)" | ||
| # Path filtering lives here, not in a workflow-level paths-ignore: a skipped workflow | ||
| # never reports quality-gate, so docs-only PRs could not merge once it is required. | ||
| changes: |
There was a problem hiding this comment.
nit: I doubt the value of this changes stuff a bit - but my measurements showed that it has benefits...later this step will become compile and it will also produce the artifacts; so I think the pattern stuff will eventually be part of a script
based on the changeset later we could do selective testing in PRs which will be faster for leaf modules.
fun fact: I did experimented with similar changes on a branch for a different reason...I haven't opened a PR because I was considering altearnates instead of this :)
| - name: "QTest 3/4" | ||
| args: "-Dtest=QTest -Dquidem.split=3/4" | ||
| - { part: 1, split: "0/4" } | ||
| - { part: 2, split: "1/4" } |
There was a problem hiding this comment.
also can we extract Embed* into a separate split?
There was a problem hiding this comment.
Happy to do that as a follow-up, so this PR stays about the gate. Two things I'd like your view on first. Embed* matches 14 test classes, about 7 of the 23 minutes of test time in the E*,G*,F* shard, and three other shards are slower (29 to 30 min), so the run would not get shorter. And the embedded-tests module has 160 test classes, of which only 10 start with Embed. Is the goal to isolate the embedded-cluster tests, or to rebalance the shards?
| # Path filtering lives here, not in a workflow-level paths-ignore: a skipped workflow | ||
| # never reports quality-gate, so docs-only PRs could not merge once it is required. |
There was a problem hiding this comment.
AI context leak of history - the value of such comment is nonexistent; and it could also confuse future agents with the nonsense they start to haul over time
There was a problem hiding this comment.
Good point, I personally also (almost always ;) hate the history part.
| pull-requests: read # paths-filter lists a PR's files; coverage-jacoco reads its labels | ||
| actions: read # actions-timeline reads the run's jobs | ||
|
|
||
| jobs: |
There was a problem hiding this comment.
note: ideally this file should not contain runs-on: things...only calls to other workflows so that this is the core orchestrating logic and execution is not blurred into it.
of course we can deviate....its just a concept :)
There was a problem hiding this comment.
Makes sense. For now, I just kept it in changes (which has no separate workflow yet) and the only reason to use ubuntu-slim is that those runners (based on Claude suggestion and running some experiments) are more readily available.
| jobs: | ||
| execute: | ||
| name: "${{ inputs.key }}" | ||
| name: "jdk ${{ inputs.jdk }}" |
There was a problem hiding this comment.
restore the usage of the key ; this job is not about jdk 25 which is invoked like like 20 times...if the jdk is important - then the caller should place that into the key
There was a problem hiding this comment.
Removed, and removed the whole the renaming portion for now. We can do it separately.
| quality-gate: | ||
| name: "quality-gate" | ||
| if: always() | ||
| needs: [changes, distribution-build, docs-and-web-console, static-analysis, error-prone, unit-tests, sql-query-tests, coverage-jacoco, docker-image-tests] |
There was a problem hiding this comment.
now that I see how this looks like on the UI not sure if we want to sunset the trigger job part or not....
change it for sure - but maybe utilize it for the quality-gate only
- it separates these things
- makes it less likely that it gets altered by mistake
- the github UI renders in a better form that way
There was a problem hiding this comment.
Ok, let's see.... With the last commit, he trigger workflow stays as a thin entry point.
On "change it for sure": the file and the workflow are still named after unit and integration tests, although they now run everything. What do you think about swapping the names so they match the roles, something like this:
| File | Role | Workflow name |
|---|---|---|
ci.yml |
Entry point: triggers, the call, quality-gate, actions-timeline |
CI |
ci-jobs.yml |
The jobs, called by ci.yml (today's ci.yml) |
CI jobs |
Pure move with no change in behaviour. Maybe I can do that in a follow-up PR if that makes sense to you as well.
| echo '```' | ||
| echo "$SCRIPT" | ||
| echo '```' | ||
| } >> "$GITHUB_STEP_SUMMARY" |
There was a problem hiding this comment.
what's the point of doing this ?
who will read these:
https://github.com/apache/druid/actions/runs/42863025850/attempts/1#summary-129999937868
?
There was a problem hiding this comment.
I kinda wanted to add job summaries less so to just describe the job but to more easily see what failed. In iow I have this junit reporter where you can quickly see which unit tests failed without reading through thousands of lines of code. But the summary the way it is is not very useful agreed, I'll defer. Maybe the summary stuff is not that useful anymore anyway with the help of agents.
Address review feedback on the single-workflow layout: - unit-and-integration-tests-unified.yml stays the entry point. It calls ci.yml as one job and holds quality-gate, which fails unless that call succeeded, so every job in ci.yml is gated without a needs list. actions-timeline is its own job again. - ci.yml stays a reusable workflow. It gains the changes job, takes over packaging-check and web-checks from static-checks.yml, and calls docker-tests.yml instead of inlining it. - worker.yml, the job names, the QTest matrix and the jacoco:skip condition are back to their state on master. - The path filter keeps the wildcard patterns only. - The explicit token permissions are dropped here.
amaechler
left a comment
There was a problem hiding this comment.
Thanks for the detailed review Zoltan! I tried to cut down on the noise a bit on this iteration and undo a bunch of changes based on your suggestions (some of it can follow later).
| quality-gate: | ||
| name: "quality-gate" | ||
| if: always() | ||
| needs: [changes, distribution-build, docs-and-web-console, static-analysis, error-prone, unit-tests, sql-query-tests, coverage-jacoco, docker-image-tests] |
There was a problem hiding this comment.
Ok, let's see.... With the last commit, he trigger workflow stays as a thin entry point.
On "change it for sure": the file and the workflow are still named after unit and integration tests, although they now run everything. What do you think about swapping the names so they match the roles, something like this:
| File | Role | Workflow name |
|---|---|---|
ci.yml |
Entry point: triggers, the call, quality-gate, actions-timeline |
CI |
ci-jobs.yml |
The jobs, called by ci.yml (today's ci.yml) |
CI jobs |
Pure move with no change in behaviour. Maybe I can do that in a follow-up PR if that makes sense to you as well.
|
|
||
| # The one check to require in .asf.yaml. A job skipped by a condition reports success, | ||
| # so this job always runs and fails itself unless every job it needs passed. | ||
| quality-gate: |
There was a problem hiding this comment.
I undid the timeline change. With the quality-gate back in unit test workflow, I didn't use an aggregate job for now and it's simply:
quality-gate:
needs: [ci]
runs-on: ubuntu-slim
if: ${{ !cancelled() }}
steps:
- name: Fail unless CI passed
if: ${{ needs.ci.result != 'success' }}
run: exit 1| pull-requests: read # paths-filter lists a PR's files; coverage-jacoco reads its labels | ||
| actions: read # actions-timeline reads the run's jobs | ||
|
|
||
| jobs: |
There was a problem hiding this comment.
Makes sense. For now, I just kept it in changes (which has no separate workflow yet) and the only reason to use ubuntu-slim is that those runners (based on Claude suggestion and running some experiments) are more readily available.
| echo '```' | ||
| } >> "$GITHUB_STEP_SUMMARY" | ||
|
|
||
| - name: 🏷️ skip label |
| - name: "QTest 3/4" | ||
| args: "-Dtest=QTest -Dquidem.split=3/4" | ||
| - { part: 1, split: "0/4" } | ||
| - { part: 2, split: "1/4" } |
There was a problem hiding this comment.
Happy to do that as a follow-up, so this PR stays about the gate. Two things I'd like your view on first. Embed* matches 14 test classes, about 7 of the 23 minutes of test time in the E*,G*,F* shard, and three other shards are slower (29 to 30 min), so the run would not get shorter. And the embedded-tests module has 160 test classes, of which only 10 start with Embed. Is the goal to isolate the embedded-cluster tests, or to rebalance the shards?
| pull-requests: read # paths-filter lists a PR's files; coverage-jacoco reads its labels | ||
| actions: read # actions-timeline reads the run's jobs |
There was a problem hiding this comment.
Removed for now and we don't really need it. It was a bot suggestion because apparently when merging a PR and pushing to master, the run has write access to everything (contents, actions, packages, pull requests and more), and every Maven build, npm install and third-party action in these jobs runs with write permissions. I might suggest a read-only token in a separate PR and we can discuss more separately. The only benefit I really see is the "third-party action" piece.
| # Path filtering lives here, not in a workflow-level paths-ignore: a skipped workflow | ||
| # never reports quality-gate, so docs-only PRs could not merge once it is required. |
There was a problem hiding this comment.
Good point, I personally also (almost always ;) hate the history part.
FrankChen021
left a comment
There was a problem hiding this comment.
🟡 Changes recommended
The current workflow refactor correctly consolidates the CI jobs behind the reusable workflow and the current head's GitHub check run completed successfully. Before making this gate the basis for merge protection, restore explicit least-privilege token permissions: the current layout lets repository defaults determine the token available to PR-controlled build scripts and third-party actions, so a write-default configuration creates a repository mutation risk.
Reviewed 6 of 6 changed files.
Validation: inspected the current head's GitHub check rollup with gh pr checks https://github.com/apache/druid/pull/20878; all reported checks completed successfully. No local build or test commands were run.
| Severity | Findings |
|---|---|
| P0 | 0 |
| P1 | 1 |
| P2 | 0 |
| P3 | 0 |
| Total | 1 |
This is an automated review by Codex GPT-5.6 Luna(Max)
After addressing the findings or replying to the comments, you can request another review from me to trigger a new automated review.
| - id: filter | ||
| # Third-party actions must be pinned to a SHA on the ASF allowlist | ||
| # https://github.com/apache/infrastructure-actions/blob/main/approved_patterns.yml | ||
| uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3 |
There was a problem hiding this comment.
[P1] Restore least-privilege workflow-token permissions
Finding: The refactor removes the former read-only permissions declaration from the CI workflow and leaves the new caller workflow without one as well. On same-repository pull requests and pushes where repository defaults grant write access, the jobs execute PR-controlled Maven/npm scripts and third-party actions with that write-capable GITHUB_TOKEN, so a compromised build step can modify repository or pull-request state.
Suggestion: Declare least-privilege permissions at both the caller and reusable-workflow boundaries, granting only the contents, pull-requests, and actions read access required by checkout, paths-filter, coverage, and actions-timeline.
There was a problem hiding this comment.
Agreed that it is worth having. master has no permissions block in these workflows today, so this PR does not change what the token can do. The first revision added one, and I took it out again after this discussion to keep this PR about the gate. I'll open a separate PR for a read-only token once this one is in.
kgyrtkirk
left a comment
There was a problem hiding this comment.
cool!
Thank you for the updates!
| @@ -14,49 +14,14 @@ | |||
| # limitations under the License. | |||
|
|
|||
| name: "Unit & Integration tests CI" | |||
There was a problem hiding this comment.
note: we could probably rename this to something shorter like CI
There was a problem hiding this comment.
Done in 3729896: the trigger workflow is now CI, and the called one is CI jobs. I'll rename the files in a follow-up.
Description
This PR merges
Static Checks CIintoUnit & Integration tests CI, so PR CI is one workflow that ends in aquality-gatejob.quality-gatefails unless every other job passed, so a follow-up.asf.yamlchange can require that single check to merge (which is the main reason I started this).Today no check is required. This means an approved PR shows a green merge button even when CI never ran: #19884 merged with no check runs during a CI outage, and #20413 merged after its only unit-test run ended in
startup_failure.This PR changes workflows only. Nothing becomes required yet.
What changes
ci.ymlas one job,ci.quality-gateneeds only that call and fails unless it succeeded, so every job inci.ymlis gated with noneedslist to maintain. On a cancelled run it reports "cancelled", which does not satisfy a required check.paths-ignorepatterns move from the trigger into achangesjob (dorny/paths-filter, pinned to a SHA on the ASF allowlist). A filter on the trigger starts no run for a docs-only PR, so a required check would never report and would block that PR (GitHub docs). Now the workflow always starts, and the Java jobs skip when only ignored paths changed.packaging-checkandweb-checksmove fromstatic-checks.ymlintoci.yml, which now also callsdocker-tests.yml. The jobs themselves andworker.ymlare unchanged.static-checks-maven.sh. This saves one job and onemvn installper push;static-checks-mavengrows to about 33 min.Two costs: check names gain a
ci /prefix, and the Java jobs wait forchangesto get a runner before they can queue (so one extra wait).Follow-ups
To make the check required:
.asf.yaml: aRequire CIruleset that requiresquality-gateonmaster. Committers can still merge a red PR by bypassing.allow_auto_merge, once the gate has proven itself.Split out of this PR during review, each as its own PR:
permissionsblock for the CI workflows, which have none onmastertoday (discussion, discussion).key(discussion).Embed*tests: their own unit-test job (discussion).ci.ymlas the entry point with the jobs inci-jobs.yml, if we agree on it (discussion).Testing
I ran this branch on my fork with a push and three test PRs:
coverage-jacocoskipped, as intended on push;quality-gategreencoverage-jacocoandquality-gatequality-gategreenquality-gateredquality-gatereported "cancelled"quality-gateran againquality-gateredThis PR has: