Skip to content

Check that test JVM constraints are satisfiable - #13210

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
masterfrom
bbujon/test-jvm-constraints-check
Nov 5, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
masterfrom
bbujon/test-jvm-constraints-check

Conversation

@PerfectSlayer

@PerfectSlayer PerfectSlayer commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

What Does This Do

Adds a checkTestJvmConstraints task to every project with test JVM constraints, run by check. It fails when no JVM can satisfy a test task's testJvmConstraints:

  • every includeJdk entry is outside the version range and not forced; or
  • minJavaVersion is above maxJavaVersion, with no forceJdk entry left to run.

Motivation

A test task with unsatisfiable constraints is skipped on every JVM. The CI job still passes, so its tests silently stop running.

This happened twice, and both went unnoticed for a long time:

The check fails the existing check_* CI jobs as soon as such a combination is introduced.

Additional Notes

Contributor Checklist

Jira ticket: APMLP-2187

🤖 Generated with Claude Code

@PerfectSlayer PerfectSlayer added type: feature Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling tag: ai generated Largely based on code generated by an AI or LLM labels Nov 4, 2026
@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.97% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 42db38d | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.02 s 14.01 s [-0.7%; +0.8%] (no difference)
startup:insecure-bank:tracing:Agent 13.00 s 12.99 s [-0.7%; +0.8%] (no difference)
startup:petclinic:appsec:Agent 17.71 s 17.53 s [+0.2%; +1.8%] (maybe worse)
startup:petclinic:iast:Agent 17.46 s 17.56 s [-1.3%; +0.1%] (no difference)
startup:petclinic:profiling:Agent 17.30 s 17.29 s [-1.0%; +1.2%] (no difference)
startup:petclinic:sca:Agent 17.02 s 17.48 s [-6.8%; +1.6%] (no difference)
startup:petclinic:tracing:Agent 16.55 s 16.77 s [-2.2%; -0.4%] (maybe better)

Commit: 42db38d6 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@PerfectSlayer
PerfectSlayer marked this pull request as ready for review November 4, 2026 12:11
@PerfectSlayer
PerfectSlayer requested review from a team as code owners November 4, 2026 12:11
@PerfectSlayer
PerfectSlayer requested review from bric3 and cecile75 and removed request for a team November 4, 2026 12:11
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T12:17:20.983783Z 609dac6 Draft marked ready
🔒 Security Review ✅ Completed 2026-11-04T12:14:41.484515Z 609dac6 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 609dac6ad6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

A versionless JVM alias can make the new check accept contradictory Java bounds, leaving tests silently skipped on every JVM.

Open Bits AI session

🤖 Bits Code Review · Commit 609dac6

@PerfectSlayer

Copy link
Copy Markdown
Collaborator Author

@DataDog review

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The shared eligibility logic preserves exclusion precedence over forced JDKs, and the checker handles the discussed tip and versionless-name contradictions. Confidence is limited to static inspection because Gradle tests were not executed.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 63d8dd7

@bric3 bric3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about replacing "contradictory" with "unsatisfiable" which looks more precise in this context ?

My comments concern regression coverage for forceJdk=[tip] although don't seem blocking.

Running this check, exposed two projects

  • profiling-controller-oracle, minimum is Java 11, but only ORACLE8 was allowed ; fixed by #13185
  • ignite-2.0, latest dep tests had minimum of Java 11 but a maximum Java 8, actually the same reported in check_inst 3/4

Pre-approving

Comment thread docs/how_to_work_with_gradle.md Resolved
@PerfectSlayer PerfectSlayer changed the title Check test JVM constraints for contradictions Check that test JVM constraints are satisfiable Nov 4, 2026
@PerfectSlayer

Copy link
Copy Markdown
Collaborator Author

Thanks @bric3. Switched to "unsatisfiable" in 5892d90: function name, task description, docs and PR title.

Comment thread docs/how_to_work_with_gradle.md Resolved
@PerfectSlayer

PerfectSlayer commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator Author

Great, thanks for the feedback.
Waiting for #13185 #13207 #13206 to be merged first :)

PerfectSlayer and others added 3 commits November 5, 2026 14:10
A test task whose testJvmConstraints no JVM can satisfy is skipped on
every JVM while the build passes, so its tests silently stop running.
Add a checkTestJvmConstraints task, run by check, that fails on such
constraints. It shares the eligibility rules with the runtime onlyIf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An includeJdk entry without a version was always counted as allowed, even
with an empty version range. It is now assumed to be within the range,
so it only passes when the range is not empty or it is forced. Entries
named tip never match at runtime, because tip is replaced by its version
first, so they no longer count as allowed or forced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Call the rejected constraints unsatisfiable, move the check rules into
the testJvmConstraints property table (including that includeJdk is only
read from test tasks), and cover a forced tip entry with an empty range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PerfectSlayer
PerfectSlayer force-pushed the bbujon/test-jvm-constraints-check branch from 5892d90 to 42db38d Compare November 5, 2026 13:12
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 69c1aab into master Nov 5, 2026
683 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the bbujon/test-jvm-constraints-check branch November 5, 2026 15:06
@github-actions github-actions Bot added this to the 1.69.0 milestone Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants