Repository navigation
Run the forked tests of every test suite - #13218
Conversation
PerfectSlayer
added
type: bug fix
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Wrong button, I wanted to check the merge conflict, not to open for review 🙏 |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd0dd5d89d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![datadog-official[bot]](https://avatars.githubusercontent.com/in/1358509?s=60&v=4){kind=small}
There was a problem hiding this comment.
Jetty 10’s latest-dependency forked tests now combine SLF4J 2.0.13 with Logback 1.2.13, silently disabling their diagnostic logging.
🤖 Bits Code Review · Commit bd0dd5d
06f7014 to
796a8dd
Compare
Test suites created by the test-suites.gradle helpers had no forked task unless the module opted in, so their *ForkedTest classes never ran. The helpers now register the forked task automatically. It reuses the suite classes and inherits the suite test JVM constraints. Remove the duplicate forked suites and the opt-in addForkedTestTask calls this makes redundant, and drop their configurations from the lockfiles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skip the latest dependency forked tests of openai-java-3.0 (APMLP-2192) and mongo-driver-3.6 (APMLP-2193) until they are fixed. The jdbc old driver suites predate the forked tests and only check the regular tests against old drivers, so they don't run the forked tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skip them until the AppSec redirect analysis tests pass with the latest OkHttp 3 (APMLP-2194). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skip them until thread registration works with the latest Netty (APMLP-2195). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The forked tests used to force SLF4J 1.7 on their own runtime classpath, which they now share with latestDepTest. Force it there, so Logback 1.2 binds and both suites log again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
796a8dd to
26adffb
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
More details
Companion tasks preserve suite classpaths and inherit JVM constraints, while known latest-dependency failures remain explicitly opted out. CI execution and the expected runtime increase remain unverified.
🤖 Bits Code Review · Commit 46ecbac
A suite can force its Java launcher, like the armeria Jetty 11 suites do. Its forked companion now uses the same launcher. It is set rather than used as a convention: the test JVM constraints plugin already sets the launcher of every Test task when testJvm selects another JVM than the Gradle daemon one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The latest dependency suites no longer get finalized by their forked companion. CI already runs every forked task through allLatestDepTests, and the default test task is not finalized by forkedTest either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bric3
left a comment
bric3
left a comment
There was a problem hiding this comment.
Thanks for this, it paves the way to a convention plugin!
What Does This Do
Every test suite created by the
gradle/test-suites.gradlehelpers now gets a forked test task, so its*ForkedTestclasses run in CI. The task reuses the suite's compiled classes and runs on the same JVMs as the suite.This activates 85 suites whose forked tests never ran: 75 latest-dep suites and 10 version-variant suites, with about 300 forked test classes.
Motivation
A suite only got a forked task when its module opted in. 82 suites did, by declaring a duplicate forked suite or calling
addForkedTestTask. 85 others didn't, so their forked tests were silently skipped while the job stayed green. For example,jedis-4.0,hazelcast-3.x/4.0andjavax-servlet-2.2ran zero latest-dep tests.Registering the forked task by default matches the common case. A suite with
*ForkedTestclasses almost always means to run them. With a default, new suites get their forked tests run without anyone having to know about an opt-in, so this coverage hole can't quietly come back. Not running them becomes the exception: an explicit opt-out that a comment or a ticket has to justify. There are 11 today.Running the forked tests for the first time surfaced real failures. This PR skips them, and each has a ticket:
openai-java-3.0LLM Obs tests fail against the latest OpenAI SDK, which requires an API key or workload identity.mongo-driver-3.6doesn't inject the DBMcommentfield with the latest 3.x driver (8 tests). This may be a real product gap.okhttp-3.0AppSec redirect analysis fails against the latest OkHttp 3 (5 tests): the redirected request is never captured.enable-wallclock-profilingnever attaches the profiling context with the latest Netty.Additional Notes
Testreplaced byForkedTest(latestDepTest→latestDepForkedTest). This keeps the existing names, so most test history is preserved.falseon the suite helper:javax-jms-1.1andscala-promise-2.10/2.13declare their own forked suite over different sources.aws-java-sdk-2.2already kept its latest-dep forked tests disabled because the S3 tests break (APMLP-2196).openai-java-3.0(APMLP-2192),mongo-driver-3.6(APMLP-2193),okhttp-3.0(APMLP-2194) andenable-wallclock-profiling(APMLP-2195) fail against their latest dependencies and are skipped until fixed.oldH2TestandoldPostgresTestsuites predate the forked tests and only check the regular tests against old drivers.test_inst_latesttakes +20% total job time (14.1 h vs 11.8 h). It's mostly on the slot holdingjdbc, +11 min; the other slots gain 2 to 5 min.test_insttakes +13% (23.7 h vs 21.0 h on the 6 PR JVMs), mostly thelettuce-5.0variant suites (+12 min on their slot). The slowest job takes 52 min, under the 1-hour job timeout.Jira ticket: [APMLP-2188]
🤖 Generated with Claude Code