[alextwoods]

Motivation and Context

Verifying a change to a single module (checkstyle plus that module's unit tests) with the default build takes 25 s
to 8 min. Measured with mvn clean verify -pl :<module> (macOS, 14 cores, JDK 21, mean of 2 runs): utils 106 s,
sdk-core 161 s, s3 224 s, codegen-generated-classes-test 480 s. About 85% of that time is surefire. Surefire runs
with one fork, so test classes run one after another, followed by a second, separate TestNG provider run for the
reactive-streams TCK tests.

This PR is the first in a series that aims to make single-module verification about 10x faster. It adds an opt-in
way to run checkstyle and every unit test of a module with the test classes spread across parallel test JVMs. It
also removes two sources of per-build overhead that every build pays, CI included. CI keeps using the default build
and still runs every check and every test. Follow-up PRs fix the slow test classes that set each module's time once
the forks are parallel.

Modifications

1. Opt-in dev profile (root pom.xml)

The profile only sets properties:

	-P dev	default build (CI)
checkstyle	runs	runs
unit tests	all of them	all of them
surefire forks	forkCount=1C, reuseForks=true (one reused JVM per core; classes run in parallel)	unchanged (1 fork)
spotbugs, japicmp, javadoc, dependency:analyze, jacoco	skipped	run

Why it's faster. The forks run test classes in parallel. Each module's time drops from the sum of its test
classes to roughly its slowest class plus overhead. The skipped plugins cost 1 to 15 s per module and are about 25% of
the time for small modules such as aws-core and auth. The property names are the same ones the existing quick
profile already uses.

Why coverage doesn't drop.

• The profile is opt-in, and the default build is unchanged: without -P dev, help:evaluate reports forkCount,
  reuseForks and jacoco.skip as undefined.
• -P dev still runs checkstyle and every unit test. The skipped checks still run in CI and in the default build
  that contributors run before opening a PR.
• Each test JVM is still isolated, so static state doesn't leak across forks. JUnit's in-JVM parallel execution is
  not enabled: turning it on globally broke 34 tests in utils.

2. BlockHound is installed by an SDK TestExecutionListener

Problem. With forkCount > 1, surefire creates a JUnit Platform launcher in the Maven JVM to discover test
classes before it starts the forks. Reactor's blockhound-junit-platform listener installs BlockHound in its static
initializer, so discovery installed BlockHound into the Maven JVM. On JDK 13+ that fails unless Maven itself runs with
-XX:+AllowRedefinitionToAddDeleteMethods (reactor/BlockHound#33).

Change. BlockHoundInstallingTestExecutionListener (in test/service-test-utils, registered through
META-INF/services) calls BlockHound.install() in testPlanExecutionStarted. Discovery never executes a test plan,
so the listener installs BlockHound only in the JVMs that run tests, before the first test. Those JVMs already get the
flag from the existing jdk-13-plus argLine. services/pom.xml, test/stability-tests/pom.xml and
service-test-utils no longer depend on blockhound-junit-platform.

Why not put the flag in .mvn/jvm.config instead.

• CI builds on JDK 8 through 21, and older JVMs refuse to start with a -XX flag they don't recognize.
• The flag is deprecated.
• It would still instrument the Maven JVM itself.

Why coverage doesn't drop. This also changes default and CI builds. The same modules still get BlockHound, and it
is installed before any test runs. BlockHoundInstalledTest passes, and the s3 test counts are identical to before.

3. maven-surefire-plugin 3.1.2 → 3.5.6, one JUnit Platform provider

Change.

• Remove the two explicitly configured providers (surefire-junit-platform and surefire-testng), the
  junit=false property that only surefire-testng read, and sdk-core's redeclaration of the provider.
• Surefire now auto-detects the JUnit Platform provider.
• org.junit.support:testng-engine 1.1.0, a global test dependency next to reactive-streams-tck, runs the TestNG
  TCK classes on the JUnit Platform.

Why it's faster.

• The TCK tests run in the same test run, and in the same parallel forks, as the JUnit tests, not afterwards in a
  sequential second pass.
• Modules without TCK tests no longer start an empty TestNG fork, which costs about 0.6 s per module.
• Under -P dev this takes 7% to 14% off utils (44.2 → 38.0 s), auth (23.6 → 20.5 s), aws-core (19.8 → 17.6 s),
  sdk-core (37.0 → 33.2 s) and codegen (24.8 → 23.1 s).
• s3 measured +7% and the remaining modules -1% to -2%; both are within run-to-run noise.

Why coverage doesn't drop.

• In default builds, per-class run, failed and skipped counts are identical to 3.1.2 for utils, sdk-core, aws-core,
  auth, codegen, s3, sqs, codegen-generated-classes-test and netty-nio-client. The only exception is codegen's new
  test in section 4.
• <includes> and <excludes> are unchanged. testng-engine only claims classes with TestNG annotations, so no JUnit
  test runs under TestNG and no test runs twice in the default build.
• Over the whole repo, reported totals went from 20,059 to 21,600 tests. Two things account for all of the
  difference:
  • The 583 new equivalence tests from section 4.
  • 958 tests in 15 modules that 3.1.2 under-reported: the empty TestNG pass overwrote their JUnit XML with
    tests="0". The console output shows those tests ran before too.

Reporting changes to be aware of.

• For 21 classes that have a class-level @DisplayName, the XML classname is now the display name.
• @Nested results are written into the outer class's TEST-*.xml.
• Test counts are unchanged, but tools that map classname to source files (Sonar, CodeBuild report groups) will
  show different names for those classes.

Why 3.5.6 and not 3.6.0. 3.6.0 hits
SUREFIRE-3460. Its console capture fails when an agent attached
at runtime (BlockHound, Mockito inline) writes to stderr, so s3 and sqs could not run tests. maven-failsafe-plugin is
versioned separately and is unchanged.

4. UnusedImportRemover scans each generated file once (codegen)

Problem. JFR on s3 code generation put 79% of CPU samples in UnusedImportRemover. It compiled a regex per
import and scanned the whole file twice for each one.

Change. The class tokenizes each file once and checks every import against that token set. Anything it doesn't
handle falls back to the original regex logic. The commit includes a changelog entry.

Why it's faster. ec2 codegen:generate went from 42.5 s to 7.3 s (5.8x, mean of 2). This speeds up every build
that generates a large service, including CI. s3 is unchanged at about 14.6 s: it is bound by the global
JavaCodeFormatter lock instead, which is a follow-up.

Why coverage doesn't drop.

• Generated sources are byte-identical between the old and new generator (diff -r of generated sources, generated
  test sources and generated resources):

  module	protocol	.java files
  s3	rest-xml	719
  ec2	ec2	5,053
  sqs	json	181
  dynamodb	json	597
  sts	query	99
  lambda	rest-json	623
  cloudwatch	smithy-rpc-v2-cbor	383
  codegen-generated-classes-test	test models	2,528
  protocol-tests	test models	1,443

  The only differences were key order in 17 protocol-tests *-intermediate.json files. Jackson writes those files,
  not UnusedImportRemover, the parsed JSON is equal, and two runs of the old generator differ the same way.

• UnusedImportRemoverEquivalenceTest (583 cases) compares the new class against a verbatim copy of the old one. It
  covers static and wildcard imports, names in comments, strings and javadoc {@link}, qualified names, duplicates,
  and the same simple name in different packages. It passes on JDK 11, 17 and 21. That covers both the JDK 8–18 and
  the JDK 19+ behavior of \b in regexes.

5. Documentation

CONTRIBUTING.md and docs/GettingStarted.md document the dev loop:

mvn install -P quick -pl :<module> -am -T 1C   # once, and after pulling or changing a dependency
mvn verify -pl :<module> -P dev                # iterate: checkstyle + all unit tests, parallel forks

Results

Default build = mvn clean verify -pl :<m> on the base commit. -P dev = mvn verify -pl :<m> -P dev without
clean, after one earlier build. Both are the mean of 2 runs on macOS, 14 cores, JDK 21, with no other builds
running.

module	default build	-P dev	speedup
utils	105.6 s	35.7 s	3.0x
sdk-core	160.6 s	26.9 s	6.0x
aws-core	29.0 s	14.1 s	2.1x
auth	30.4 s	17.5 s	1.7x
codegen (includes the 583 new tests)	38.7 s	15.0 s	2.6x
s3	224.0 s	79.1 s	2.8x
sqs	62.8 s	43.0 s	1.5x
codegen-generated-classes-test	480.0 s	148.4 s (1)	3.2x

(1) One run. The other failed on the known flaky AsyncRetrySetupTest timeout.

These gains don't reach 10x because, once forks are parallel, each module's time is set by its slowest test classes,
for example CachedSupplierTest (28 s) and Sync/AsyncRetrySetupTest (about 120 s). The next PR fixes those, and it
also speeds up the default build.

Default builds take the same time as before, apart from the removed empty TestNG forks and faster codegen for large
services: utils 104.8 s, sdk-core 153.2 s, codegen 35.4 s, s3 220.2 s.

Testing

• Default build (mvn clean install -pl :<m>, every check: checkstyle, spotbugs, javadoc, japicmp, dependency
  analysis, jacoco) passes for utils, sdk-core, codegen, service-test-utils and s3.

• Test-count parity against 3.1.2, per class, for 9 modules (section 3), and over the whole repo with
  mvn -T 1C clean verify (checks skipped, -Dmaven.test.failure.ignore=true).

• Codegen equivalence for the modules in section 4. The ec2 output generated on JDK 17 also matches the JDK 21
  output.

• Other JDKs: -P dev on JDK 11 and 17 for utils and sdk-core gives the same counts. Fork mode works on JDK 11,
  17 and 21 without any Maven JVM flag.

• Behavior checks:

  • -Dtest=SimplePublisherTckTest selects a single TCK class, and -Dtest=CachedSupplierTest a single JUnit class.
  • -P quick and -DskipTests still skip tests.
  • -DforkCount=1C without -P dev works for sqs (service-test-utils,sqs) without any Maven JVM flag.

• Not verified locally:

  • JDK 8, including the Windows JDK 8 leg. The listener and testng-engine target Java 8, and BlockHound needs no
    flag before JDK 13.
  • The native-image leg.
  • The failsafe and integration-test profiles.

  These need a green CI run.

Known issues that exist without this change:

• ServerConnectivityErrorMessageTest.closeTimeHasCorrectMessage (netty-nio-client) fails intermittently, with
  and without forks.
• AppIdResolutionTest and AppIdUserAgentTest fail when AWS_SDK_UA_APP_ID is set in the environment.
• In fork mode only (-P dev), 16 @Nested tests in codegen-generated-classes-test run twice. The root pom's
  <excludes> replaces surefire's default **/*$*. Default builds are not affected. This adds runs; it doesn't
  remove any.
• A few timing-sensitive tests failed once each in fork mode and passed on re-run. They are listed in the follow-up
  PR.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist

• I have read the CONTRIBUTING document
• Local run of mvn install succeeds
• My code follows the code style of this project
• My change requires a change to the Javadoc documentation
• I have updated the Javadoc documentation accordingly
• I have added tests to cover my changes
• All new and existing tests passed
• I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
• My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

• I confirm that this pull request can be released under the Apache 2 license