[IGNORE] performance/flaky-benchmarks-monitoring by igoragoli · Pull Request #12580 · DataDog/dd-trace-java

igoragoli · 2026-09-20T15:48:08Z

Automation branch for flaky benchmarks monitoring. See docs.

dd-octo-sts · 2026-09-20T17:10:49Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass
Load	🟢 pass
DaCapo	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Variant	Metric	Candidate	master	Δ (95% CI of mean)
dacapo:biojava:tracing	datadog	execution_time	14.68 s	14.70 s	[-1.0%; +0.7%] (no difference)
dacapo:tomcat:tracing	datadog	execution_time	1.66 ms	1.67 ms	[-1.6%; +1.3%] (no difference)
startup:insecure-bank:iast:Agent	datadog	execution_time	14.79 s	14.66 s	[+0.0%; +1.7%] (maybe worse)
startup:insecure-bank:tracing:Agent	datadog	execution_time	13.63 s	13.56 s	[-0.1%; +1.2%] (no difference)
webserver--insecure-bank--tracing--high_load	load	agg_http_req_duration_p50	806.69 µs	796.25 µs	[-1.1%; +3.7%] (no difference)
startup:petclinic:appsec:Agent	datadog	execution_time	17.15 s	16.96 s	[+0.2%; +2.0%] (maybe worse)
startup:petclinic:iast:Agent	datadog	execution_time	17.03 s	17.06 s	[-1.0%; +0.6%] (no difference)
startup:petclinic:profiling:Agent	datadog	execution_time	16.76 s	16.78 s	[-0.8%; +0.5%] (no difference)
startup:petclinic:sca:Agent	datadog	execution_time	17.08 s	16.98 s	[-0.2%; +1.4%] (no difference)
startup:petclinic:tracing:Agent	datadog	execution_time	15.76 s	16.17 s	[-6.7%; +1.6%] (no difference)
webserver--spring-petclinic--sca--high_load	load	agg_http_req_duration_p50	25.85 ms	25.18 ms	[-2.6%; +8.0%] (unstable)
webserver--spring-petclinic--tracing--high_load	load	agg_http_req_duration_p50	24.92 ms	28.13 ms	[-17.8%; -5.1%] (unstable)

Commit: aecd5f87 · CI Pipeline · Benchmarking Platform UI

pr-commenter · 2026-09-20T17:27:51Z

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	performance/flaky-benchmarks-monitoring
git_commit_date	1790669113	1790674524
git_commit_sha	`3901ee2`	`c3eeec7`

See matching parameters

	Baseline	Candidate
ci_job_date	1790678183	1790678183
ci_job_id	2088895948	2088895948
ci_pipeline_id	140857403	140857403
cpu_model	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
jdkVersion	11.0.31	11.0.31
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost	*.localhost
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ugqogpe0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ugqogpe0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu	11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	unsure [-7601.862op/s; -269.687op/s] or [-4.258%; -0.151%]
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

pr-commenter · 2026-09-20T17:40:17Z

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	performance/flaky-benchmarks-monitoring
git_commit_date	1790669113	1790674524
git_commit_sha	`3901ee2`	`c3eeec7`

See matching parameters

	Baseline	Candidate
ci_job_date	1790678155	1790678155
ci_job_id	2088895949	2088895949
ci_pipeline_id	140857403	140857403
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.31	11.0.31
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost	*.localhost
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-6ltgxhxi 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-6ltgxhxi 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu	11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	unsure [+334.976op/s; +15755.119op/s] or [+0.111%; +5.216%]
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

pr-commenter · 2026-09-21T22:08:48Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1793578115	1793578441
end_time	2026-11-02T00:10:02	2026-11-02T00:15:29
git_branch	master	performance/flaky-benchmarks-monitoring
git_commit_sha	`3ab1b20`	`aecd5f8`
start_time	2026-11-02T00:08:36	2026-11-02T00:14:02

See matching parameters

	Baseline	Candidate
ci_job_id	2199874864	2199874864
ci_pipeline_id	149265110	149265110
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1793280422	1793280422

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-27.150µs; +43.363µs] or [-8.573%; +13.693%]	unstable [-36.123µs; +58.723µs] or [-10.336%; +16.803%]	unstable [-45.497µs; +70.080µs] or [-12.497%; +19.249%]	unstable [-144.092µs; +118.294µs] or [-12.746%; +10.464%]	same
scenario:basic	same	same	same	unstable [-164.118µs; +34.071µs] or [-14.434%; +2.997%]	unstable [-180.413op/s; +74.704op/s] or [-7.758%; +3.212%]
scenario:loop	same	same	same	same	same

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (349.474 µs) : 310, 389
.   : milestone, 349,
basic (327.893 µs) : 323, 332
.   : milestone, 328,
loop (8.093 ms) : 8029, 8156
.   : milestone, 8093,
section candidate
noprobe (360.774 µs) : 310, 411
.   : milestone, 361,
basic (329.212 µs) : 324, 335
.   : milestone, 329,
loop (8.1 ms) : 8037, 8163
.   : milestone, 8100,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	349.474 µs [310.43 µs, 388.518 µs]
basic	327.893 µs [323.361 µs, 332.425 µs]
loop	8.093 ms [8.029 ms, 8.156 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	360.774 µs [310.409 µs, 411.139 µs]
basic	329.212 µs [323.798 µs, 334.626 µs]
loop	8.1 ms [8.037 ms, 8.163 ms]

Generated by stability-generate-patch from benchmarking-platform-tools patch-specs/dd-trace-java.yml. Base: master @ abe07de. Do not edit by hand — this branch is force-pushed on each refresh.

igoragoli force-pushed the performance/flaky-benchmarks-monitoring branch from 88fbaea to 623d388 Compare September 20, 2026 16:04

igoragoli force-pushed the performance/flaky-benchmarks-monitoring branch from 623d388 to 054c861 Compare September 21, 2026 21:48

igoragoli force-pushed the performance/flaky-benchmarks-monitoring branch 2 times, most recently from 5955793 to e9bad03 Compare September 21, 2026 23:32

igoragoli changed the title ~~performance/flaky-benchmarks-monitoring~~ [IGNORE] performance/flaky-benchmarks-monitoring Sep 22, 2026

igoragoli force-pushed the performance/flaky-benchmarks-monitoring branch 5 times, most recently from c3eeec7 to f2a8ff6 Compare September 29, 2026 13:30

igoragoli force-pushed the performance/flaky-benchmarks-monitoring branch 3 times, most recently from c95d92d to 5b33e0e Compare September 30, 2026 19:17

ci: gate pipeline to benchmark jobs for stability monitoring

aecd5f8

Generated by stability-generate-patch from benchmarking-platform-tools patch-specs/dd-trace-java.yml. Base: master @ abe07de. Do not edit by hand — this branch is force-pushed on each refresh.

igoragoli force-pushed the performance/flaky-benchmarks-monitoring branch from 5b33e0e to aecd5f8 Compare October 29, 2026 13:27

Navigation Menu

Repository navigation

[IGNORE] performance/flaky-benchmarks-monitoring - #12580

[IGNORE] performance/flaky-benchmarks-monitoring#12580
igoragoli wants to merge 1 commit into
masterfrom
performance/flaky-benchmarks-monitoring

igoragoli commented Sep 20, 2026 •

edited

Loading

Uh oh!

dd-octo-sts Bot commented Sep 20, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Sep 20, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Sep 20, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Sep 21, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

igoragoli commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

pr-commenter Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Kafka / producer-benchmark

Parameters

Summary

Uh oh!

pr-commenter Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Kafka / consumer-benchmark

Parameters

Summary

Uh oh!

pr-commenter Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Debugger benchmarks

Parameters

Summary

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

igoragoli commented Sep 20, 2026 •

edited

Loading

dd-octo-sts Bot commented Sep 20, 2026 •

edited

Loading

pr-commenter Bot commented Sep 20, 2026 •

edited

Loading

pr-commenter Bot commented Sep 20, 2026 •

edited

Loading

pr-commenter Bot commented Sep 21, 2026 •

edited

Loading