Skip to content

PROF-16527: fix inverted jniExceptionCheck() guards in HeapUsage::initJMXUsage() - #947

Open
zhengyu123 wants to merge 5 commits into
mainfrom
zgu/jvmtiEventTimingTest
Open

zhengyu123 wants to merge 5 commits into
mainfrom
zgu/jvmtiEventTimingTest

Conversation

@zhengyu123

@zhengyu123 zhengyu123 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

jniExceptionCheck() returns true when an exception was pending, but every guard in initJMXUsage() returned on false, i.e. whenever the JNI call succeeded. initJMXUsage() therefore returned right after its first FindClass and never reached the JMX calls.

Fixing the guards would make VMInit's initJMXUsage() call actually bootstrap JMX (ManagementFactory, getMemoryMXBean, getHeapMemoryUsage) before main(). Nothing reads that result: HeapUsage::get(true) takes the JMX branch only when VMStructs cannot read the heap, its sole caller uses it only when VMStructs can, and GC callbacks pass allow_jmx=false. So VMInit no longer calls it at all.

  • Invert the guards in initJMXUsage().
  • Drop the initJMXUsage() call from VMInit, so startup never loads JMX.
  • initJMXUsage() releases its JNI local refs on every exit path (non-copyable LocalRefs guard), and returns without using up its one attempt when there is no JNIEnv.
  • Bound the delayed NativeMethodBind list (512 pairs, previously no bounds check).
  • heapUsage_ut drives initJMXUsage() through a fake JNIEnv: success, a throw or a null at each step, no leaked local refs, and the null-JNIEnv case.
  • JvmtiEventTimingTest.agentStartupDoesNotBootstrapJmx checks getMemoryUsage0 is not bound at startup on JVMs that intercept native bindings (skipped elsewhere).

What does this PR do?:
Fixes the inverted exception checks in HeapUsage::initJMXUsage() and stops the agent from bootstrapping JMX at VMInit.

Motivation:
PROF-16527. With the guards fixed, VMInit would have loaded JMX on every JVM that needs native-binding interception (JDK 8 HotSpot, musl JDK 11, Zing) for a value nothing uses. Early JMX initialization can also interfere with containers such as JBoss/WildFly that configure JMX themselves.

Additional Notes:

  • The JMX fallback in HeapUsage::get(), the NativeMethodBind interception and _memory_usage_func are unreachable and still add a small cost to every native method bind on JVMs that intercept. This is unchanged from main, and removing them is left for a follow-up.
  • Review: Sphinx multi-stream review over the full range found no Critical or High findings (2 Medium, 6 Low). The null-JNIEnv guard, LocalRefs copy and capacity safety and a stale comment were fixed. Follow-up findings about races, pending exceptions and GC-callback use in the lazy JMX path were not fixed, because that path is unreachable (see above). A /code-review of the fix commits found nothing further that was reachable.

How to test the change?:

  • ./.claude/commands/build-and-summarize gtestDebug_heapUsage_ut gtestRelease_heapUsage_ut: 4 tests, passing.
  • ./.claude/commands/build-and-summarize :ddprof-test:testDebug -Ptests=JvmtiEventTimingTest: asserts only on JVMs that intercept native bindings (e.g. musl JDK 11). It skips on JDKs that don't, including Zulu 8 on macOS.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-16527

Unsure? Have a question? Request a review!

🤖 Generated with Claude Code

…tJMXUsage()

jniExceptionCheck() returns true when an exception was pending, but every
guard in initJMXUsage() returned on false, i.e. whenever the JNI call
succeeded. The JMX heap-usage fallback never initialized and nothing called
MemoryImpl.getMemoryUsage0, so VMStructs::NativeMethodBind never captured it.

- Invert the guards.
- Call initJMXUsage() from VMInit only when the native binding is
  intercepted (HeapUsage::needsNativeBindingInterception()); with the
  guards fixed, every other JVM would otherwise start loading JMX at
  startup.
- Bound the delayed NativeMethodBind list (512 pairs, no bounds check).
- heapUsage_ut drives initJMXUsage() through a fake JNIEnv.
- JvmtiEventTimingTest.agentOnLoadSeesNativeBindingsBeforeFirstStart
  checks getMemoryUsage0 is bound where interception is needed (musl
  JDK 11).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervm7aw5e
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 18.1.3 (1ubuntu1)
Date:Fri Nov 6 21:15:51 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs2
Logic error
Dereference of null pointer1
Unused code
Dead initialization1

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Unused codeDead initializationreferenceChains.cppapplyOomRamp4971
Logic errorDereference of null pointerperfEvents_linux.cppunregisterThread72026

An agent that failed before printing the NativeMethodBind interception
marker left inTime true and the interception flag false, so the test
skipped. Require exit code 0 and one of the two markers, and skip only on
the explicit "disabled" marker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #42295008440 | Commit: ded52d9 | Duration: 20m 23s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-11-06 21:37:27 UTC

@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 9a354fe6

zhengyu123 and others added 3 commits November 6, 2026 15:48
… refs

With the jniExceptionCheck() guards fixed, VMInit's initJMXUsage() call
really ran ManagementFactory/getMemoryMXBean/getHeapMemoryUsage before
main(). Nothing reads the JMX heap usage: HeapUsage::get(true) only takes
the JMX branch when VMStructs can't read the heap, and its sole caller uses
it only when VMStructs can; GC callbacks pass allow_jmx=false. Drop the
VMInit call so startup never loads JMX.

- initJMXUsage() deletes its JNI local refs on every exit path.
- heapUsage_ut checks no local ref leaks on success, throw or null.
- JvmtiEventTimingTest now asserts getMemoryUsage0 is not bound at
  startup (agentStartupDoesNotBootstrapJmx).
- Correct the delayed-binding comment; mark the test-only friend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…NIEnv

- LocalRefs deletes copy operations and asserts on capacity overflow.
- initJMXUsage(nullptr) returns without consuming the once-only attempt.
- heapUsage_ut covers the null-JNIEnv case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…flags

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Reliability & Chaos Results

✅ All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/151141206

@zhengyu123
zhengyu123 marked this pull request as ready for review November 6, 2026 22:12
@zhengyu123
zhengyu123 requested a review from a team as a code owner November 6, 2026 22:12
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T22:15:48.915448Z 9a354fe Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Static inspection supports removing startup JMX initialization while preserving the heap-usage path used by current callers. Runtime behavior on JVMs requiring native-binding interception was not exercised.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 9a354fe

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9a354fe6af

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

_lock.lock();
if (delayed != NULL) {
// Past the capacity a binding is dropped rather than overflowing.
if (delayed != NULL && delayedCounter < MAX_DELAYED * 2) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Test the delayed-binding capacity boundary

When more than 512 bindings arrive before a JNIEnv is available, this guard prevents writes past the allocated array, but none of the added tests exercises NativeMethodBind or verifies that the 513th pair is dropped. Add a boundary test for the first rejected pair so a regression in this overflow protection is caught.

AGENTS.md reference: AGENTS.md:L381-L385

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant