Skip to content

Recognise glibc by its chunk layout instead of probing its header (PROF-16484) - #949

Open
rkennke wants to merge 1 commit into
mainfrom
rkennke/PROF-16484-malloc-header-probe
Open

rkennke wants to merge 1 commit into
mainfrom
rkennke/PROF-16484-malloc-header-probe

Conversation

@rkennke

@rkennke rkennke commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:

Makes MallocFootprint::headerBytes() independent of heap state on glibc.

  • Before: the header was the smallest address stride between 16 fresh 48-byte blocks, minus their usable size. That is exact only if two blocks are adjacent. With freed same-size chunks scattered between live ones (tcache, fastbins), none are; the derived header exceeds the sanity bound and the probe falls back to 0.
  • After: glibc is recognised by malloc_usable_size. A glibc chunk's usable size holds the request and is congruent to size_t modulo 2 * size_t (a free chunk too small to split is handed out whole, so it may be larger). Then the header is sizeof(size_t). Size-class allocators (jemalloc, tcmalloc, mimalloc), sanitizers and glibc's debug modes report other sizes and keep the stride probe, as does any non-glibc build.

Motivation:

PROF-16484.

Additional Notes:

  • probeHeaderBytes(), glibcChunks(), glibcUsable() and strideHeaderBytes() are public so tests can call them in a prepared heap state; production code still uses the cached headerBytes(). A gtest friend would pull gtest into a production header.
  • The recognition assumes malloc_usable_size belongs to the allocator in force; of() already relied on that.
  • Not changed, pre-existing:
    • glibc's own mmap'd large chunks carry a 16-byte header, so of() stays 8 bytes low for them;
    • the test's allocatorExposesOverhead() comment says sanitizers make the probe yield 0; under ASan the stride probe yields 16;
    • the 512 KB < 1 % assertion would fail on arm64 kernels with 16 or 64 KiB pages.

How to test the change?:

Check Result
gtestRelease_mallocFootprint_ut, gtestDebug_…, gtestAsan_… (x64) passed
FragmentedHeapStillYieldsGlibcHeader: after freeing every other one of 512 blocks, the stride probe alone misses the header, the full probe returns sizeof(size_t) passed; the precondition is reached on glibc
Mutation check: drop glibc recognition, glibc header 0, recognition always false, exact-size signature, no alignment check, no size check each fails a test
Run under preloaded jemalloc (ubuntu:24.04 container) glibc not recognised, header 0, glibc tests skip

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-16484

🤖 Generated with Claude Code

…OF-16484)

MallocFootprint::headerBytes() took the smallest address stride between 16
fresh 48-byte blocks as usable + header. That holds only if two blocks are
adjacent; with freed same-size chunks scattered between live ones (tcache,
fastbins), none are, the derived header exceeds the sanity bound, and the
probe fell back to 0. The NM_* overhead gauges then undercounted every
small allocation for the process lifetime, and mallocFootprint_ut flaked on
arm64 CI runs.

glibc is now recognised by malloc_usable_size: a glibc chunk's usable size
holds the request and is congruent to size_t modulo 2 * size_t (a free
chunk too small to split is handed out whole, so it may be larger).
Size-class allocators and sanitizers report a multiple of 2 * size_t for
the probe sizes 48, 96 and 112 and keep the stride probe, as does any
non-glibc build; the glibc header is then sizeof(size_t).

Tests: a fragmented heap where the stride probe alone misses the header
while the full probe returns sizeof(size_t); the layout rule on fixed
values, including an unsplit chunk; a plain glibc build (no preload, no
malloc checking, no sanitizer) is recognised.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rkennke
rkennke requested a review from a team as a code owner November 6, 2026 21:07
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T21:10:40.813759Z 3d57e13 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-us1-prod datadog-datadog-us1-prod Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Usable-size recognition survived a focused glibc fragmentation check, supporting the replacement of heap-dependent stride detection. The Google Test suite could not run because Google Test was unavailable.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 3d57e13

@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 3d57e13d

@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #42294125733 | Commit: 6606344 | Duration: 22m 47s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-11-06 21:46:49 UTC

@dd-octo-sts

dd-octo-sts Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervm7aw5e
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 18.1.3 (1ubuntu1)
Date:Fri Nov 6 21:46:47 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs2
Logic error
Dereference of null pointer1
Unused code
Dead initialization1

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Unused codeDead initializationreferenceChains.cppapplyOomRamp4971
Logic errorDereference of null pointerperfEvents_linux.cppunregisterThread72026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant