Skip to content

feat: build a jemalloc jar alongside each released jar - #7635

Open
andygrove wants to merge 5 commits into
apache:mainfrom
andygrove:release-jemalloc-jars
Open

andygrove wants to merge 5 commits into
apache:mainfrom
andygrove:release-jemalloc-jars

Conversation

@andygrove

@andygrove andygrove commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Which issue does this PR close?

Part of #7344.

Rationale for this change

#7356 added a jemalloc Maven profile, but the published jars still use the system allocator, which is glibc malloc on Linux. Users who want to try jemalloc in an executor, and anyone gathering the data that #7344 asks for, have to build Comet from source. Publishing a jemalloc jar with each release lets them swap the jar and compare. The jar with the system allocator stays the artifact, so nothing changes for existing users.

What changes are included in this PR?

  • dev/release/comet-rm/build-comet-native-libs.sh builds libcomet.so twice in the same container, with the system allocator and with the jemalloc Cargo feature, and fails if the two builds are identical. jemalloc is built for 4 KiB memory pages on both architectures, set explicitly with JEMALLOC_SYS_WITH_LG_PAGE=12 so that the library does not depend on the machine that built it. jemalloc fails every allocation on a kernel with larger pages, and some arm64 kernels use 64 KiB pages, for example RHEL 8 and its rebuilds. The release does not build for them, because a build for 64 KiB pages can raise memory use on the kernels with smaller pages, which are most of them. The user guide and the FAQ tell users on 64 KiB-page kernels to use the jar with the system allocator or to build from source with JEMALLOC_SYS_WITH_LG_PAGE=16.
  • dev/release/build-release-comet.sh copies both sets of libraries out of the containers and builds every Spark and Scala combination twice. The jars with the system allocator are installed as before. The script then runs clean, packages the jemalloc libraries with -P jemalloc, and copies each jar into the staging repository with the jemalloc classifier, for example comet-spark-spark4.1_2.13-1.2.0-jemalloc.jar. These builds run package rather than install, because installing would replace the jars with the system allocator. The jemalloc jar has no pom or sources jar of its own.
  • The script checks that each jar contains the native libraries built for its variant, which catches a jar that kept the libraries of an earlier incremental build.
  • publish-to-maven.sh is unchanged: it already uploads and signs every .jar and .pom under the staging repository, which includes the classifier jar.
  • release_process.md, the script usage text, source.md and the FAQ describe the two jars and the limitation on 64 KiB-page kernels.

publish_snapshot.yml is not changed, so snapshots still publish only the jar with the system allocator.

How are these changes tested?

The Docker half of the release script takes hours and has not been run. The shell scripts pass bash -n. For the Maven half, I built spark-4.1 with stand-in libraries on a throwaway checkout: a default package, then clean, then a -P jemalloc package. The jar name matches what the script copies, clean removes the stale libraries, and the jemalloc jar holds the jemalloc libraries. I also ran check_native_libs against that jar. It passes for the matching variant and fails for the other one. The first release candidate built with these scripts is the end-to-end test.

build-release-comet.sh now builds the native library twice per architecture,
with the system allocator and with jemalloc, and packages every jar twice.
The jar with the system allocator stays the artifact. The jemalloc jar is
installed next to it with the jemalloc classifier, for example
comet-spark-spark4.1_2.13-1.2.0-jemalloc.jar, so publish-to-maven.sh uploads
it without changes.

The jemalloc jars are packaged after a clean build rather than installed,
because installing would replace the jars that use the system allocator. The
script checks that every jar holds the native libraries built for its variant.
The arm64 jemalloc library is built for 64 KiB pages so that it loads on every
arm64 kernel.
@github-actions github-actions Bot added the enhancement New feature or request label Nov 15, 2026
@andygrove
andygrove requested a review from comphead November 15, 2026 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant