Skip to content

Spark 4.1: Add JMH benchmark for deeply nested Parquet projection - #18985

Open
jzhuge wants to merge 2 commits into
apache:mainfrom
jzhuge:spark-deep-nested-read-benchmark
Open

jzhuge wants to merge 2 commits into
apache:mainfrom
jzhuge:spark-deep-nested-read-benchmark

Conversation

@jzhuge

@jzhuge jzhuge commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Adds a JMH benchmark that reads single leaves at depth 1-3 of a wide nested struct, plus struct+child and whole-struct selections, and reports bytes read per op as a JMH counter. Before #18336 a deep leaf read its whole intermediate struct; this makes that visible, and it matters because local timing barely moves when the extra columns are fetched but not decoded. The #18336 results are in that PR.

Results (Spark 4.1, JDK 17, local FS, 10 files x 100k rows, 690 MB, SingleShotTime, 5 measured invocations; cd3ae3c80 before #18336 vs 843d36de after):

Arm Projection MB read per op s per op
id id 1.6 to 1.6 0.118 to 0.123
depth 1 person.name 36.2 to 36.1 0.257 to 0.274
depth 2 person.home_address.city 143.0 to 36.1 (3.96x) 0.294 to 0.261
depth 3 person.l2.l3.leaf 263.6 to 1.2 (222x) 0.256 to 0.179 (1.4x)
struct + child person.l2, person.l2.x 263.6 to 263.6 1.128 to 1.147
whole struct person 584.1 to 584.1 2.483 to 2.491

Timing is only comparable within a run pair: the same commit drifted 24% on the slowest arm (2.48 to 3.08 s/op) between two runs, while its bytes moved 0.3%. Anything smaller than that drift has to be argued in bytes, not seconds.

Reading it: pruned arms are compared against same-type baselines, so readDepth3Leaf (a long at depth 3) is expected to cost about what readId (a long at the top level) costs, and readDepth1Leaf about what readDepth2Leaf does; every leaf is orders of magnitude below readWholeStruct. Before the fix, the depth-3 leaf read exactly what person.l2, person.l2.x reads, which is the whole l2 subtree.

bytes is a @AuxCounters(AuxCounters.Type.EVENTS) column, so it appears next to the timing in the JMH results file and excludes warmup. Each measured invocation resets it and reads one op, so the score is the sum over the five measured invocations and scorePercentiles carries the per-op bytes. Type.OPERATIONS is not usable here: in SingleShotTime mode it drops the column entirely. The value is read from /proc/self/io rchar, which counts read syscalls rather than Hadoop FS statistics (those miss Parquet reads entirely); it is Linux-only, and elsewhere the counter logs once and reports nothing. A per-op floor fails the run when the counter sees an implausibly small read, so a blind counter cannot be mistaken for a pruned read.

Test plan: ran the benchmark on main and on the parent of 843d36de, and re-ran it on this branch.


AI Disclosure

Measures reads of single leaves at depth 1-3 of a wide nested struct
against struct and whole-struct selections, and logs bytes read per
op so pruning regressions show up even when local timing does not.

Generated-by: Claude Code (Claude Opus 5.5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bytes per op only reached the log, so the JMH results file carried timing
alone and the number averaged the warmup runs in. Pass an @AuxCounters
state to each arm so the value lands in the results, warn once when
/proc/self/io is unreadable, and fail the run when an arm records
implausibly few bytes.

State the pruning oracle in type-matched terms while here: a depth-3 long
leaf against the top-level long, not against a 64-char string. Drop the
fork's --add-opens, which every arm runs without on JDK 17.

Generated-by: Claude Code (Claude Opus 5.5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant