Repository navigation
OPcache: CE-cache flag on a shared interned class name becomes visible before its map_ptr offset; FPM child SIGSEGV on a cold cache #24577
Description
Description
With OPcache shared memory and opcache.interned_strings_buffer > 0, a php-fpm child can use the class-entry (CE) cache of a shared interned class-name string after another child has set IS_STR_CLASS_NAME_MAP_PTR on it, but before that child has stored the map_ptr offset in gc.refcount. It can occur only while the cache is cold and two or more children compile at the same time.
Writer (PHP 8.3.36, zend_accel_pending_ce_cache_commit(), ext/opcache/zend_persist.c#L345-L346; same two lines in zend_accel_get_class_name_map_ptr() in 8.3.35, L336-L337):
GC_SET_REFCOUNT(type_name, (uint32_t)Z_LVAL_P(zv));
GC_ADD_FLAGS(type_name, IS_STR_CLASS_NAME_MAP_PTR);These are two plain stores with no barrier. The writer holds the SHM lock, but the reader takes no lock (Zend/zend_execute_API.c#L1151, macros Zend/zend_types.h#L861-L868):
if (ZSTR_HAS_CE_CACHE(name) && ZSTR_VALID_CE_CACHE(name)) {
ce_cache = GC_REFCOUNT(name);
ce = GET_CE_CACHE(ce_cache);
if (EXPECTED(ce)) return ce;The compilers emit the stores in the reverse order: flag first, then offset. Disassembly of the commit loop in opcache.so:
php:8.3.36-fpm-alpine3.24(GCC 15.2.0):orl $0x20,0x4(%rdx)thenmov %esi,(%rdx)(2 copies of the loop)php:8.3.36-fpm-bookworm(GCC 12.2.0):orl $0x20,0x4(%rdx)thenmov %eax,(%rdx)- 8.3.35 Alpine (GCC 15.2.0) and Debian 12 (GCC 12.2.0) builds of
zend_accel_get_class_name_map_ptr(): the same order
A reader that sees the flag but the old refcount gets refcount 2. That is the value OPcache gives every new SHM interned string (GC_SET_REFCOUNT(s, 2), ZendAccelerator.c L546). ZSTR_VALID_CE_CACHE() accepts it: (2-1)/8 = 0 < CG(map_ptr_last). GET_CE_CACHE(2) then reads 8 unaligned bytes at map_ptr_base + 2, a non-null garbage pointer, and uses it as a zend_class_entry *. Note that zend_accel_get_class_name_map_ptr() already avoids offsets <= 2 (while (ret <= 2)), so a refcount of 2 can never be a real offset.
Crash we analysed (PHP 8.3.35, core dump):
WARNING: [pool www] child 38 exited on signal 11 (SIGSEGV - core dumped) after 2.104541 seconds from start
#7 object_init_ex+8 mov 0x1c(%rsi),%eax rsi = 0x3800007f7ffe2770 (non-canonical)
#8 ZEND_NEW handler
#9 execute_ex
#10 zend_generator_resume
#12 execute_ex
#13 zend_execute
#14 zend_execute_scripts
#15 php_execute_script
(Frames above #7 were a third-party crash handler that re-raises the signal.)
- PHP position: Symfony
MicroKernelTrait::registerBundles(),yield new $class();. OplineZEND_FETCH_CLASS(CV$class) thenZEND_NEW. $classis a shared interned string in OPcache SHM.
| Value in the core | |
|---|---|
CG(map_ptr_real_base) |
0x7f7ffaebb070 |
CG(map_ptr_base) |
0x7f7ffaebb06f |
8 bytes at map_ptr_base + 2 |
0x3800007f7ffe2770 |
class entry passed to ZEND_NEW |
0x3800007f7ffe2770 |
string gc.refcount at dump time |
44105 (slot 5513) |
| string flags at dump time | 0x170 (interned, permanent, IS_STR_CLASS_NAME_MAP_PTR) |
CG(map_ptr_last) of the crashed process |
5513 |
So the lookup ran with refcount 2. Another process stored the real offset (44105) afterwards, for a slot one past this process's map_ptr_last.
Sequence (inferred from the values above):
- Child A persists a script that names the class and sets the flag on the shared string.
- Child B runs
new $classon the same string. It sees the flag and refcount 2, and gets a garbage class entry. - Child B crashes in
object_init_exwith a general protection fault (SIGSEGV,SI_KERNEL).
Steps to reproduce
Natural rate (stock build):
- Set up php-fpm with OPcache SHM and
interned_strings_buffer=8:pm=dynamic, 40 start servers, max 80. - Use a Symfony 5.4 app whose kernel boot instantiates classes by dynamic name.
- Start from a cold cache and send 40 concurrent first requests.
- Result: 1 crash in 4,763 cold starts (792,967 requests). In a Kubernetes environment with the same image and lower concurrency we saw about 1 crash per 531 pool starts on 8.3.35. We did not get a core there, so we cannot confirm it was the same race.
Amplified rate:
- Use a copy of
opcache.sowith only a delay (about 2,000pauseinstructions) inserted between the flag store and the offset store. - Result: a crash in 30 of 30 cold starts with
interned_strings_buffer=8, and in 0 of 180 withinterned_strings_buffer=0. - A source-level way to widen the window (not tried in this form): swap the two stores in the writer and add a short spin between them.
Resulted in this output:
WARNING: [pool www] child N exited on signal 11 (SIGSEGV - core dumped)
But I expected this output instead: no crash.
Workaround
opcache.interned_strings_buffer=0: no shared interned strings, so every string the writer marks belongs to the script being stored and is not yet visible to other processes. Measured cost in our app: no change in CPU per request, startup time or memory.
Suggested fix
Either:
- Publish the offset with release ordering before the flag (
ZEND_ATOMIC_FENCE/release store between the two writes, acquire on the reader), or - In
ZSTR_VALID_CE_CACHE(), also rejectGC_REFCOUNT(s) <= 2, since a real offset is always> 2.
On weakly ordered CPUs the hardware can also reorder the stores even when the compiler does not (not tested).
Notes
- We reproduced the crash on 8.3.35. On 8.3.36 the code moved to
zend_accel_pending_ce_cache_commit()(commit 3584a98), but the two stores and their compiled order are unchanged. We have not re-run the reproducer on 8.3.36. - The same writer lines are on
PHP-8.4andmaster(source only; compiled order not checked). - Related but different: PHP-FPM worker SIGSEGV:
run_time_cachemap_ptr offset beyondCG(map_ptr_last)when a class is reached through the CE cache #23637 / PR Extend map_ptr on CE cache hit like other SHM paths #23881 (CE-cache hit without map_ptr extend), PHP-FPM SIGSEGV with OPcache, an observer, and an extension loaded per pool #23355 (per-pool extensions), bug #81607. - Loaded extensions: opcache, redis 6.3.0, pdo_mysql, zip, gd, bcmath, sockets, and Datadog dd-trace-php 1.25.1 (tracer, profiler, appsec). The crash occurred with the tracer disabled and the profiler and appsec enabled. We have not tried without third-party extensions. The mechanism above does not need an observer.
- JIT off (
opcache.jit_buffer_size=0), no preload, no file cache,opcache.enable_cli=0.
Relevant ini at the time of the crash:
opcache.enable=1
opcache.memory_consumption=256
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=20000
opcache.validate_timestamps=0
opcache.huge_code_pages=1
PHP Version
PHP 8.3.35 (NTS, crash reproduced, core analysed). PHP 8.3.36 (same source order and same compiled order, not re-run).
Operating System
Alpine Linux 3.24 (musl 1.2.6), x86_64, official php:8.3-fpm-alpine3.24 Docker image, opcache built with docker-php-ext-install (GCC 15.2.0). The same store order is in php:8.3.36-fpm-bookworm (GCC 12.2.0).
Activity
Fix phpGH-24577: Invariant check and store ordering for interned clas…
Fix phpGH-24577: Invariant check and store ordering for interned clas…
Fix phpGH-24577: Reject uncommitted refcount in interned class CE-cache