Skip to content

feat(io): NUMA-aware IO reactor placement and routing (IO split step 4) - #224

Draft
aminaramoon wants to merge 28 commits into
NVIDIA:mainfrom
aminaramoon:io/s4-numa-routing
Draft

aminaramoon wants to merge 28 commits into
NVIDIA:mainfrom
aminaramoon:io/s4-numa-routing

Conversation

@aminaramoon

@aminaramoon aminaramoon commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Draft · stacked PR 7 of the IO split of #216 (step 4). Base: #222 (REMOTE tier), which sits on #221, #220, #219 and then main.
Review only the 2 commits after d1baf92: aminaramoon/cucs@io/s3b-remote-space...io/s4-numa-routing

Stack (re-ordered): #209 (merged) → #218 (merged) → #227 (merged) → #219 (GDS read) → #220 (write API + DISK via io_context) → #221 (S3 writes) → #222 (REMOTE tier) → this (4: NUMA-aware reactor placement and routing for uring and cuFile).
Sirius companion: sirius-db/sirius#2406.

What

This ports the NUMA placement and routing of #216 (457ba8b) onto the stack.

Today a reactor pool is N interchangeable reactors. They all stage from the first HOST space, and their workers are not pinned. On a multi-socket, multi-GPU host, device IO therefore goes through the wrong socket's memory and CPUs.

A pool built with a topology index now places *_reactors_per_gpu reactors per GPU on that GPU's NUMA node:

  • each reactor stages from the HOST space of its GPU's NUMA node;
  • each worker is pinned to that node's CPUs;
  • device requests are routed to the reactors local to their GPU.
Commit Summary
feat(io): place reactors per GPU on their NUMA node and route device IO to them Adds reactor_placement, reactor_layout, numa_spill_policy, make_reactor_placements and pin_reactor_thread. Layout constructors on templated_ioctx and the uring, REST and cuFile ioctxs. Placement-aware reactors, NUMA-local REST staging, topology-aware registry and factories, and the config rename.
test(io): cover NUMA-aware reactor placement and routing Covers placement, pinning, staging, routing and spill, registry and factory pool sizes, validation, and the cuFile fan-out.

Routing:

  • A request goes to the GPU of its first device slice.
  • Its candidates are that GPU's local reactors: those on the same NUMA node, or those homed on the GPU when the node is unknown.
  • It spills to the whole pool only when the least-backlogged local reactor holds more than min_backlog_bytes (64 MiB) and more than ratio× (2.0) the least-backlogged remote one.
  • The whole pool is used for:
    • host-only requests;
    • GPUs with no placed reactors;
    • pools whose reactors are all local.
  • Within the candidates, selection is unchanged: the two least-backlogged reactors, with rotation breaking ties.
  • The cuFile vectored-read fan-out spans only the routable reactors.

Config rename (io_config), breaking for embedders:

Old New
uring_n_reactors{4} uring_reactors_per_gpu{1}
cufile_n_reactors{4} cufile_reactors_per_gpu{1}
— pin_reactor_threads{true} (new)
— reactor_spill (numa_spill_policy, new)

The counts are per GPU of the registry's topology index. ⚠️ Without a topology index, a pool has *_reactors_per_gpu reactors in total. That drops the defaults from 4 to 1 for uring and cuFile, and from 2 to 1 for REST. This includes make_disk_io_context, which receives no index. This is #216's design and is kept as is; raise the per-GPU count, or pass an index, to keep the old pool sizes.

The PTDS stand-in streams, null-stream pinning, write splitting and every other dispatch rule from the stack are unchanged.

Changes beyond #216 (review round)

An independent reviewer (Fable) found the port faithful to 457ba8b, with no regressions to PTDS/null streams, cuFile, REST, the write API, DISK or REMOTE. These findings were fixed and folded in, each with a regression test that fails without the fix:

Finding Fix
Major (reproduced, also present in #216): with pin_reactor_threads, a worker was pinned to the GPU's full CPU list, widening past a taskset/numactl -C mask. Under taskset -c 0-3, workers ran on 0-79. The mask is now the placement's CPUs intersected with the inherited affinity. If the intersection is empty, or the mask can't be read or set, the thread stays unpinned; this is logged as a warning once, then at debug level.
A layout whose reactors stage in different block sizes was accepted, while the pool reported only the first reactor's size. The prefetching cache relies on that one size. Such a layout now throws std::invalid_argument.
numa_spill_policy was not validated. validate_spill_policy(): ratio must be finite and ≥ 1. The registry checks it up front, and every layout-built pool checks it again.
Missing coverage: a started placed reactor really pins its worker; io_context_registry built with an index. Tests added.
The doc didn't say that an unplaced reactor in a placed pool is remote to every GPU. Documented.

Tests

  • pixi run test 10/10.
  • The targeted io filters all pass: [reactor_placement], [templated_ioctx], [uring], [cufile], [rest], [write], [remote], [registry].
  • So do the cucascade/cudf [disk], [remote] and [multi_gpu] filters.
  • Cudf-free build passes; 0 warnings; lint and cmake-format are clean.
  • The 11 [rest] skips are environmental: no kTLS module, and an external LIST endpoint.
  • Not tested here: a real multi-socket NUMA host. This machine has 4 GPUs and loopback networking only.

Rebase note (REST re-split)

Rebased onto the re-split stack. Per the owner's decision, REST reactors are not placed per GPU: rest_n_reactors stays a total (0 = auto from the NIC's IRQ steering, see #227) and REST reactors are pinned by the NIC IRQ plan, not by NUMA placement. The REST hunks of this PR (rest_reactors_per_gpu, the REST placement constructor and placement(), REST staging from a placement, the topology parameter of make_rest_ioctx_factory, the REST placement test) were dropped; pin_reactor_threads and reactor_spill apply to uring and cuFile only. include/cucascade/io/reactor_placement.hpp (GPU placement) and src/io/rest/reactor_placement.hpp (REST pool plan) cross-reference each other.

Rebase onto main (2026-11-06)

Rebased with the whole stack onto main 5489112: #218 was squash-merged as 394465c (with its 20 review-round commits that this stack did not have yet), followed by #212 (ORDERED/UNORDERED repositories) and #240 (pool construction cleanup, one-shot future terminal, list_entry.etag). It was then rebased again onto main a11ff66, which adds #236 (atomics.hpp includes what it uses) and #232 (PR label checker off), with no conflicts. No conflicts in this PR's commits.

Verified at this branch's tip (Release, sm_89, CUCASCADE_WARNINGS_AS_ERRORS=ON, libcudf 26.12 nightly (00a724), 2× RTX 6000 Ada): cucascade_tests 170 cases, cucascade_io_tests 765 (1 skipped: rest_ioctx over HTTPS with kTLS required fails without tls.ko, as this host has no tls.ko), cucascade_cudf_tests 336, all passing; the benchmarks build.

Rebase onto main (2026-11-13)

Rebased onto the rebased #222 (d1baf92, on main 3db0c56; previous head 2daac9a). Sirius companion pin: 7d97059. No conflicts; the 2 commits are unchanged.

Verified on 2× RTX 6000 Ada in the cuda-13-stable env (Release, -Werror, CUDA 13.2, gcc 15.3, libcudf 26.10.02): a full cmake --preset release build (libraries, tests and benchmarks) has 0 errors and ctest passes 11/11; the multi-GPU tests pass as for #222, and the placement and routing tests (22 cases, 466 assertions) pass. The pinned pre-commit hooks pass.

@copy-pr-bot

copy-pr-bot Bot commented Oct 29, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@mbrobbel

Copy link
Copy Markdown
Member

/ok to test 987a686

aminaramoon added a commit to aminaramoon/sirius that referenced this pull request Oct 31, 2026
Bumps cuCascade to aa09ce3 (NVIDIA/cuCascade#224), whose io registry takes a
topology index and homes each uring and cuFile reactor on a GPU: pinned staging
from that GPU's NUMA node, worker pinned to its local CPUs, and device reads
preferring the reactors of their GPU's node, spilling to the rest of the pool
when those are much busier (io_config::reactor_spill).

The scan manager passes its topology_index to the io registry, including the
throwaway registry of path-scoped S3 configs, and builds its mandatory cuFile
context from the same per-GPU reactor layout. A test with a two-GPU topology
checks that the shared and the path-scoped registries both build per-GPU uring
pools and a REST pool of exactly rest_n_reactors reactors (reactor count and
warmup connections), whatever the GPU count.

cuCascade's uring and cuFile reactor counts are per GPU now, so Sirius's are
too: uring_n_reactors / cufile_n_reactors become uring_reactors_per_gpu /
cufile_reactors_per_gpu. uring defaults to 1 per GPU (was 4 in total); cuFile
keeps 4, being single-GPU only. The old keys are rejected with an error naming
the rename. The REST count is not renamed: rest_n_reactors stays a total for
the process (0 = sized from the egress NIC's IRQs), because REST reactors follow
the NIC, not the GPUs, and a per-GPU count would multiply the connection pool
with the GPU count. Config, YAML fixtures, bench configs, tests and docs follow.
@mbrobbel

mbrobbel commented Nov 2, 2026

Copy link
Copy Markdown
Member

/ok to test aa09ce3

aminaramoon added a commit to aminaramoon/sirius that referenced this pull request Nov 6, 2026
Bumps cuCascade to aa09ce3 (NVIDIA/cuCascade#224), whose io registry takes a
topology index and homes each uring and cuFile reactor on a GPU: pinned staging
from that GPU's NUMA node, worker pinned to its local CPUs, and device reads
preferring the reactors of their GPU's node, spilling to the rest of the pool
when those are much busier (io_config::reactor_spill).

The scan manager passes its topology_index to the io registry, including the
throwaway registry of path-scoped S3 configs, and builds its mandatory cuFile
context from the same per-GPU reactor layout. A test with a two-GPU topology
checks that the shared and the path-scoped registries both build per-GPU uring
pools and a REST pool of exactly rest_n_reactors reactors (reactor count and
warmup connections), whatever the GPU count.

cuCascade's uring and cuFile reactor counts are per GPU now, so Sirius's are
too: uring_n_reactors / cufile_n_reactors become uring_reactors_per_gpu /
cufile_reactors_per_gpu. uring defaults to 1 per GPU (was 4 in total); cuFile
keeps 4, being single-GPU only. The old keys are rejected with an error naming
the rename. The REST count is not renamed: rest_n_reactors stays a total for
the process (0 = sized from the egress NIC's IRQs), because REST reactors follow
the NIC, not the GPUs, and a per-GPU count would multiply the connection pool
with the GPU count. Config, YAML fixtures, bench configs, tests and docs follow.
aminaramoon added a commit to aminaramoon/sirius that referenced this pull request Nov 6, 2026
Bumps cuCascade to aa09ce3 (NVIDIA/cuCascade#224), whose io registry takes a
topology index and homes each uring and cuFile reactor on a GPU: pinned staging
from that GPU's NUMA node, worker pinned to its local CPUs, and device reads
preferring the reactors of their GPU's node, spilling to the rest of the pool
when those are much busier (io_config::reactor_spill).

The scan manager passes its topology_index to the io registry, including the
throwaway registry of path-scoped S3 configs, and builds its mandatory cuFile
context from the same per-GPU reactor layout. A test with a two-GPU topology
checks that the shared and the path-scoped registries both build per-GPU uring
pools and a REST pool of exactly rest_n_reactors reactors (reactor count and
warmup connections), whatever the GPU count.

cuCascade's uring and cuFile reactor counts are per GPU now, so Sirius's are
too: uring_n_reactors / cufile_n_reactors become uring_reactors_per_gpu /
cufile_reactors_per_gpu. uring defaults to 1 per GPU (was 4 in total); cuFile
keeps 4, being single-GPU only. The old keys are rejected with an error naming
the rename. The REST count is not renamed: rest_n_reactors stays a total for
the process (0 = sized from the egress NIC's IRQs), because REST reactors follow
the NIC, not the GPUs, and a per-GPU count would multiply the connection pool
with the GPU count. Config, YAML fixtures, bench configs, tests and docs follow.
@aminaramoon
aminaramoon force-pushed the io/s4-numa-routing branch 2 times, most recently from 8617f58 to 2daac9a Compare November 12, 2026 16:47
mike-wendt pushed a commit that referenced this pull request Nov 12, 2026
…LS; drop libcurl and kvikIO (IO split step 0b) (#227)

> **Draft · stacked PR 2 of the IO split of #216 (step 0b).** Base:
`main`.
> **Review:** the single squashed commit on this branch.
>
> Stack (re-ordered): #209 (merged) → #218 (merged) → **this (0b:
io_uring REST read + TLS)** → #219 (GDS read) → #220 (write API + DISK
via io_context) → #221 (S3 writes) → #222 (REMOTE tier) → #224 (NUMA
placement).

## Why this PR exists, and why it moved
#221 used to carry the io_uring REST transport, the TLS layer, S3 writes
and the kvikIO removal, late in the stack. A scan-replay benchmark of
TPC-H SF1000 on S3 (`tpch_scan_bench`) showed the simpler S3 reactor
beating that design: all 22 queries 59 s vs 80 s, q1 io-only 15.8 vs 9.8
GiB/s, up to 36 GiB/s with `min_mss` and 8 pinned reactors. So the READ
half moves here, right after parity; S3 writes stay in #221.

## What

**Transport (`src/io/rest/net`).** One io_uring per reactor, one
keep-alive connection per slot; at most one primary + one cancel SQE per
slot; slot never reused before both CQEs are reaped. Connects over a
resolver thread's addresses (DNS never blocks the loop). Body bytes of a
data GET land **directly into the destination iovecs** (pinned staging,
host memory or `fs_cache` chunks); headers go through a scratch buffer.

**TLS (`src/io/tls`, `include/cucascade/io/tls`).** `tls::config`
(version bounds, `ktls = prefer | require | disable`, user-space RX via
a built-in record layer or OpenSSL, io-wq placement, trust-store
discovery, generation-safe session cache). After the OpenSSL handshake,
installs kTLS RX+TX when policy and kernel allow (plain
`SEND`/`RECVMSG`; tickets skipped; TLS 1.3 KeyUpdate re-keyed where the
kernel can, else retire + retry); otherwise the built-in user-space
record layer decrypts the ciphertext ring straight into the destination.
`ktls = require` fails with `errc::ktls_unavailable`.

**HTTP/1.1 codec (`src/io/rest/http`).** Zero-copy, allocation-free
response parser, chunked body decoder, `Content-Range` value parser,
per-field S3-focused dispatch table, iovec sink and a retry/reuse
classifier. Request-head writer emits a single contiguous SEND and
records field spans for the SigV4 signer. Chunked and close-delimited
GET bodies are reported as `unsupported_framing` instead of being
silently coalesced.

**Reactor and pool (`src/io/rest`).** Adaptive physical GETs (`backlog /
free connections` clamped to `[min_segment_bytes 4 MiB,
max_segment_bytes 16 MiB]`), retry heap with backoff, jitter and
Retry-After, bounded 403 re-signs, Content-Range validation, "server
ignored Range", short-read retry, demand before prefetch. Device
destinations stage through the HOST memory resource blocks and copy on
the slice's stream with the slice's device current; the slot is held
until the copy completes. Staging exhaustion **parks** (leases are
counted per reactor; a read fails with `out_of_memory` only when the
reactor holds no staging).

**Control plane** (HEAD, footer probe, LIST, `resolve_footer_objects`)
runs on the pool's own connections: least-backlogged reactor, at most
half of a reactor's connections while data waits; never on the caller's
thread (a private helper serves calls made before `start()` or from a
reactor thread).

**Wait modes.** `epoll_pwait2` over {eventfd, retry timerfd, ring fd}
(default) or `io_uring_submit_and_wait_timeout` (`wait_mode = uring`,
with `defer_taskrun` / `single_issuer`).

**Generic exchange.** An optional request body (iovecs or generator) and
`Expect: 100-continue` are already supported and tested against a
loopback echo. This is the hook #221 (S3 writes) builds on; no S3 write
semantics here.

### Pool behaviour: parity with the libcurl stack it replaces
Checked against the curl source (`conncache.c`, `cf-socket.c`): a pooled
connection is reused only if idle for less than `conn_max_age` (**20
s**, as `CURLOPT_MAXAGE_CONN`; #221 had silently moved this to 4 s) and
it looks alive (zero-timeout poll: input pending or HUP with no response
outstanding = dead). Dead or too-old connections are replaced **lazily,
one slot at a time**, when a request needs them: there is never a mass
reconnect. A request that dies on a reused connection before any
response byte is retried once for free on a **fresh** connection.
`warmup(bucket)` stays rate-limited to once per `conn_max_age` per
bucket and pre-opens up to `max_connections` per reactor with real `LIST
max-keys=0` exchanges on fresh connections. TCP keepalive 60/60 and
`TCP_NODELAY` on. curl's upkeep is a no-op for HTTP/1.1, so nothing is
sent while idle by default; `rest.upkeep_http1` (off) sends staggered
zero-key LIST exchanges on idle connections every `upkeep_interval`
instead.

### NIC-driven reactor sizing and pinning
`rest_n_reactors` is a **total** (never per GPU). With `0`, the egress
NIC (`rest.interface`, else the default route) is inspected: when its
queue IRQs are steered to a strict subset of the NIC's NUMA node's cores
(requested **and** effective masks narrow), the pool gets one reactor
per IRQ CPU, each pinned to it before it creates its ring and raised to
the highest priority the process may set (SCHED_FIFO if permitted, else
nice; never widening the inherited affinity). Otherwise: the configured
count (default 2), unpinned. The reactor's io-wq is kept off the pool's
IRQ CPUs. Startup log reports the plan; `planned_reactor_cpus()` exposes
it for test/introspection.

### `min_mtu`
With jumbo frames, some pool connections negotiate small segments, which
costs decryption time per byte. After the TCP connect, the effective MTU
= `TCP_MAXSEG` + 40/60 (+12 with TCP timestamps) is compared with
`rest.min_mtu`; too small → RST and reconnect on the next address, at
most `mtu_reject_max_attempts` (8) times per request, then accept
(`mtu_fallbacks`). Every close uses `SO_LINGER{on, 0}`
(`close_with_rst`, default on) so no socket lingers in TIME_WAIT /
FIN_WAIT. Stats: `mtu_rejects`, `mtu_fallbacks`, `mss_min`, `mss_max`.

### Removed
- **kvikIO backend** (`refactor(io)!`): config, backend, tests and docs;
the datasource registry has no catch-all any more (unclaimed paths
return "no backend").
- **libcurl**: no longer a dependency of the REST path.

## Config
Every #218 field keeps its name and default (`max_connections 64`,
`conn_max_age 20 s`, `upkeep_interval 15 s`, `request_timeout_s 30`,
stall detector 64 KiB/s × 30 s, `footer_probe_bytes 512 KiB`, list caps,
retries). `ca_bundle_path` / `tls_verify` fill the new `tls` block. New:
`tls`, `min/max_segment_bytes`, `receive_buffer_bytes`,
`max_recv_iovecs`, `recv_waitall`, `queue_depth`, `sqpoll`,
`single_issuer`, `defer_taskrun`, `register_ring_fd`,
`iowq_max_unbound/bound`, `wait_mode`, `copy_poll_interval`,
`hold_slot_during_copy`, `dns_ttl` / `accumulate` / `max_addresses`,
`connect_timeout`, `keepalive_*`, `so_rcvbuf` / `sndbuf`,
`busy_poll_us`, `tcp_user_timeout_ms`, `upkeep_http1`, `min_mtu`,
`mtu_reject_max_attempts`, `close_with_rst`, `fan_out`, `interface`,
`pin_to_irq_cpus`; `rest_n_reactors = 0` means auto.
`config::validate()` rejects invalid values (including
`max_retry_attempts` / `max_auth_retry_attempts` of 0, which used to be
clamped silently).

⚠️ API notes: `reactor_context` is no longer copyable (it holds a mutex;
`make_shared` users are unaffected); `rest_ioctx::stats()` is new;
libcurl is no longer a dependency.

## Behaviour differences from #221 that an embedder could notice
A cut body is retried whole, not resumed; a 200 for a sub-range fails as
"server ignored Range"; no link estimator/controller; the #221 fields
`keep_warm_connections`, `enable_controller`, `link_gbps`,
`ring_entries` (→ `queue_depth`), `tls_handshake_timeout` (→
`connect_timeout`), `write_split_bytes`, `expect_continue` do not exist
here (the last two return with #221).

## Review round
Two independent reviews (transport/TLS/pool;
scheduler/destinations/contract/NIC) with mandatory reproduction: 0
blockers. Fixed with regression tests: staging exhaustion parked instead
of failing (reproduced OOM on a 2 MiB pool); `upkeep_http1` busy-spin
while a GET was in flight (1.9 M passes/s → < 100); control jobs no
longer pinned to reactor 0 nor starving data (a footer batch delayed a 4
KiB read by 3 s → < 400 ms); `_inflight` leak on a throwing submit;
control call from a reactor thread deadlocked; stale retry now goes to a
fresh connection; CQEs reaped before deadline checks; kTLS KeyUpdate
reply `EWOULDBLOCK` no longer fatal; idle drain loops while readable;
pins re-intersected with the worker's affinity; pool plan kept per
ioctx. Verified clean by the reviewers: coordinator / `queued_bytes`
accounting, multi-GPU staging on 4 GPUs, the #218 public contract (base
REST tests byte-identical and passing), a 400-exchange fault-injection
run in all three ring modes.

## Tests
`pixi run test` 10 / 10; 262 REST/TLS cases (8 skip: kTLS without
`tls.ko`); 257 cudf cache tests through the REST backend; cudf-free
build links without curl; 0 compiler warnings; lint clean. Loopback
smoke (user-space TLS, 1 GiB in 16 MiB GETs, 2 reactors × 16
connections): 2.8–3.1 GiB/s. **Not tested here:** kTLS data path (no
`tls.ko`; unit-tested with fakes), real S3, jumbo-frame MSS negotiation
(the probe is tested through a seam), a NIC with steered IRQs (this
host's are not steered; fake sysfs trees cover both cases).

## Branch state
The 16 development commits have been **squashed into one** against the
PR base. Verified at the squashed tip (Release, `sm_89`,
`CUCASCADE_WARNINGS_AS_ERRORS=ON`, libcudf 26.12 nightly, 2× RTX 6000
Ada): `cucascade_tests` 166 cases, `cucascade_io_tests` 511 (1 skipped:
`rest_ioctx over HTTPS with kTLS required fails without tls.ko`, as this
host has no `tls.ko`), `cucascade_cudf_tests` 263, all passing; the
benchmarks build.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Add the private cufile_api dlopen loader (libcufile.so.0, RTLD_NODELETE,
read entry points only) and the process-wide cuFile platform probe
(probe_platform / gds_available).  The cuFile backend never links
libcufile: it needs only cufile.h at configure time.  find_path(cufile.h)
gates the sources, the tests and the propagated CUCASCADE_HAS_CUFILE
definition, independently of CUCASCADE_BUILD_CUDF, so a cudf-free build
keeps the backend.

The loader resolves read entry points only; write, buffer-registration and
stream entry points are left out.
Add cufile_reactor / cufile_ioctx, a read-only GPUDirect Storage backend:

- device slices of at least gds_threshold bytes of a registered file are
  read by synchronous cuFileRead calls of at most max_op_bytes (64 MiB),
  split on 4 KiB file boundaries with the EOF tail read on its own; short,
  zero and oversized returns are errors, a failing cuFileRead degrades to
  POSIX.  Destinations are never registered (cuFileBufRegister): every
  read passes the exact device pointer.
- everything else goes through POSIX: host-only reads O_DIRECT (staged
  through the scratch when unaligned), host+device pairs pread + H2D
  copy, device-only reads double-buffered through two 8 MiB pinned scratch
  slots on a reactor-owned non-blocking copy stream.
- a device slice whose stream still has queued work is parked behind an
  event instead of blocking the worker; a parked slice of a request that
  already failed settles at once instead of waiting for its stream, and
  shutdown cancels parked slices and returns the pinned scratch to a
  process-wide pool rather than calling cudaFreeHost (no device
  synchronization).
- a read whose H2D copy was enqueued but whose event could not be recorded
  waits for the copy stream before settling, so neither the destination
  nor the scratch is released under a running copy.
- strict priority with bounded low service, per-pass byte quantum.
- opens reopen the buffered fd for O_DIRECT through /proc/self/fd, check
  dev/inode before registering (retry once, then descriptor_mismatch_error)
  and carry the buffered fd's local_stat identity and size.
- cufile_ioctx fans a vectored read with more than two slices out over
  every reactor of its count-based pool.

Routing: io_backend::cufile and io_context_type::cufile;
io_config::{cufile_n_reactors, cufile} under CUCASCADE_HAS_CUFILE.  With
backend=cufile, local regular files resolve to cuFile and s3:// stays on
REST; native routing is unchanged, and an explicit cufile
request in a build without cuFile is refused instead of served by another
backend.  The reactor traits opt out of the prefetching cache.  Vectored
device-read signatures are unchanged; their completion contract is now
documented at ioctx::device_readv_async_io and
datasource::device_read_ranges_async.
Cover read scheduling: another stream's read and a host read are served
while a device slice waits for its held stream, shutdown cancels a parked
slice promptly on the cuFile and the staged path (with pooled scratch
already handed out), reads submitted before start are cancelled, and an
active low request is parked for a high one with strict or bounded low
service.

Add the hidden read-only throughput probe [.cufile-bench]
(CUCASCADE_IO_BENCH_FILE/BYTES/ROUNDS/REACTORS/BACKEND/ONLY/GDS_THRESHOLD).
It validates that the operator-supplied input exists and is large enough
and never creates, resizes or syncs it; the raw cuFile submission and
write probes stay out.
Add topology_discovery::discover_storage_for_path and
discover_storage_for_descriptor, returning a path_storage_info snapshot:
the opened file's resolved path and device numbers, its mount (statx mount
ID when the kernel reports it, else the longest component-boundary mount
point of the file's device, with bind mounts and shadowed mounts handled
and escaped mountinfo fields decoded), filesystem type / source / options,
and for block filesystems the leaf disks reached through partition parents
and device-mapper / md slaves (cycle-protected), each with drive class,
PCI bus ID and NUMA node.

Network and memory filesystems are classified without inventing devices;
overlay / FUSE / pseudo filesystems, unreadable procfs or sysfs and
unresolvable stacks are reported as unknown, which is distinct from
missing, non-regular and inaccessible paths.  Discovery is passive
(procfs / sysfs only, no CUDA, no cuFile, no writes), never throws and
caches nothing.  It lives in its own source of the topology-only library;
fixture procfs / sysfs roots are injected through a src-private header
used by the tests only.
Add cufile::probe_path(path, config) and probe_object(object, config), and
cufile_ioctx::path_capability(io_object) for objects it opened.  They
combine the effective compat / use_odirect policy, the process-wide
platform probe, passive storage discovery of the opened descriptor
(topology_discovery::discover_storage_for_descriptor, so the verdict
describes the file actually read) and the open's O_DIRECT / registration
outcome into a path_info with every piece of evidence and one verdict:
unavailable, compatibility_only, direct_candidate or unknown, with a
reason.

Only the dstatusflags bit of the file's own storage class counts (NVMe,
NVMe-oF, NFS, Lustre, WekaFS, GPFS, BeeGFS, VirtioFS), never any other
bit; ext4 (not data=journal) and xfs are the supported local
filesystems.  A direct candidate is not a DMA guarantee.  Unknown stacks
and failed property queries stay unknown; compat OFF refusals are
reported as unavailable with their cause.  probe_path is explicitly
active (it opens the cuFile driver and registers the file, reads nothing,
writes nothing); verdicts are logged once per device and verdict.

platform_info now distinguishes an opened driver whose properties could
not be read (properties_available) from a compatibility-only one;
gds_available() requires readable properties.
Document path_storage_status and the storage stack discovery that
cufile::probe_path builds its per-file GDS verdict on.
Add the ioctx write API: write_mode / write_open_options / write_slice,
open_io_object_for_write, synchronous host_write, asynchronous host / device
/ vectored / mixed writes, flush and flush_async, supports_write and
supports_device_write, and the backend hooks behind them (defaults report
std::errc::not_supported).  Asynchronous forms are noexcept and report every
error through the future.

templated_ioctx gains the optional io_write_reactor_c contract with separate
reactor_write_traits, so non-writing reactors keep compiling unchanged.  Read
and write dispatch share dispatch_slices; reads keep their batching, callback
and priority behaviour.  Write slices are validated (null, overlap, overflow,
past off_t), never clamped to EOF, and cut at granule-aligned multiples of the
reactor config's write_split_bytes.  io_op_request::finish_success raises a
writable object's size before the coordinator settles.

A write invalidates its path in the writing ioctx's fs_cache and metadata
store before dispatch and again before success or failure is visible (the
completion is fully allocated before the write is dispatched, so nothing can
fail between starting physical work and attaching the second pass); a
truncating or replacing open invalidates too.  fs_cache retires the path's
generations and never hands an invalidated generation out again; a
pending_invalidation holds an in-flight slot until it fires, so
shutdown_cache waits for it.  Retiring the generations is allocation-free and
happens first, so an allocation failure can only defer the eager reclaim of
their idle chunks, never leave a generation reusable.  metadata_store gains a fixed 256-slot
per-path-hash write epoch: objects are stamped at open, pre-write opens can
neither observe nor publish, loads never join across a write, and entries are
returned only to opens of the epoch they were published under.
invalidate_written retires every profile of the path; targeted invalidate
leaves the epoch alone.  Writable objects carry an opened_handle identity.
Add io::not_implemented_error, a std::system_error with
std::errc::not_supported whose message names the backend and operation and
says "not yet implemented".  The ioctx write front ends report it when the
backend cannot write at all (a host-only writer asked for a device source
still gets a plain not_supported), and so do the default write hooks.

The REST context stays read-only and reports it from explicit write, flush
and write-open hooks: synchronous hooks throw, asynchronous ones return a
failed future, with no network request.  The REST reactor gains static
write-open / flush and member host_write stubs, and its enqueue fails write
and flush slices instead of reading them as GETs.  Because the stubs satisfy
io_write_reactor_c structurally, reactor_write_traits<rest_reactor> is
specialized to false/false next to the reactor, before any context
instantiation.

Tests cover every entry point through ioctx&, the direct hooks and direct
reactor submission: exception type, code and message, no network side
effects, and zero-size no-ops.
…e read

start_device_copy enqueued the staged H2D copy and its completion event in
one call.  When copy_async failed part-way, or only the event record failed,
the slot was settled at once: its staging block went back to the pool and
the request resolved while the copy might still read the block and write the
caller's destination.

Record the event separately and, on any failure after the copy may have been
queued, synchronize the caller's stream before settling, as the cuFile
reactor already does.  A process-wide test seam
(uring::detail::set_copy_event_record_failure_for_testing) makes the record
fail; the regression holds the caller's stream so the copy cannot have run,
and checks the destination holds the file's bytes once the failed read
settled.
A read future resolves with its coordinator's bytes_requested, the bytes of
its slices as clamped at dispatch to the size recorded on the io_object.  That
count is the bytes delivered only because every reactor path fails a transfer
that comes up short of the clamped range instead of settling it as a success:
uring resubmits short reads and fails a zero-byte one with EIO (also when its
O_DIRECT range was widened), and the cuFile reactor fails a short cuFileRead,
staged pread or host pread with EIO.  Nothing pinned that contract.

Document it on grouped_coordinator and device_readv_async_io, and test it
for uring (O_DIRECT and buffered) and both cuFile read paths: reads running
past EOF resolve with exactly the bytes up to EOF (scalar and vector forms,
bytes checked), a read starting at EOF resolves 0, and reads of a file
truncated after it was opened fail with std::errc::io_error for device and
asynchronous host reads, while a part the file still holds reads in full and
the synchronous host read reports what pread delivered.
cudaStreamPerThread names the per-thread default stream of whichever thread
uses the handle.  The reactors interpret a device slice's stream on their
worker thread: the uring reactor enqueues the H2D copy of a device read (and
now the D2H copy of a staged write) on it, and the cuFile reactor records its
stream gate on it.  With cudaStreamPerThread both ordered the transfer after
the worker's own, empty, per-thread stream, so a read could land before work
the caller had queued ahead of it (and be overwritten by it), and a write
could read its source before the caller's producer ran.

templated_ioctx's dispatch now replaces every cudaStreamPerThread of a
request's device slices, on the submitting thread, by a non-blocking stand-in
stream (one per device and request) that waits there on an event recorded on
the caller's per-thread stream; the slice keeps the stand-in alive.  No
per-thread handle crosses threads any more.  The fs_cache's cache-hit device copies, enqueued on the caller's stream by the
submitting thread, were retired through a completion lane keyed by stream
handle; cudaStreamPerThread has one handle for every thread's stream, so one
thread's completion could retire another thread's copies (releasing pins and
resolving the read before its DMA finished).  A cache-hit read on the
per-thread stream is now retired by its own event, recorded on the submitting
thread after its copies.  The legacy default stream needs
nothing: it is one stream per device and synchronizes with every blocking
stream.

Regression tests hold the caller's per-thread (and, for coverage, legacy)
default stream on the host, queue a memset of the destination behind it and
check the device read's bytes, for the uring reactor and for both cuFile read
paths; a two-thread test holds one thread's per-thread stream under a cache-hit
read and checks another thread's cache-hit read neither retires nor waits for
it.
The io_uring reactor models io_write_reactor_c.  A write slice is split at
the write granule (max(4 KiB, page size)) into an unaligned head, an aligned
body and an unaligned tail: the body goes O_DIRECT straight from a congruent
host source or through the pinned staging blocks (a D2H copy on the caller's
stream for device sources, a memcpy for host ones), head and tail through
the page cache.  Writes use write / writev / write_fixed SQEs; short writes
are resubmitted, a zero-byte completion is EIO, and O_DIRECT or fixed-buffer
runtime errors fall back to buffered I/O as reads do.  Flush is an
IORING_OP_FSYNC with IORING_FSYNC_DATASYNC; the synchronous host_write and
flush run on the caller.  write_split_bytes (default 32 MiB) spreads large
slices over the selected reactors.

The worker never blocks on a caller's stream: pending D2H and H2D copies are
polled with a capped CQE wait, and staged-write bytes whose copy is pending
count in queued_bytes().  start() waits for the worker's ring setup and
rethrows its error (e.g. EPERM under seccomp); requests enqueued before or
after such a failure fail with that error, and start() may be retried.
Clean shutdown still cancels with operation_canceled.

Writable local objects grow atomically as writes complete and carry an
opened_handle identity.  The write open anchors its O_DIRECT descriptor to
the already-created buffered one through /proc/self/fd instead of reopening
the path, never repeats the destructive create / truncate, and fails with
descriptor_mismatch_error when a pathname fallback cannot prove both name the
same file; a test seam renames the path at that point.

Tests cover open modes, aligned / unaligned host and device scalar, vector
and mixed writes, holes and size growth, stream ordering, priorities,
concurrent block-sharing writers, shutdown and pre-start cancellation, the
mincore direct-write check, stalled streams, short writes, RLIMIT_FSIZE,
sibling errors, seccomp start-up failure, the anchored write open, and fs_cache
/ metadata invalidation by writes.
The cuFile reactor models io_write_reactor_c.  Write opens create / truncate
the buffered fd per write_mode, preallocate best-effort, and reopen the
O_DIRECT fd from it through /proc/self/fd (never by path; a fallback that
opens another file fails with descriptor_mismatch_error rather than repeat
the destructive open), then register it with cuFile under the usual compat
rules.  Writable objects grow atomically and carry an opened_handle identity.

Writes split at the write granule.  On a real GDS platform the aligned body
of a device source of at least gds_threshold goes to chunked synchronous
cuFileWrite after the slice passed its stream gate; head, tail, smaller bodies
and, on a compatibility-only platform, every device body are staged through
the double-buffered pinned scratch by D2H copies on the reactor's copy stream
and written with pwrite (O_DIRECT for aligned chunks).  Large host bodies
congruent with their offset go O_DIRECT from the caller; other host sources
go through the page cache with writeback started at once, and a host-only
vectored write goes to one reactor.  Short writes resubmit, zero-byte writes
are EIO, flush is fdatasync.  Staging copies drain before the scratch is
reused or a request settles, including when an event record fails after a
copy was queued.

cufile_api resolves cuFileWrite, cuFileBufRegister and cuFileBufDeregister.
config::register_min_bytes (default 0) opts into per-transfer buffer
registration through a process-wide, reference-counted table keyed by base
address.  Every non-empty cuFile read and write takes part in that table,
also with the knob at 0 (which only stops a context from creating
registrations), so a context that registers never covers memory another
context's unregistered transfer is using.  write_split_bytes defaults to 0 for cuFile.

Tests cover every GDS / staged / compat / unregistered / buffered write path,
writes ordered after the caller's per-thread and legacy default streams,
default and opt-in contexts sharing a buffer,
registration sharing and exclusion, caller stream ordering, progress of
unrelated host writes, shutdown cancellation, short and error paths and
8-byte-aligned host-block vectors.  The practices doc gains a writes section
(measured in compatibility mode only).
Pin null device streams to cudaStreamLegacy before dispatch, and gate cuFile on a recorded legacy-stream event instead of assuming cudaStreamQuery captures blocking-stream dependencies. Canonicalize cache-hit retirement lanes too.

Cover dispatch sentinels, null read/write streams and PTDS-built callers on uring and direct/staged cuFile paths. The deterministic enqueue seam fails without the fix; the normal legacy-built backend already passes the PTDS-caller test. An isolated PTDS-built cuFile backend reproduces early completion and stale writes without the fix (0ms, 0x11 written), and waits ~700ms with the fix (0xab written).
DISK memory spaces now read and write their files through a per-space
std::shared_ptr<io::ioctx> instead of the bespoke idisk_io_backend and its
pipeline worker.

- disk_memory_space_config gains a defaulted io_context, and
  reservation_manager_configurator::set_disk_mounting_point a defaulted
  fourth argument.  memory_space gets DISK-only, mutex-guarded
  get_io_context(), set_io_context() (non-null) and
  get_or_init_io_context(make_default); a failed or null factory installs
  nothing, and in-flight conversions keep their own reference.  The core only
  forward-declares ioctx: no core -> io link.
- cucascade/cudf/disk_io_context.hpp: make_disk_io_context() builds and
  starts an automatic / uring / cufile context (uring owns private portable
  pinned staging, 64 MiB per reactor), and default_disk_io_context() is the
  weakly cached, process-shared default adopted lazily on first conversion.
  automatic = cuFile with real GDS, else uring when this process may set up
  io_uring (also falling back when the reactors' own ring setup fails at
  start), else the cuFile reactor's POSIX path; forced uring fails with the
  setup errno.  The cuFile paths are gated on CUCASCADE_HAS_CUFILE.
- The four HOST/GPU <-> DISK converters open one object and issue one
  vectored request per table; the GPU read allocates the whole buffer tree
  before submission, waits once, and records the writer event.  Every
  allocation outlives its I/O on every unwind path; spill files keep their
  RAII removal and layout; no durability flush in the spill path.  The
  redundant post-read host stream sync is gone: a completed device read is
  host-observed.
- A cudf build now requires CUCASCADE_BUILD_IO (configure-time diagnostic);
  cudf-free and topology-only builds are unaffected.
- Removed: include/cucascade/data/{disk_io_backend,io_backend_registry}.hpp,
  src/data/{pipeline_io_backend.cpp,io_worker.hpp,io_backend_registry.cpp,
  io_backend_internal.hpp} and their tests.

AGENTS.md describes the io_context-based DISK tier.

BREAKING CHANGE: the custom-backend memory_space constructor and getter,
idisk_io_backend, io_batch_entry and the io backend registry are removed.
Three-field disk_memory_space_config aggregates and three-argument
set_disk_mounting_point calls still compile; pass an io::ioctx (or nothing,
for the shared default) instead of a backend.

Keep the source device current from wait_for_writer through the write: a null stream otherwise named two different default streams and the spill could read stale source bytes before its writer finished. Reject an explicit stream of another device with cucascade::logic_error before generating a file path.

Both regression cases ran on the 4xA100 host for uring and cuFile compatibility mode. Without this fix the null-stream test spills stale bytes on both backends; foreign streams yield a CUDA error on uring or no error on cuFile. With it, both cases pass (22 assertions).
- test_disk_randomized: randomized mixed / nested / empty tables with
  non-repeating data, round-tripped through HOST and GPU on the default
  context and on forced uring and cuFile contexts; only forced uring is
  skipped, and only where io_uring is proven unusable in this process.
- benchmark_disk_converter measures the DISK tier per io context with
  backend-neutral names (automatic, forced cufile, optional forced uring),
  CUCASCADE_BENCH_DISK_DIR, cold reads and an explicit benchmark-only
  fdatasync for comparable write numbers.  No throughput assertions.
…eries

The S3 write path needs PUT (object, UploadPart), POST
(CreateMultipartUpload, CompleteMultipartUpload) and DELETE
(AbortMultipartUpload) requests, most with a sub-resource query
(?uploads, ?partNumber=N&uploadId=U, ?uploadId=U).

- request_method gains PUT, POST and DELETE, plus to_string();
- request_spec describes one object-level request (method, pre-encoded
  canonical sub-resource query, whether it has a body);
- request_authorizer::authorize_request(spec, timeout): a new virtual whose
  default delegates plain GET / HEAD to authorize() (existing authorizers
  keep working) and throws credential_error for anything else;
- the SigV4 header and presigned authorizers implement it: the query is
  merged into the canonical request, a body is signed as UNSIGNED-PAYLOAD
  (sha256("") without one), and X-Amz-* keys in the query are rejected
  (no signing-parameter smuggling).

Tests ([s3][sigv4]): canonical requests and signatures for every write
request kind against independently computed SigV4, query validation, and
the mock authorizer's recording of request specs.
S3 writes without a transport (src/io/rest/s3/multipart): nothing here
performs I/O, allocates pinned memory or touches CUDA.

- protocol helpers: the S3 limits (10000 parts, 5 MiB minimum part, 5 GiB
  part / single PUT), part numbering for a fixed part size, the canonical
  sub-resource queries of CreateMultipartUpload / UploadPart /
  CompleteMultipartUpload / AbortMultipartUpload, the Complete request
  XML, and parsers for InitiateMultipartUploadResult,
  CompleteMultipartUploadResult (including a 200 OK carrying an <Error>
  body, which S3 says to retry) and part ETags;
- upload_state: the thread-safe state machine of one object being written
  (parts assembled from out-of-order writes, uploaded, retried, committed
  as one PUT for an object of at most one part or as a multipart upload
  otherwise, or aborted), which the REST reactor drives.

This includes NVIDIA#221's fix 5ff07f6: a Complete 2xx without a result
document fails.  interpret_complete_response used to accept any 2xx body
that was not an S3 <Error> as success, so a proxy page or keep-alive
whitespace yielded an empty ETag and the object counted as committed
although the commit was never acknowledged.  A
CompleteMultipartUploadResult element is now required (attributes such
as xmlns accepted); anything else is http_bad_message and not retried.
A result without an ETag is still accepted, for S3-compatible stores
that omit it.

Tests ([s3]): part planning, request and response XML ("multipart:
CompleteMultipartUpload response (200 OK + <Error>)"), ETag handling, and
upload_state transitions under concurrent writers, failures and aborts.
…s in the transport

What the S3 write path needs from the io_uring transport before it sends
PUT and POST bodies:

- The stall detector counts request body bytes sent as well as response
  bytes received.  It used to count received bytes only, so an upload
  slower than one stall window was cancelled while it was still sending
  (curl's low-speed check measures the transfer in either direction).  A
  server that never answers still stalls.
- exchange_result::request_sent: the head and the whole body were written.
  With a transport error and no response byte the server may have acted
  on a request whose answer was lost; CreateMultipartUpload must never be
  re-sent then, and a conditional commit that retries into a 412 cannot
  tell whether its own earlier attempt took effect.
- exchange_result::error_body_complete and response_meta::amz_error_code:
  a drained error body's S3 <Code> (http::extract_error_code), and whether
  the body was drained to its end.  A 404 alone does not say NoSuchUpload
  from NoSuchKey, and a cut body must not be read as a known code.
  Accepted (2xx) bodies stay the caller's to interpret.
- exchange::continue_timeout: with Expect: 100-continue, the held body goes
  out anyway when no interim or final response arrived in time (a server
  that ignores Expect; curl's EXPECT_100_TIMEOUT).  The wait is a POLL_ADD
  that the deadline cancels softly (both CQEs reaped before the body is
  sent), so a user-space TLS stream's receive state is never disturbed.
- The user-space TLS stream seals stream_options::tx_batch_records (16)
  records per ciphertext SEND instead of a fixed 3: one SEND is in flight
  at a time (record order), so a 16 MiB part took ~340 SEND round trips;
  now ~64.  The FIFO holds 17 records; it is not value-initialised, so a
  connection that never sends a body touches one page of it.

Tests ([rest][net]): a slowly read 8 MiB upload under a 1 s stall window
completes (and fails without the send-side progress), a silent server
still stalls, request_sent on a refused connection and on a dropped
response, the error code of a drained 404 and an incomplete cut body, a
held body sent after continue_timeout with the connection reused after,
and the sealer's record budget per stream_options.
The write half of NVIDIA#221's loopback S3 fixture, re-applied onto the read
server of the io_uring reactor (HTTPS mode, mid-body KeyUpdate and drop
policies, chunked GET bodies, trailing garbage, data-GET waits are kept):

- an in-memory object store: PutObject (If-None-Match: * -> 412 when the
  key exists), CreateMultipartUpload, UploadPart, CompleteMultipartUpload
  (part order, ETags and the minimum part size validated),
  AbortMultipartUpload and DeleteObject; written keys are what later
  GET / HEAD / LIST see, missing_key_404 turns other keys into NoSuchKey;
- write faults (write_kind, write_fault: status, ok_with_error,
  ok_with_body, drop_mid_body, drop_after_body, hold), Expect:
  100-continue handling (honor / ignore / reject), keep-alive whitespace
  before the Create / Complete XML;
- counters (put_object_count, create_upload_count, upload_part_count,
  complete_upload_count, abort_upload_count, delete_object_count,
  active_upload_count, ...), request records and put_stored_object
  seeding.

ETags are S3's: the MD5 of a body for a single PUT and a part (hashed
outside the store lock, so uploads of several connections hash in
parallel), md5(concat(binary part MD5s))-N for a completed upload, so a
client can verify a commit against its part ETags.

Every public name is NVIDIA#221's (the REMOTE tier's test utilities use them),
s3_error_body is public, and a constructor overload keeps the read-only
HTTPS call sites (object, faults, listing, scripts, tls) unchanged.  No
library change.

Tests ([s3]): test_loopback_s3_server.cpp, the fixture itself, against
independently signed requests and raw sockets.
…art upload, commit on flush

Enable writes on the REST backend, replacing the not-implemented stubs:
rest_reactor models io_write_reactor_c with supports_device_write (the
reactor_write_traits specialization is gone), so rest_ioctx reports
supports_write() and supports_device_write().  The upload state machine
(s3::upload_state) decides; the reactor's worker moves bytes and settles
every claim exactly once.  The decision logic is NVIDIA#221's one-for-one; the
plumbing is rebuilt on the slot / exchange model of the read reactor.

- Open: create_or_truncate and create_new (If-None-Match: * on the final
  PUT / Complete); create_or_open and open_existing are not_supported.  No
  request at open; size_hint raises the part size past 10000 parts.
- Writes land in parts of write_split_bytes.  A host slice covering a part
  is uploaded from the caller's buffer; anything else is copied (memcpy on
  the worker, or D2H on the caller's stream, polled like the H2D copies of
  reads) into a part buffer from the context's host resource; the write
  completing a part uploads it.  rest_ioctx cuts slices at part boundaries.
- Part buffers count as staging: on exhaustion a write parks while write
  work or staged reads of the reactor are in flight and retries when they
  settle or a buffer is released (on any thread: the release wakes the
  loop); with nothing in flight it fails with rmm::out_of_memory.  A
  parked staged read no longer ends the submit pass (it used to `break`):
  the write calls behind it, which return the blocks, still get their
  connections, and wait() never blocks while a write call could start;
  otherwise a burst of parts and a staged read sharing a small pool
  deadlocked.  Write and flush requests have their own intake, expanded
  while reads are parked (most writes need no staging).
- flush / flush_async commit (one flush slice on the least-backlogged
  reactor; rest_ioctx::flush_io waits on flush_async_io): the remaining
  parts (gaps zeroed; never-written parts from a static 1 MiB zero block in
  windows of 64 iovecs, so a zero body can no longer be freed while it is
  sent: NVIDIA#221's b9d9090), then CompleteMultipartUpload, or one PUT for an
  object of at most one part.  The path's cached bytes and metadata are
  invalidated once the commit settled (ioctx::invalidate_cached is
  protected for that).
- Each S3 request is an s3_call with its own retry envelope on the
  worker's connections.  A call is owned by the worker's queues or slot and
  owns its job (a part, a flush, a Create), never the reverse (NVIDIA#221's
  a16aa5a): the last call of a failed commit releases the job, and with it
  the session whose destruction aborts the upload.  Work queued during a
  pass, including a Create's follow-up UploadParts and the waiters its
  settlement hands back, starts in that pass (NVIDIA#221's fc28071 invariant).
- Rules (owner's decisions): PUT / UploadPart re-sign a 403 up to
  max_auth_retry_attempts; Create / Complete / Abort stay final on 403.
  CreateMultipartUpload is never re-sent once it went out whole and its
  answer was lost, except for the free stale retry (a pooled connection
  that died before any response byte: the request was not read).  The
  object's session remembers, across flushes, that a commit request lost
  its answer: a create_new commit answering 412 then reports io_error
  ("commit outcome unknown"), never EEXIST, and a Complete answered
  NoSuchUpload is checked with one HEAD against md5(concat(binary part
  MD5s))-N of the recorded part ETags; a mismatch (or non-MD5 part ETags)
  fails the flush as "outcome unknown".  Without a lost answer a
  NoSuchUpload fails the flush.  A 409 ConditionalRequestConflict on a
  conditional commit is retried.  A 2xx Complete without a result
  document fails (NVIDIA#221's 5ff07f6).  Never a false commit success.
- Cross-reactor work: parts of one object may run on several reactors;
  work parked on the object and released by a settlement runs on the
  settling reactor in that pass when it parked there, and is handed back
  to the reactor that parked it otherwise (an intake queue and an eventfd
  wake; it starts in the pass that wakes it), so the parts of a burst
  dispatched before the UploadId exists upload on every reactor instead
  of all on the one that created it.  The abort of an object destroyed
  uncommitted goes through an abort sink owned by rest_ioctx that posts a
  control job (signed from a request spec, so DeleteObject needs only a
  new kind) to a running reactor.  Lifetime: the sink's close (first in
  ~rest_ioctx) makes every write object still alive give back its part
  buffers while the host memory resource lives; such an object then fails
  its writes and flushes with operation_canceled.
- Config: write_split_bytes (16 MiB), expect_continue (off),
  continue_timeout (1 s), complete_timeout_s (300), max_write_connections
  (0 = all).  Stats: puts, put_bytes, upload_parts, upload_part_bytes,
  zero_part_bytes, creates, completes, complete_error_retries, aborts,
  commits, commit_failures, write_failures, d2h_copies, part_buffer_waits,
  part_buffers_live.  Test hooks: on_call_started / on_call_finished.
- The reactor_worker class moves to src/io/rest/reactor_worker.hpp (no
  behaviour change) so the write path is its own TU.

Tests ([rest][write]): every NVIDIA#221 case, including the regressions "aborts
an upload whose commit failed once the object is destroyed" (a16aa5a), "a
200 Complete without a result document fails the flush; flushing again
commits" (5ff07f6) and "starts a write's first request without waiting
for a loop tick" (fc28071, now asserted with the worker's wait count);
"uploads parts parked on another reactor's CreateMultipartUpload on their
own reactor" replaces the mailbox case.  New: the zero-body generator
and a 72 MiB never-written part, the multipart ETag rule, 403 re-sign on
UploadPart and final on Create, a lost conditional PUT then 412, a lost
Complete recovered by HEAD and a NoSuchUpload that is not ours, a PUT torn
mid-body retried on a new connection, part-buffer parking and the
out_of_memory rule, device writes on every GPU, HTTPS writes on the
user-space record layer (kTLS required skips on hosts without it),
max_write_connections, expect_continue (honored and ignored), write
stats, UploadParts starting in the Create's pass while a staged read is
parked on a shared small pool (deadlocked before), a lost conditional
answer of an earlier flush then 412, a lost Complete then a failed flush
then NoSuchUpload, a Create on a stale pooled connection, writes and a
flush beside a parked read, an ioctx destroyed before a half-written
object, and a hidden throughput benchmark ([bench_rest_write]).
test_rest_write_unimplemented.cpp is gone with the stubs.
…urce device

Bugs of the converters that copy a GPU table's buffers one by one
(GPU -> host_data_representation, GPU -> DISK and GPU -> GPU); they predate
the io stack split (plan_column_copy and the fast GPU -> host converter come
from NVIDIA#52, the GPU -> GPU column walk from NVIDIA#52 / NVIDIA#116, the GPU -> DISK
converter from NVIDIA#96, view-backed gpu_table_representation from NVIDIA#116).

- Sliced sources. A view-backed representation may hold a slice, but the
  serializers copy every buffer of a column and its children from the start,
  and plan_column_copy() / reconstruct_column_p2p() only asserted offset 0 in
  debug builds: in release builds a nullable slice starting at row 37
  round-tripped as the table's leading rows with unshifted validity, and a
  prefix slice (offset 0) kept the extent of the table it was cut from in its
  children (DICTIONARY32 indices, STRING / LIST offsets, STRUCT fields), so it
  was restored with the wrong row count (GPU -> GPU failed with
  cudaErrorInvalidValue or "Invalid offsets column size" instead, or, for a
  fixed-width slice, silently returned the wrong rows). needs_compaction()
  detects both (non-zero offset anywhere, or a child extent that does not
  match its parent's rows), and serializable_view() compacts such a table
  first (cudf::table's deep copy), with the source device current, after the
  writer ordering; the copy lives until the transfer completed. GPU -> host
  and GPU -> DISK compact on the conversion stream; GPU -> GPU compacts on a
  stream of the source space and synchronizes it before the cross-device
  copies on the target stream read the copy. It needs device memory for the
  whole table on the source GPU; if that fails, the conversion fails and the
  batch stays where it is.
- Failures after enqueued work. If the compaction (or anything later) throws
  after work reading the source was enqueued on the conversion stream, the
  stream is synchronized (drain_on_unwind, under the source-device guard)
  before the exception propagates, so the caller may release or mutate the
  source. GPU -> GPU does the same for its target stream: a failed allocation
  for column i+1 no longer returns while column i's peer copy is still queued
  (also before the compacted copy is freed), and the host-staged peer copy
  drains its stream before its pinned blocks go back to the pool.
- Source device. GPU -> host_data made the source device current only around
  the compaction; it now keeps it current from the writer ordering through
  the copies and the release of the compacted copy. GPU -> DISK already
  does (and refuses a stream of another device); the compaction now runs in
  that scope too.

Regressions [disk]: view-backed nested nullable slice [37, 2011) and prefix
slices [0, 10) of a 100-row DICTIONARY32 / STRING / LIST / STRUCT table
through host_data and DISK; a compaction whose second column allocation is
refused (seam: a memory resource refusing large allocations) leaves the
conversion stream idle when the error arrives, on both routes; [multi_gpu]: a
GPU 1 table (also sliced) written late by a non-blocking stream spilled to DISK
from GPU 0 on the default stream, a GPU 1 table converted to host_data from
GPU 0 on a GPU 1 stream, nested / prefix / fixed-width slices converted GPU 0
-> GPU 1 and GPU 1 -> GPU 0 and back, and failed GPU -> GPU conversions (an
allocation refused on the target, a compaction refused on the source) that
leave the source's writer finished when the error arrives. The test sources
that own a raw stream move it out of the moved-from object.
Add ioctx::remove_io_object_async / remove_io_object: the front end strips a
file: scheme, invalidates the path in the ioctx's cache and metadata store
before the request and again once it settled (as the write front ends do),
and forwards to the new protected hook remove_io_object_async_io. The default
hook resolves with not_implemented_error (errc::not_supported): the local
backends do not implement it (DISK spaces keep removing their files with
std::filesystem).

rest_ioctx implements it with S3 DeleteObject on its control reactor (least
backlogged, ties rotated): rest_reactor::delete_object_async posts a
control_job of the new kind delete_object (DELETE /bucket/key, signed from its
request spec through authorize_request, like AbortMultipartUpload), retried
like the other control jobs with max_retry_attempts. Classification: 2xx is
success; a 404 NoSuchKey, or a 404 without an error code whose error body was
read to its end, is success too (the key is gone, also when the 404 arrives
while the reactor stops); any other 404 (NoSuchBucket, ...) fails; a 404 whose
error body was cut or not drained (the code cannot be known) is retried like a
transport error (finish_control); a 403 is final. io_priority::low marks the
job background: it starts only when no reads or writes wait, behind data work
instead of ahead of it like blocking control calls (the reactor has no
separate bulk class).

The job owns the promise its on_done settles, and the intake / slot / retry
queue own the job, so the returned future may be dropped without any
reactor-owned bookkeeping. Before start() / after shutdown() (or from a
reactor thread) it runs blocking on a private helper like head_object and
returns a ready future. A delete queued or in flight at shutdown fails with
operation_canceled (intake::close / drain). Stats: deletes (attempts sent).

Tests: test/io/rest/test_rest_delete.cpp [rest][delete] -- 204 (presigned
and header signing), 404 NoSuchKey and code-less 404 ok, 404 NoSuchBucket
fails without a retry, a 404 whose (oversized) error body was not read is
retried, 403 final, 503-then-204 retry, exhausted retries, unstarted reactor,
dropped futures (incl. many low-priority ones) settle, shutdown with deletes
in flight, non-s3 path, uring -> not_supported. The loopback server's status
fault takes an optional body for the oversized 404.
Tier gains REMOTE, appended before SIZE (GPU 0, HOST 1, DISK 2 unchanged,
REMOTE 3, SIZE 3 -> 4). A REMOTE space keeps spilled batches as objects
under an object-store prefix; the core library stays io-free (it only
stores the io context pointer, as for DISK).

- memory/config.hpp: remote_memory_space_config {remote_id, memory_capacity
  (default unbounded_remote_capacity = 1 PiB), uri_prefix, io_context},
  thresholds as DISK (trigger = capacity, stop = 0.99 x capacity); appended to
  the memory_space_config variant (existing indices unchanged); public
  validate_remote_uri_prefix() (s3://bucket[/prefix]; rejects other schemes,
  empty bucket, userinfo / port in the bucket, query / fragment, whitespace and
  control characters, empty segments; returns it normalized).
  ABI note: memory_space_config gains an alternative. It is appended, so the
  existing alternatives keep their indices, but the variant's size and layout
  change: code compiled against the old header must be rebuilt.
- memory_space: REMOTE constructor (reuses disk_access_limiter: capacity
  bounds the bytes of reservations in flight, not the bytes stored), new
  get_remote_uri_prefix(); get/set/get_or_init_io_context accept DISK or
  REMOTE; to_string prints REMOTE.
- reservation_manager_configurator::add_remote_space(remote_id, uri_prefix,
  capacity = unbounded_remote_capacity, io_context = nullptr); build() emits
  REMOTE configs after the DISK ones.
- memory/disk_table.hpp: generate_remote_object_uri(prefix) ->
  {prefix}/{hostname-pid-start_ms}/batch_{seq:016x}_{rand:08x}.cucascade (no
  I/O), implemented in src/memory/remote_uri.cpp.

Tier audit (switches have `default:`, so the compiler does not flag missing
REMOTE cases; every use in the tree checked by hand):
  changed here:
  - memory/common.hpp Tier enum
  - memory/config.hpp *_config::tier(), memory_space_config variant
  - memory_space.cpp ctor overloads, get/set/get_or_init_io_context (DISK|REMOTE),
    to_string switch (+REMOTE); get_disk_mount_path stays DISK-only
  - reservation_manager_configurator build() (REMOTE after DISK)
  - test/utils/mock_test_utils.hpp make_mock_memory_space (+REMOTE)
  changed in the next commit (cudf layer):
  - cudf/bandwidth_profiler.cpp canonical_type_for switch (default throws),
    should_skip_pair DISK rules; test_bandwidth_profiler.cpp operator<< switch
  - cudf/representation_converter_builtins.cpp registry (REMOTE converters)
  no change needed (GPU/HOST-specific or tier-generic):
  - memory_space.cpp dtor (HOST pool unregistration), make_reservation* /
    get_available_memory / ... visits over the reservation allocator variant
  - memory_reservation.hpp tier_memory_resource_trait (void, like DISK)
  - memory_reservation_manager.{hpp,cpp} config visit and Tier-keyed lookups,
    request strategies (any_memory_space_in_tier[s], *_to_downgrade/upgrade)
  - memory/common.cpp make_default_allocator_for_tier (null resource branch)
  - memory/topology_index.hpp GPU/HOST id extraction
  - memory/fixed_size_host_memory_resource.cpp HOST space id
  - data/data_batch.hpp GPU-only checks (:398, :677), data/common.hpp
  - io/cache/types.cpp, io/datasource_factory.cpp HOST pool lookups
  - cudf/host_data_representation.cpp HOST resource, bandwidth_profiler GPU
    stream-owner checks, cudf/bandwidth_profiler.hpp GPU defaults
  - benchmark/benchmark_{disk,representation}_converter.cpp (DISK/GPU/HOST only)
  - tests enumerating explicit tier subsets (test_memory_reservation_manager,
    test_data_*), test_disk_memory_space accessor rejection (GPU)
  docs (AGENTS.md, README.md: "three tiers") are updated with the docs commit.

Tests: test/memory/test_remote_memory_space.cpp [memory][remote].
… context

HOST and GPU batches can now be converted to and from REMOTE memory spaces:
each batch becomes one object in an S3(-compatible) store.

- cucascade::remote_data_representation (core, io-free): the column layout
  (disk_table_allocation, file_path = object URI) and an opaque handle to the
  written object (detail::remote_object, defined by the cudf layer). clone() is
  unsupported (as DISK).
- Object removal: releasing the handle never does I/O or blocks (it may run on
  a reactor thread of its own context); it hands the context, the committed
  write object and the URI to a process-wide removal queue. Its thread (never
  a reactor thread) issues the DeleteObject (io_priority::low), owns the
  context reference until the removal settled and releases it there: removals
  survive the spaces / managers / representations that held their context (a
  context's destruction would cancel them), and a context's last reference is
  never dropped on its own reactor thread. The queue is created eagerly (by
  make_remote_io_context and before every REMOTE write), entries are allocated
  before the context is moved in (a failed allocation keeps the reference for
  good rather than releasing it on the caller's thread) and moved by splicing,
  and its thread never terminates on an exception. Failures are WARNed
  (rate-limited) and left to the bucket's lifecycle rule. Each pass first
  releases settled removals, then issues at most the ones queued when it
  began, one at a time. Public drain_remote_object_removals() waits for all of
  them (also run at exit; limits documented: late static releases, no
  deadline, unbounded, no fork). The scoped, bounded
  drain_remote_object_removals(contexts, timeout) is what embedders tear down
  with: it waits only for the removals of the given contexts (counted per
  context), for at most the timeout; at the deadline, removals still queued
  for them are never issued, each context is shut down (settling its calls in
  flight) and their entries are released on the calling thread, independent
  of the queue's thread; it reports how many were abandoned (WARN; left to the
  lifecycle rule) and whether it timed out.
- make_remote_io_context(remote_io_context_options) (cudf/remote_io_context.hpp):
  a started REST context (no fs_cache, no default context) owning its pinned
  staging (n_reactors x staging_bytes_per_reactor, 1 MiB blocks; the
  base-from-member pattern of make_disk_io_context). Validates the endpoint
  (exactly http(s)://host[:port]), region and keys (std::invalid_argument),
  that the staging holds two parts, and that io_uring is usable
  (std::system_error). Applies object_store CA/verify over rest.tls as the
  registry does.
- io::rest::s3::make_authorizer(object_store_config) is now public and shared
  by the io_context registry and make_remote_io_context.
- Converters HOST<->REMOTE and GPU<->REMOTE, factored with the DISK ones over
  layout_{host,gpu}_batch / restore_{host,gpu}_batch (same 4 KiB-aligned
  layout, metadata in memory). Writing (src/cudf/remote_io.cpp): a fresh name
  (generate_remote_object_uri), open create_new, the slices plus zeros for the
  alignment gaps so every part completes while the batch is written, flush =
  commit (single PUT or CompleteMultipartUpload). Contexts from
  make_remote_io_context write in windows of whole parts that fit half their
  staging and admit concurrent conversions against it (exec::admission_control),
  so a batch larger than the staging or concurrent spills never exhaust it.
  Device reads are counted in staging blocks (every GET allocates whole
  blocks; the GET count comes from the new public io::rest::max_read_gets(),
  the reactor's segment planner's worst case: its smallest target,
  max(1, n / min_segment_bytes, ceil(n / max_segment_bytes)), sharing the
  count with physical_ranges; GETs are not cut at part boundaries),
  so a wide table of tiny buffers is read in windows whose block count fits
  half the staging, each admitted for exactly that. A conditional commit
  refused with EEXIST, or one whose outcome the store left unknown (the new
  public io::rest::may_be_commit_outcome_unknown: an attempt's answer was lost
  and the retry got 412, or a lost Complete could not be verified; reported
  as io_error, never EEXIST), renames once and retries; a key left by an
  unknown outcome is not deleted (it could be another writer's) and is left
  to the bucket's lifecycle rule. Any failure destroys the unflushed object
  (the upload is aborted) and rethrows, leaving the source intact. Reads go
  through the committed write object (known size, If-Match: no HEAD), with
  bytes-delivered checks; a failed restore leaves the REMOTE representation
  valid. A REMOTE space without an io context -> std::logic_error. GPU
  sources: the source device is current from the writer ordering through the
  write (a null stream resolves to that device's default stream for both; an
  explicit stream of another device is refused, as for DISK), a
  sliced view-backed source is compacted first (layout_gpu_batch, shared with
  GPU -> DISK), and a failure after work reading the source was enqueued
  drains the conversion stream before it propagates.
- bandwidth profiler: canonical type REMOTE -> remote_data_representation;
  storage tiers (DISK, REMOTE) are never paired with each other, and REMOTE
  pairs are measured only with the new bandwidth_profile_config::
  measure_remote_pairs (default false).

Tests ([remote]): test/data/test_remote_converters.cpp (loopback S3 server:
round trips GPU and HOST of numeric, strings + nulls, lists/structs with nulls,
nullable, sliced, empty, single PUT, multi-part, larger than the staging, one
HTTPS case, view-backed sliced nested nullable source, GPU 1 source with a
late non-blocking writer converted from GPU 0 on the default stream
[multi_gpu] (also sliced), an explicit stream of another GPU refused before
any upload [multi_gpu], view-backed prefix slices (dictionary / string /
list / struct), a compaction failing part-way (stream drained), wide table of
tiny columns with slow GETs and 4 KiB segments (staging windows);
DeleteObject on destruction, removal outliving its space and manager
(throttled retry after the manager's teardown), release on a reactor thread of
the representation's own context, delete failure only warns; the scoped
drain: cancels a held removal at its deadline and releases the context, waits
only for its own contexts, never issues a removal still queued and does not
depend on the queue's thread (blocked in a removal through a shut-down
context), cancels nothing when settled in time, returns at once with a zero
timeout and nothing outstanding; persistent UploadPart 5xx and Complete 5xx -> throw +
abort + no object; 412 collision rename (PUT and Complete) and twice -> fail;
unknown commit outcome (lost PUT answer then 412; Complete NoSuchUpload not
verified) -> one retry under a new name;
GET failure on restore -> throw, representation intact; no io context ->
logic_error; concurrent spills over shared staging),
test/data/test_remote_io_context.cpp, and a REMOTE case in
test_bandwidth_profiler.cpp; shared helpers in test/utils/remote_test_utils.hpp;
max_read_gets against the reactor's GET count in test_rest_read_robustness.cpp
([rest]).
- docs/io/remote-tier-design.md: configuration (space prefix vs. io context,
  no default context, endpoint/credential validation), the memory_space_config
  variant change (appended alternative: indices kept, size/ABI changed),
  capacity semantics (in-flight bytes, no stored-bytes quota), object naming
  and layout, spill and restore steps (source device current from the writer
  ordering through the write, sliced sources compacted), staging windows
  (writes in whole parts, device reads counted in staging blocks) and
  admission, failure handling, lifecycle (the process-wide removal queue that
  owns contexts until their deletes settled, drain_remote_object_removals()
  and the scoped, bounded drain_remote_object_removals(contexts, timeout)
  embedders tear down with)
  and the requirement of an S3 lifecycle rule (expiration +
  AbortIncompleteMultipartUpload) for objects left by crashes or failed
  deletes; what is not implemented.
- docs/io/rest-transport.md "Writes": DeleteObject (a control job on the
  reactor; 2xx, 404 NoSuchKey and a complete code-less 404 are success, other
  404s fail, a cut 404 is retried; io_priority::low behind data work) and the
  `deletes` counter. remote-tier-design.md describes the segment planner's GET
  bound and the fresh-key retry after an unknown commit outcome.
- README, docs/README, ARCHITECTURE.md (tier table and diagram), AGENTS.md,
  memory-management.md, bandwidth-profiler.md: the fourth tier.
…IO to them

A uring or cuFile reactor pool used to be N interchangeable reactors staged
from the first HOST space with unpinned workers. On a multi-socket, multi-GPU
host that bounces device IO through the wrong socket's memory and CPUs. A pool
built with a topology index now runs its reactors per GPU, each placed on that
GPU's NUMA node, and device requests are routed to the reactors local to their
GPU.

REST is deliberately not placed per GPU: its reactors serve the NIC, not a
GPU, so io_config::rest_n_reactors stays a total (0 = sized from the NIC) and
the REST pool keeps its NIC IRQ plan (rest::plan_rest_pool in
src/io/rest/reactor_placement.hpp). The two reactor_placement headers point at
each other so the shared file name does not mislead.

Placement (include/cucascade/io/reactor_placement.hpp, src/io/reactor_placement.cpp):
- reactor_placement: home GPU, its NUMA node, the reactor's pinned staging
  (null = the pool context's resource) and the CPUs its worker is pinned to.
  A default placement is "unplaced" (the previous behaviour).
- numa_spill_policy { min_backlog_bytes = 64 MiB, ratio = 2.0 } and
  reactor_layout { placements, spill } (reactor_layout::unplaced(n)).
- make_reactor_placements(reactors_per_gpu, topology, reservation_manager,
  pin_threads): reactors_per_gpu placements per GPU of the index, staging from
  the HOST space of the GPU's NUMA node (kept on the shared staging when the
  node is unknown, has no HOST space, or stages in a block size other than the
  first HOST space's: a pool's cache chunks need one staging block size), CPUs
  from the topology when pin_threads. No topology: reactors_per_gpu unplaced.
- pin_reactor_thread(): workers pin themselves first thing (uring before it
  sets up its ring), so the ring and per-worker allocations land on the home
  node. The mask is the placement's CPUs intersected with the
  thread's inherited affinity, so a process confined by taskset / numactl -C
  is never widened (the kernel intersects only with the cgroup cpuset); an
  empty intersection or a failure leaves the thread unpinned and is logged
  (warning once, debug after).
- validate_spill_policy(): numa_spill_policy::ratio must be finite and >= 1,
  else std::invalid_argument (from io_context_registry eagerly, and from every
  layout-built pool).

Routing (templated_ioctx):
- New constructors taking a reactor_layout (a factory called per placement, or
  pre-built reactors plus their placements; a count mismatch, an invalid
  spill policy, or reactors staging in different block sizes throw
  std::invalid_argument -- the pool reports one staging_block_size, which a
  prefetching cache's chunks must match). The old constructors build an
  unplaced pool. An unplaced reactor in a placed pool is remote to every GPU.
- A request is routed by the GPU of its first device slice (an unset device id
  still resolves to the current device, now only when needed). Its candidates
  are the reactors local to that GPU (same NUMA node, or homed on the GPU when
  the node is unknown); the request spills to the whole pool only when the
  least backlogged local reactor holds more than min_backlog_bytes AND more
  than ratio x the least backlogged remote one. Host-only requests, GPUs
  without placed reactors and all-local pools use the whole pool. Within the
  candidates selection is unchanged: the dispatch_fanout least backlogged
  (the config's fan_out, else 2), rotation breaking ties. The REST pool is
  unplaced, so it keeps using the whole pool. cufile_ioctx's vectored-read
  fan-out spans the routable reactors only; its host-only vectored write still
  goes to one reactor.
- PTDS stand-in streams, null-stream pinning, write splitting and every other
  dispatch rule are untouched.

Backends:
- uring_reactor / cufile_reactor take a reactor_placement (defaulted);
  uring_reactor exposes placement(). uring bounce slots and
  staging_block_size() come from the placement's staging when set; cuFile only
  pins its worker (its scratch comes from the process-wide pool).
- uring_ioctx / cufile_ioctx gain (reactor_layout const&, ctx) constructors;
  (n, ctx) delegates to reactor_layout::unplaced(n).
- io_context_registry, make_uring_ioctx_factory and make_cufile_ioctx_factory
  take an optional std::shared_ptr<const memory::topology_index> (default null
  keeps one GPU's worth of unplaced reactors). make_rest_ioctx_factory is
  unchanged: the registry does not hand it the index.

Config rename (io_config), breaking for embedders:
- uring_n_reactors{4}  -> uring_reactors_per_gpu{1}
- cufile_n_reactors{4} -> cufile_reactors_per_gpu{1}
- new: pin_reactor_threads{true}, reactor_spill (numa_spill_policy), both for
  the uring and cuFile pools
The counts are now per GPU of the registry's topology index; without one a
pool has *_reactors_per_gpu reactors. rest_n_reactors is not renamed and stays
a total. make_disk_io_context keeps building one GPU's worth of unplaced
reactors from io.uring_reactors_per_gpu / io.cufile_reactors_per_gpu. Callers,
tests and the docs naming the old keys are updated.
- test/io/test_reactor_placement.cpp: unplaced pools without a topology;
  per-GPU placements carry the GPU's NUMA node and CPUs (or none with pinning
  off); staging from the HOST space of the GPU's node, the shared staging for
  a node without one; pin_reactor_thread pins (and an unplaced placement
  leaves the thread alone), never past the inherited affinity (narrowed to the
  intersection, left alone when it is empty); starting a placed uring reactor
  pins its worker within the starting thread's affinity; a placed uring
  reactor stages from its placement's resource while an unplaced one uses the
  context's; a layout mixing staging block sizes is rejected; the uring
  factory and an io_context_registry built with a topology index run
  reactors_per_gpu reactors per GPU of the index (one GPU's worth without
  one); invalid spill ratios (< 1, NaN, inf) are rejected; a vectored cuFile
  read of a GPU fans out over that GPU's NUMA-local reactors only, a
  host-only one over the pool, and a host-only vectored write still picks one
  reactor.
- test/io/test_templated_ioctx.cpp: next_reactor keeps a device request on its
  GPU's NUMA node while it keeps up, uses the whole pool for host-only
  requests and unknown GPUs, spills only past both the backlog floor and the
  ratio, falls back to the GPU's own reactors when the node is unknown; a read
  is routed by its first device slice's GPU; a layout must place every
  reactor and carry a valid spill policy.
aminaramoon added a commit to aminaramoon/sirius that referenced this pull request Nov 14, 2026
Bumps cuCascade to aa09ce3 (NVIDIA/cuCascade#224), whose io registry takes a
topology index and homes each uring and cuFile reactor on a GPU: pinned staging
from that GPU's NUMA node, worker pinned to its local CPUs, and device reads
preferring the reactors of their GPU's node, spilling to the rest of the pool
when those are much busier (io_config::reactor_spill).

The scan manager passes its topology_index to the io registry, including the
throwaway registry of path-scoped S3 configs, and builds its mandatory cuFile
context from the same per-GPU reactor layout. A test with a two-GPU topology
checks that the shared and the path-scoped registries both build per-GPU uring
pools and a REST pool of exactly rest_n_reactors reactors (reactor count and
warmup connections), whatever the GPU count.

cuCascade's uring and cuFile reactor counts are per GPU now, so Sirius's are
too: uring_n_reactors / cufile_n_reactors become uring_reactors_per_gpu /
cufile_reactors_per_gpu. uring defaults to 1 per GPU (was 4 in total); cuFile
keeps 4, being single-GPU only. The old keys are rejected with an error naming
the rename. The REST count is not renamed: rest_n_reactors stays a total for
the process (0 = sized from the egress NIC's IRQs), because REST reactors follow
the NIC, not the GPUs, and a per-GPU count would multiply the connection pool
with the GPU count. Config, YAML fixtures, bench configs, tests and docs follow.
giancao added a commit to giancao/cuCascade that referenced this pull request Nov 14, 2026
Mark the places where NVIDIA#221 (S3 writes), NVIDIA#222 (object deletion, REMOTE
tier) and NVIDIA#224 (NUMA reactor placement) would land, so the gaps are
visible in the code.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants