Repository navigation
perf(eve): queue session stream writes off the pre-model path - #5287
chadhietala wants to merge 1 commit into
Conversation
Bundle + Package Summary:
|
| Area | Metric | Baseline | Current | Delta |
|---|---|---|---|---|
| Package | Packed tarball | 7.77 MB | 7.77 MB | +763 B |
| Package | Unpacked publish size | 27.27 MB | 27.27 MB | +2.0 kB |
| Package | Installed footprint | 75.57 MB | 75.57 MB | +2.0 kB |
| Package | Published files | 4215 | 4215 | 0 |
| Package | Installed files | 8209 | 8209 | 0 |
| Package | Installed package instances | 34 | 34 | 0 |
| Package | Distinct installed package names | 32 | 32 | 0 |
| Package | Installed dependency edges | 51 | 51 | 0 |
| Package | Installed optional peer edges | 8 | 8 | 0 |
| Runtime | Unique function payloads | 2 | 2 | 0 |
| Runtime | Total function bytes | 20.87 MB | 20.87 MB | +2.5 kB |
| Runtime | Public routes | 18 | 18 | 0 |
Changed function payloads vs main (d5b9601) (2)
| Function | Status | Baseline | Current | Delta | Route changes |
|---|---|---|---|---|---|
functions/__server.func |
changed | 10.44 MB | 10.44 MB | +1.2 kB |
none |
functions/.well-known/workflow/v1/flow.func |
changed | 10.44 MB | 10.44 MB | +1.2 kB |
none |
eve init install
| Metric | Baseline | Current | Delta |
|---|---|---|---|
| Installed footprint | 113.95 MB | 113.95 MB | +2.0 kB |
| Installed packages | 97 | 97 | 0 |
| dependencies | 4 | 4 | 0 |
| devDependencies | 2 | 2 | 0 |
| Dependency package bytes | 45.67 MB | 45.67 MB | +2.0 kB |
| devDependency package bytes | 5.11 MB | 5.11 MB | 0 B ➖ |
Build Metadata
- Preset:
vercel - Nitro:
nitro@3.0.260903-beta - Output directory:
apps/fixtures/weather-agent/.vercel/output - Build metadata timestamp: 2026-10-23T02:29:52.488Z
- Route aliases: 18 public, 1 internal (19 total aliases)
- Vercel routes in config: 21
- Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down
Package Details
- Package:
eve@0.83.0 - Package directory:
packages/eve - Tarball: 7.77 MB (
eve-0.83.0.tgz) - Unpacked payload: 27.27 MB across 4215 published files
- Installed footprint: 75.57 MB across 8209 installed files
- Installed root package: 27.27 MB
- Installed dependencies: 48.30 MB
- Installed package instances: 34
- Distinct installed package names: 32
- Installed dependency edges: 51
- Installed optional peer edges: 8
- Runtime dependencies: 2
- Peer dependencies: 6 (5 optional)
Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Graph metrics read only package.json files in package directories directly beneath a node_modules boundary, including nested boundaries. Each directory is one package instance; distinct names come from those manifests. Dependency edges count each unique name in dependencies or optionalDependencies per instance; optional peer edges count peerDependencies marked optional.
Heavy installed dependencies
eve: 27.27 MB (36.1%)@rolldown/binding-linux-x64-gnu: 19.89 MB (26.3%)ai: 8.36 MB (11.1%)zod: 6.15 MB (8.1%)undici: 3.58 MB (4.7%)
Publish payload breakdown
Published file size
🟠 dist/src/compiled/shadcn-registry/index.js [#####...................] 3.85 MB 14.1%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [###.....................] 2.29 MB 8.4%
🟠 dist/src/compiled/run/index.js [#.......................] 1.00 MB 3.7%
🟡 dist/src/compiled/@vercel/blob/index.js [#.......................] 604.9 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-D-I... [#.......................] 521.9 kB 1.9%
🔴 Other published files [########################] 19.01 MB 69.7%
Installed footprint breakdown
Installed package size
🔴 eve [########################] 27.27 MB 36.1%
🔴 @rolldown/binding-linux-x64-gnu [##################......] 19.89 MB 26.3%
🔴 ai [#######.................] 8.36 MB 11.1%
🔴 zod [#####...................] 6.15 MB 8.1%
🟠 undici [###.....................] 3.58 MB 4.7%
🟠 nitro [##......................] 1.89 MB 2.5%
🔴 Other installed packages [#######.................] 8.42 MB 11.1%
Runtime dependencies (2)
| Package | Range | Notes |
|---|---|---|
nitro |
3.0.260903-beta |
|
undici |
8.10.2 |
Peer dependencies (6)
| Package | Range | Notes |
|---|---|---|
@opentelemetry/api |
^1.0.0 |
optional peer |
ai |
catalog: |
|
braintrust |
^3.0.0 |
optional peer |
chat |
^4.42.0 |
optional peer |
dd-trace |
^6.13.0 |
optional peer |
just-bash |
^3.1.0 |
optional peer |
eve init install drill-down
eve init install details
- Command:
eve init my-agent - Package manager:
npm - Installed footprint: 113.95 MB across 10126 installed files
- Installed packages: 97 total (91 transitive-only)
- dependencies: 4 direct packages totaling 45.67 MB
- devDependencies: 2 direct packages totaling 5.11 MB
- Other transitive package files: 63.17 MB
Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.
Heavy installed dependencies
@typescript/typescript-linux-x64: 27.95 MB (24.5%)eve: 27.27 MB (23.9%)@rolldown/binding-linux-x64-gnu: 19.89 MB (17.5%)zod: 9.76 MB (8.6%)ai: 8.36 MB (7.3%)
Installed footprint breakdown
Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 24.5%
🔴 eve [#######################.] 27.27 MB 23.9%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 19.89 MB 17.5%
🔴 zod [########................] 9.76 MB 8.6%
🔴 ai [#######.................] 8.36 MB 7.3%
🟠 undici [###.....................] 3.58 MB 3.1%
🔴 Other installed packages [###############.........] 17.15 MB 15.0%
dependencies (4)
| Package | Range | Installed size | Share |
|---|---|---|---|
@vercel/connect |
2.5.2 |
276.3 kB | 0.2% |
ai |
^7.0.137 |
8.36 MB | 7.3% |
eve |
file:eve-0.83.0.tgz |
27.27 MB | 23.9% |
zod |
4.5.4 |
9.76 MB | 8.6% |
devDependencies (2)
| Package | Range | Installed size | Share |
|---|---|---|---|
@types/node |
24.x |
2.61 MB | 2.3% |
typescript |
7.0.2 |
2.50 MB | 2.2% |
Function Drill-Down
Payload Size Graph
Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func [########################] 10.44 MB 50.0%
🔴 functions/__server.func [########################] 10.44 MB 50.0%
Top Function Payloads
🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 10.44 MB
| Metric | Value |
|---|---|
| Public routes | /.well-known/workflow/v1/flow |
| Runtime | nodejs24.x |
| Handler | index.mjs |
| Payload | 10.44 MB |
| Function files | 10.44 MB across 121 files |
| Traced dependencies | 0 B |
| Signal | 🟠 Bundled file _chunks/vercel.web.mjs is 2.47 MB (23.7%) |
🟠 🔎 Dependency Analysis
📦 Bundled files:
Bundled file size
🟠 _chunks/vercel.web.mjs [#############...........] 2.47 MB 23.7%
🟡 _libs/undici.mjs [#####...................] 994.5 kB 9.5%
🟡 _chunks/sandbox2.mjs [####....................] 815.0 kB 7.8%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 732.3 kB 7.0%
🟡 _chunks/token-util-Bla5Z0G9.mjs [####....................] 704.5 kB 6.7%
🔴 Other bundled files [########################] 4.72 MB 45.2%
🧾 Vercel Config
{
"handler": "index.mjs",
"launcherType": "Nodejs",
"shouldAddHelpers": false,
"supportsResponseStreaming": true,
"runtime": "nodejs24.x",
"maxDuration": "max",
"experimentalTriggers": [
{
"type": "queue/v2beta",
"topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
"consumer": "default",
"retryAfterSeconds": 5,
"initialDelaySeconds": 0
}
],
"environment": {
"WORKFLOW_PRECONDITION_GUARD": "1"
}
}🟠 functions/__server.func • 17 public routes, 1 internal alias • 10.44 MB
| Metric | Value |
|---|---|
| Public routes | //.well-known/workflow/v1/webhook/[token]/eve/v1/callback/[token]/eve/v1/connections/[name]/callback/[attemptId]/[token]/eve/v1/connections/[name]/callback/[token]/eve/v1/health/eve/v1/info/eve/v1/session/eve/v1/session/[parentSessionId]/subagents/[callId]/[childSessionId]/stream/eve/v1/session/[sessionId]/eve/v1/session/[sessionId]/cancel/eve/v1/session/[sessionId]/clear/eve/v1/session/[sessionId]/compact/eve/v1/session/[sessionId]/reset/eve/v1/session/[sessionId]/stream/eve/v1/session/[sessionId]/stubs/eve/v1/task-input/[token] |
| Internal aliases | /__server |
| Runtime | nodejs24.x |
| Handler | index.mjs |
| Payload | 10.44 MB |
| Function files | 10.44 MB across 121 files |
| Traced dependencies | 0 B |
| Signal | 🟠 Bundled file _chunks/vercel.web.mjs is 2.47 MB (23.7%) |
🟠 🔎 Dependency Analysis
📦 Bundled files:
Bundled file size
🟠 _chunks/vercel.web.mjs [#############...........] 2.47 MB 23.7%
🟡 _libs/undici.mjs [#####...................] 994.5 kB 9.5%
🟡 _chunks/sandbox2.mjs [####....................] 815.0 kB 7.8%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 732.3 kB 7.0%
🟡 _chunks/token-util-Bla5Z0G9.mjs [####....................] 704.5 kB 6.8%
🔴 Other bundled files [########################] 4.72 MB 45.2%
🧾 Vercel Config
{
"handler": "index.mjs",
"launcherType": "Nodejs",
"shouldAddHelpers": false,
"supportsResponseStreaming": true,
"runtime": "nodejs24.x"
}Build Timing: e2e/fixtures/agent-tools-sandbox
This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.
Build mode: deployable Vercel build with sandbox template prewarm included.
- Build pipeline: 2.87 s -> 2.80 s (-72.7 ms) vs
main (d5b9601). - Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `main (d5b9601)`
| Phase | Baseline | Current | Delta |
|---|---|---|---|
extension.check |
0.4 ms | 7.4 ms | +7.0 ms |
project.resolve |
0.2 ms | 0.3 ms | +0.1 ms |
workspace.create |
0.4 ms | 0.4 ms | 0.0 ms |
host.prepare |
445.1 ms | 403.4 ms | -41.7 ms |
vercel.service-prefix.resolve |
1.1 ms | 1.1 ms | 0.0 ms |
nitro.create |
477.3 ms | 453.9 ms | -23.4 ms |
sandbox.prewarm |
477.1 ms | 496.3 ms | +19.2 ms |
nitro.cache.prepare |
0.2 ms | 0.2 ms | 0.0 ms |
nitro.prepare |
0.5 ms | 0.5 ms | 0.0 ms |
nitro.public-assets |
0.6 ms | 0.6 ms | 0.0 ms |
nitro.prerender |
0.4 ms | 0.3 ms | -0.1 ms |
nitro.bundle |
1.32 s | 1.30 s | -18.7 ms |
nitro.cache.write |
0.3 ms | 0.3 ms | 0.0 ms |
vercel.workflow-function.materialize |
83.7 ms | 70.4 ms | -13.3 ms |
agent-summary.emit |
0.7 ms | 0.6 ms | -0.1 ms |
connect-manifest.emit |
0.3 ms | 0.2 ms | -0.1 ms |
nitro.close |
0.1 ms | 0.1 ms | 0.0 ms |
output.publish |
2.9 ms | 2.5 ms | -0.4 ms |
workspace.remove |
1.6 ms | 1.7 ms | +0.1 ms |
| // `write` queues the next stream write and returns the stamped event, so publishing | ||
| // does not wait for the stream round trip. The step drains the queue before it | ||
| // returns; see `runSessionStep` and `commitSessionStep`. | ||
| const emit = async (event: UnstampedMessageStreamEvent): Promise<MessageStreamEvent> => { | ||
| const stamped = await writer.write(await dispatcher.deliver(event)); | ||
| recordPublishedEvent(ctx, stamped); |
There was a problem hiding this comment.
Queuing here also moves the turn's terminal events ahead of their durability, not just the pre-model ones. emit returns as soon as the write is queued, so for turn.completed the instrumentation wrapper (instrumentation/native-events.ts:71-100) publishes channel.delivery.completed and clears ActiveChannelDeliveriesKey, and the stream-event hooks and memory lifecycle in turn-event-handler.ts:88-103 run, all before the write settles. If that write rejects, nothing emits again before drain() at turn-step.ts:614. That drain sits outside the catch that calls failChannelDeliveries, so the step fails and retries after telemetry already reported the delivery as completed and the terminal hooks already fired.
Before this PR the same failure threw from emit inside runModelCallBatch, so no hooks ran and the delivery was reported as channel.delivery.failed. The last event of a turn is the one most likely to hit this, since no later write is around to fail fast.
Suggestion: keep the fire-and-forget behavior for pre-model and streaming events, and have emit await writer.drain() before it returns for turn-boundary events (turn.completed / turn.failed / turn.cancelled / input.requested / session.*). That only adds one round trip at the end of the turn, off the path this PR is optimizing, and it keeps "after its write" in turn-event-handler.ts:86 true for the events whose consumers report outcomes. The write doc at line 295 ("returns the event as written") also needs to say "queued".
There was a problem hiding this comment.
Good catch. Fixed in 8e580b9: emit now drains for boundary events (turn.* terminal/park, session.*, input.requested, authorization.required), so their write settles before hooks and instrumentation run and a failure surfaces inside runModelCallBatch again. Pre-model and streaming events keep the queued path. Also updated the write doc to say queued.
| if (pending.size >= MAX_PENDING_SESSION_EVENT_WRITES) { | ||
| await new Promise<void>((resolve) => capacityWaiters.add(resolve)); | ||
| if (failure !== undefined) throw failure; | ||
| } |
There was a problem hiding this comment.
The 64-write cap is soft when emitters run concurrently (for example the Promise.allSettled over approved calls in harness/hitl/approved-calls.ts:115). Once one write settles, settleCapacity resolves every waiter, and none of them checks pending.size again after waking. So N waiters behind a full queue all enqueue together, and the queue grows to 63 + N. A write that arrives in the same tick also gets ahead of the waiters that were queued before it.
A while loop fixes the bound: while (pending.size >= MAX_PENDING_SESSION_EVENT_WRITES) { await new Promise(...); if (failure !== undefined) throw failure; }. Waking one waiter per freed slot would fix it too. Also worth thinking about whether the Set, the waiter Set, and the settleAll loop can collapse into one FIFO array. WritableStream already settles writes in order and rejects every later write once it errors, so the oldest entry is the only one capacity has to wait on, and the newest entry is the only one drain has to wait on.
There was a problem hiding this comment.
Fixed. write now holds a FIFO enqueue lock while it waits for capacity and re-checks in a while loop, so wakeups cannot overshoot the cap and a later event cannot overtake a waiting one. drain and release take the same lock. capacityWaiter is a single callback because only the lock holder can wait. I kept the pending Set instead of collapsing to the newest write; it keeps the failure and backpressure bookkeeping simple.
| // The queued events must be durable before the step commits; publishing only | ||
| // waited for each write to be queued. | ||
| await publisher.writer.drain(); |
There was a problem hiding this comment.
No test covers this drain, and it is the only thing that turns a late write failure into a step failure. release() in the finally also waits for pending writes, but it uses allSettled and drops the error. So deleting this line (or the one at 590, or the one in publishInSessionScope) would let the step commit successfully after a stream write rejected, and every current test would still pass. The new writer tests only call drain() directly.
Could you add a turn-step test where the session writable rejects the final write (turn.completed) after a delay, and assert that the step rejects? It should also check that the result isn't returned before a slow write settles. A one-line note on release() saying it is cleanup only and never surfaces write failures would make that contract explicit too.
There was a problem hiding this comment.
Added a turn-step test that publishes a non-boundary step.started through HandleEventKey, holds its write, asserts the step has not settled, then rejects the write and asserts the step rejects. Also added publisher tests that a pre-model event returns while queued and a boundary event waits. release() now documents that it is cleanup only and never reports write failures.
Publish session lifecycle events without waiting for each durable stream round trip. The step drains the queued writes before it commits, so event order and failure handling stay the same. Signed-off-by: Chad Hietala <chad.hietala@vercel.com>
6ad367c to
8e580b9
Compare
Summary
Every session lifecycle event waited for its durable stream write before the turn continued. A turn’s
session.started,turn.started,message.received, andstep.startedwrites therefore sat between prompt preparation and the model call. In theeve-e2etraces,workflow.stream.writeandworkflow.stream.flushcost 55–130 ms p50. Session stream writes now queue in order, and the step drains them before it commits. Turn-boundary events keep the old contract: their write settles before their hooks and instrumentation run. The event protocol and dispatcher order do not change. A failed write still fails the step at the drain, and later writes fail fast. Workflow step lifecycle writes are unchanged.Validation
pnpm --filter eve exec vitest run --config vitest.unit.config.ts src/execution/publish-session-events.test.ts— 10 passedpnpm --filter eve exec vitest run --config vitest.unit.config.ts src/execution/session/turn-step.test.ts— 68 passedpnpm --filter eve exec vitest run --config vitest.integration.config.ts src/subagents/event-proxy-step.integration.test.ts— 12 passedpnpm --filter eve exec vitest run --config vitest.integration.config.ts src/execution/session/turn-connection-approval.integration.test.ts— 11 passedpnpm typecheck— 59 tasks passedpnpm guard:invariants— passedpnpm exec oxfmtandpnpm exec oxlinton the changed files — cleanNot run: hosted latency benchmark, full integration/scenario tiers, and e2e. The 55–130 ms figure is the existing OTel baseline, not a post-change measurement.
Checklist
evepackagegit commit --signoff)