Skip to content

[FS] Reduce rounding errors when setting file timestamps - #27794

Merged
sbc100 merged 1 commit into
emscripten-core:mainfrom
sbc100:fix_utimens_rounding
Sep 29, 2026
Merged

sbc100 merged 1 commit into
emscripten-core:mainfrom
sbc100:fix_utimens_rounding

Conversation

@sbc100

@sbc100 sbc100 commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Avoid double-conversion loss when converting struct timespec in
__syscall_utimensat and WasmFS by computing seconds directly as
seconds + (nanoseconds / 1e9) rather than adding nanoseconds divided
by 1,000,000 to large millisecond timestamps.

Additionally, clamp sub-second timestamps near the 1-second boundary
so that floating-point rounding cannot cause timestamps with
tv_nsec < 1_000_000_000 to round up to the next integer second.

Also prefer stat.atimeMs over stat.atime.getTime() in writeStat
when reading host fs.Stats in NODERAWFS to avoid millisecond
quantization rounding near second boundaries.

Fixes: #25280

@sbc100
sbc100 requested review from hoodmane and kripken September 29, 2026 00:33
Comment thread src/lib/libnoderawfs.js
atime ||= st.atimeMs;
mtime ||= st.mtimeMs;
atime ??= st.atimeMs;
mtime ??= st.mtimeMs;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a driveby fix to match libnodefs.js

@sbc100
sbc100 enabled auto-merge (squash) September 29, 2026 02:02
@sbc100
sbc100 requested a review from guybedford September 29, 2026 02:38
@sbc100
sbc100 force-pushed the fix_utimens_rounding branch from 8b0a332 to d9af0ab Compare September 29, 2026 02:44
Comment thread src/lib/libnoderawfs.js
var st = dontFollow ? fs.lstatSync(path) : fs.statSync(path);
atime ||= st.atimeMs;
mtime ||= st.mtimeMs;
atime ??= st.atimeMs;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note we could of course avoid all lossiness by using statSync with bigint, although it only works for reads not writes in Node.js.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I did a bit of deep dive into the cost/benfits of using the bigint version of the node stat APIs. Currently I think it doesn't add much sadly, but we could re-eval later.

@sbc100
sbc100 force-pushed the fix_utimens_rounding branch from d9af0ab to f96b819 Compare September 29, 2026 03:20
Avoid double-conversion loss when converting `struct timespec` in
`__syscall_utimensat` and WasmFS by computing seconds directly as
`seconds + (nanoseconds / 1e9)` rather than adding nanoseconds divided
by 1,000,000 to large millisecond timestamps.

Additionally, clamp sub-second timestamps near the 1-second boundary
so that floating-point rounding cannot cause timestamps with
`tv_nsec < 1_000_000_000` to round up to the next integer second.

Also prefer `stat.atimeMs` over `stat.atime.getTime()` in `writeStat`
when reading host `fs.Stats` in `NODERAWFS` to avoid millisecond
quantization rounding near second boundaries.

Fixes: emscripten-core#25280
@sbc100
sbc100 force-pushed the fix_utimens_rounding branch from f96b819 to 1b97901 Compare September 29, 2026 03:23
@sbc100
sbc100 merged commit 4367558 into emscripten-core:main Sep 29, 2026
42 checks passed
@sbc100
sbc100 deleted the fix_utimens_rounding branch September 29, 2026 04:24
sbc100 added a commit to sbc100/emscripten that referenced this pull request Sep 29, 2026
In emscripten-core#27794 we clamped sub-second timestamps near the 1-second boundary
to prevent double-precision rounding from incrementing the second.
However, under Windows, Node.js (`libuv`) converts Unix timestamps to
Windows `FILETIME` (100-nanosecond intervals since Jan 1, 1601 UTC):

  filetime = (time + 11644473600) * 10000000;

Adding the 11.6 billion second epoch offset reduces floating-point
precision (1 ULP becomes ~1.9 microseconds). When `tv_nsec` was close
to 1e9 (e.g. 999,999,900 ns in `test_futimens`), adding the offset
still rounded the result up into the next integer second on Windows.

Instead of performing floating-point clamping near `maxSec`, round
`tv_nsec` down to the nearest 10 microseconds (10,000 ns). This leaves
sufficient headroom below the next whole second while preserving
sub-millisecond timestamp precision.

See: emscripten-core#25280
sbc100 added a commit that referenced this pull request Sep 29, 2026
In #27794 we clamped sub-second timestamps near the 1-second boundary
to prevent double-precision rounding from incrementing the second.
However, under Windows, Node.js (`libuv`) converts Unix timestamps to
Windows `FILETIME` (100-nanosecond intervals since Jan 1, 1601 UTC):

  filetime = (time + 11644473600) * 10000000;

Adding the 11.6 billion second epoch offset reduces floating-point
precision (1 ULP becomes ~1.9 microseconds). When `tv_nsec` was close
to 1e9 (e.g. 999,999,900 ns in `test_futimens`), adding the offset
still rounded the result up into the next integer second on Windows.

Instead of performing floating-point clamping near `maxSec`, round
`tv_nsec` down to the nearest 10 microseconds (10,000 ns). This leaves
sufficient headroom below the next whole second while preserving
sub-millisecond timestamp precision.

See: #25280
@hoodmane

hoodmane commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

This appears to sometimes set timestamps slightly in the past. I think it lead to a regression in the Python test suite:
https://github.com/python/cpython/actions/runs/37066433870/job/111585512190?pr=158627
I haven't looked into why this happens though.

@hoodmane

hoodmane commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

I will investigate this more tomorrow.

sbc100 pushed a commit that referenced this pull request Oct 6, 2026
`MEMFS.node_ops.getattr` returned the times only as `Date` objects,
which hold whole milliseconds, so `stat()` truncated them even after
`utimensat()` had stored a sub-millisecond value (#27794 keeps 10 µs). 
This also returns `atimeMs`/`mtimeMs`/`ctimeMs` with the exact values,
as NODEFS does since #27787, and lets `SYSCALLS.writeStat` prefer the
`*Ms` properties in every environment, not only when 
`ENVIRONMENT_MAY_BE_NODE` (MEMFS is used in the
browser too). The `Date` properties stay, for JS users of `FS.stat`.

The new test (`test_fs_subms_times`, all filesystems) sets a time with
`utimensat()` and reads it back with `stat()` and `fstat()`; before this
change only the MEMFS variant failed. Checked as well with
`-sENVIRONMENT=web` in Chromium (`.123000000` before, `.123449951`
after).

Codesize: +54 bytes of JS where MEMFS is included; wasm unchanged.

Not included: the second suggestion in the issue (stamping files with a
sub-millisecond clock instead of `Date.now()`), which touches more
places and changes behaviour; happy to do it as a follow-up if wanted.

Fixes: #27885
sbc100 pushed a commit that referenced this pull request Oct 14, 2026
This partially reverts #27794 and #27890. They break the Python test
suite because it expects that if it calls `utimesat` and then `stat`,
the time read back by `stat` is exactly identical to the time set by
`utimesat`. I think this is a contradictory goal to the goal of
test_fs_subms_times.c because node's `fs.utimesSync()` function takes a
float argument so with sub-millisecond times, we get floating precision
errors. On the other hand, it would be possible to support
sub-millisecond times without breaking round trips for the memfs.

This is a second attempt, #27896 was the first.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_futimens is flaky on ARM64 macOS.

3 participants