Repository navigation
[FS] Reduce rounding errors when setting file timestamps - #27794
Conversation
| atime ||= st.atimeMs; | ||
| mtime ||= st.mtimeMs; | ||
| atime ??= st.atimeMs; | ||
| mtime ??= st.mtimeMs; |
There was a problem hiding this comment.
This is just a driveby fix to match libnodefs.js
8b0a332 to
d9af0ab
Compare
| var st = dontFollow ? fs.lstatSync(path) : fs.statSync(path); | ||
| atime ||= st.atimeMs; | ||
| mtime ||= st.mtimeMs; | ||
| atime ??= st.atimeMs; |
There was a problem hiding this comment.
Note we could of course avoid all lossiness by using statSync with bigint, although it only works for reads not writes in Node.js.
There was a problem hiding this comment.
Yes, I did a bit of deep dive into the cost/benfits of using the bigint version of the node stat APIs. Currently I think it doesn't add much sadly, but we could re-eval later.
d9af0ab to
f96b819
Compare
Avoid double-conversion loss when converting `struct timespec` in `__syscall_utimensat` and WasmFS by computing seconds directly as `seconds + (nanoseconds / 1e9)` rather than adding nanoseconds divided by 1,000,000 to large millisecond timestamps. Additionally, clamp sub-second timestamps near the 1-second boundary so that floating-point rounding cannot cause timestamps with `tv_nsec < 1_000_000_000` to round up to the next integer second. Also prefer `stat.atimeMs` over `stat.atime.getTime()` in `writeStat` when reading host `fs.Stats` in `NODERAWFS` to avoid millisecond quantization rounding near second boundaries. Fixes: emscripten-core#25280
f96b819 to
1b97901
Compare
In emscripten-core#27794 we clamped sub-second timestamps near the 1-second boundary to prevent double-precision rounding from incrementing the second. However, under Windows, Node.js (`libuv`) converts Unix timestamps to Windows `FILETIME` (100-nanosecond intervals since Jan 1, 1601 UTC): filetime = (time + 11644473600) * 10000000; Adding the 11.6 billion second epoch offset reduces floating-point precision (1 ULP becomes ~1.9 microseconds). When `tv_nsec` was close to 1e9 (e.g. 999,999,900 ns in `test_futimens`), adding the offset still rounded the result up into the next integer second on Windows. Instead of performing floating-point clamping near `maxSec`, round `tv_nsec` down to the nearest 10 microseconds (10,000 ns). This leaves sufficient headroom below the next whole second while preserving sub-millisecond timestamp precision. See: emscripten-core#25280
In #27794 we clamped sub-second timestamps near the 1-second boundary to prevent double-precision rounding from incrementing the second. However, under Windows, Node.js (`libuv`) converts Unix timestamps to Windows `FILETIME` (100-nanosecond intervals since Jan 1, 1601 UTC): filetime = (time + 11644473600) * 10000000; Adding the 11.6 billion second epoch offset reduces floating-point precision (1 ULP becomes ~1.9 microseconds). When `tv_nsec` was close to 1e9 (e.g. 999,999,900 ns in `test_futimens`), adding the offset still rounded the result up into the next integer second on Windows. Instead of performing floating-point clamping near `maxSec`, round `tv_nsec` down to the nearest 10 microseconds (10,000 ns). This leaves sufficient headroom below the next whole second while preserving sub-millisecond timestamp precision. See: #25280
|
This appears to sometimes set timestamps slightly in the past. I think it lead to a regression in the Python test suite: |
|
I will investigate this more tomorrow. |
`MEMFS.node_ops.getattr` returned the times only as `Date` objects, which hold whole milliseconds, so `stat()` truncated them even after `utimensat()` had stored a sub-millisecond value (#27794 keeps 10 µs). This also returns `atimeMs`/`mtimeMs`/`ctimeMs` with the exact values, as NODEFS does since #27787, and lets `SYSCALLS.writeStat` prefer the `*Ms` properties in every environment, not only when `ENVIRONMENT_MAY_BE_NODE` (MEMFS is used in the browser too). The `Date` properties stay, for JS users of `FS.stat`. The new test (`test_fs_subms_times`, all filesystems) sets a time with `utimensat()` and reads it back with `stat()` and `fstat()`; before this change only the MEMFS variant failed. Checked as well with `-sENVIRONMENT=web` in Chromium (`.123000000` before, `.123449951` after). Codesize: +54 bytes of JS where MEMFS is included; wasm unchanged. Not included: the second suggestion in the issue (stamping files with a sub-millisecond clock instead of `Date.now()`), which touches more places and changes behaviour; happy to do it as a follow-up if wanted. Fixes: #27885
This partially reverts #27794 and #27890. They break the Python test suite because it expects that if it calls `utimesat` and then `stat`, the time read back by `stat` is exactly identical to the time set by `utimesat`. I think this is a contradictory goal to the goal of test_fs_subms_times.c because node's `fs.utimesSync()` function takes a float argument so with sub-millisecond times, we get floating precision errors. On the other hand, it would be possible to support sub-millisecond times without breaking round trips for the memfs. This is a second attempt, #27896 was the first.
Avoid double-conversion loss when converting
struct timespecin__syscall_utimensatand WasmFS by computing seconds directly asseconds + (nanoseconds / 1e9)rather than adding nanoseconds dividedby 1,000,000 to large millisecond timestamps.
Additionally, clamp sub-second timestamps near the 1-second boundary
so that floating-point rounding cannot cause timestamps with
tv_nsec < 1_000_000_000to round up to the next integer second.Also prefer
stat.atimeMsoverstat.atime.getTime()inwriteStatwhen reading host
fs.StatsinNODERAWFSto avoid millisecondquantization rounding near second boundaries.
Fixes: #25280