Skip to content

fix(windows): support long runfiles paths without LongPathsEnabled - #4572

Open
AustinSchuh wants to merge 2 commits into
bazel-contrib:mainfrom
AustinSchuh:fix-windows-no-symlinks-long-paths
Open

AustinSchuh wants to merge 2 commits into
bazel-contrib:mainfrom
AustinSchuh:fix-windows-no-symlinks-long-paths

Conversation

@AustinSchuh

Copy link
Copy Markdown
Contributor

DO NOT MERGE: awaiting #4571

Without LongPathsEnabled, which is off by default, Windows paths of
260 characters or more fail unless they have the \\?\ prefix. The
bootstrap skipped the prefix on Windows 10 1607 and later, so long
runfiles paths broke binaries that can't create symlinks (#4509):

  • The fallback fails to hard link or copy runfiles files
    (WinError 3).
  • os.path.exists() is False for a long Lib path, so the runtime
    venv silently has no site-packages.
  • The RUNFILES_DIR check fails, so it's cleared, and the site init
    then uses C:\ as the runfiles root and puts every directory in
    C:\ on sys.path.

This adds the prefix to paths that are long enough to need it, on any
Windows version, and uses it in the fallback, the venv setup and the
runfiles root checks. Short paths are unchanged. It also fixes a
NameError in the stage 2 copy of the helper, which never defined
unicode_prefix.

Note for review: this applies to all Windows users, not only those
without symlinks. Also, _symlink_tree now walks the runfiles venv
with the prefix, so on machines that can create symlinks, the venv's
symlinks store \\?\ targets.

Tested on Windows 11 26H2 with Developer Mode off and
LongPathsEnabled=0, Bazel 9.1.1, --nowindows_enable_symlinks:

  • run_binary_without_symlinks_test,
    cached_venv_replaces_dangling_package_symlink_test and
    abi3_headers_linkage_test_py fail without this and pass with it.
  • The new windows_long_paths_test fails without this and passes
    with it.
  • //tests/bootstrap_impls/..., //tests/cc/... and
    //tests/runfiles/... pass on Linux.

Two other failures without symlinks are Bazel issues and aren't fixed
here:

  • On a clean build, runfiles entries for .exe, .dll and .pyd
    outputs are junctions to files, which can't be opened.
    --nobuild_runfile_links avoids it.
  • Copied runfiles aren't refreshed when only their contents change.

…symlinks

`test_replaces_dangling_package_link_before_creating_child_link` needs to
create symlinks, so it fails on Windows without the privilege to create
them. Skip it there. The junction case still runs.
Without `LongPathsEnabled`, which is off by default, Windows paths of 260
characters or more fail unless they have the `\\?\` prefix. The bootstrap
skipped the prefix on any Windows 10 1607 or later, so long runfiles paths
broke binaries that can't create symlinks:

* The fallback for when symlinks can't be created opens the runfiles
  files to hard link or copy them, which fails with `WinError 3`.
* `os.path.exists()` is False for a long `Lib` path, so the runtime venv
  silently had no `site-packages`.
* The check of `RUNFILES_DIR` failed, so it was cleared, and the site
  init then used `C:\` as the runfiles root and put every directory in
  `C:\` on `sys.path`.

Add the prefix to paths that are long enough to need it, on any Windows
version, and use it in the fallback, the venv setup and the runfiles root
checks. Short paths are unchanged. This also fixes a `NameError` in the
stage 2 copy of the helper, which never defined `unicode_prefix`.

Share the bootstrap loader of `cached_venv_dangling_symlink_test` with a
new test for long paths.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant