You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I can confirm that the difference in performance is around 6x:
listless% hyperfine ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc rustc
Benchmark #1: /home/dsilvers/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc
Time (mean ± σ): 11.3 ms ± 0.9 ms [User: 6.9 ms, System: 4.3 ms]
Range (min … max): 10.1 ms … 15.4 ms 252 runs
Benchmark #2: rustc
Time (mean ± σ): 71.2 ms ± 2.3 ms [User: 60.0 ms, System: 11.2 ms]
Range (min … max): 67.3 ms … 76.2 ms 41 runs
Summary
'/home/dsilvers/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc' ran
6.31 ± 0.54 times faster than 'rustc'
listless%
Looking locally, ca. 15ms are before main() is entered (presumably due to dynamic link time for all the libraries rustup uses by dint of being linked to curl).
It looks like about 25ms is taken parsing manifest files, sadly this is also done twice, and thus accounts for the vast majority of the extra time. The toml parser must be quite inefficient.
It might be possible to knock one of those parse steps on the head since in theory we only need to parse once, but it'll involve quite a bit of refactoring.
If someone wants to begin to look at this, then the place to start is the config.rs file, around line 620 which is find_or_install_override_toolchain_or_default(). In the sub-function components_exist() you can see it acquire the manifest (since it needs it) and then it lists the components in order to check that everything needed is installed. The list_components() function though ends up loading the manifest itself. Perhaps the distributable toolchain could cache the manifest having loaded it?
To help triage this, how much does this contribute to typical build times? like, - sure we can put a chunk of time into making this snappy, and that would be good, but if a typical build is 1500ms ... ?
@rbtcollins I wouldn't notice so much on builds, but this hurts things like rustc --explain E0111 or rustc --help.
$ time /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --help >/dev/null
real 0m0.050s
user 0m0.026s
sys 0m0.020s
$ time rustc --help >/dev/null
real 0m0.157s
user 0m0.129s
sys 0m0.028s
$ time pip --help >/dev/null
real 0m0.206s
user 0m0.178s
sys 0m0.028s
$ time go help >/dev/null
real 0m0.022s
user 0m0.008s
sys 0m0.017s
time clang >&/dev/null
clang >&/dev/null 0,04s user 0,03s system 96% cpu 0,064 total
time gcc >& /dev/null
gcc >&/dev/null 0,00s user 0,01s system 86% cpu 0,009 total
Comparing with C compilers is disingenuous since the number of rustc invocations for a similarly sized codebase will be many fewer. For example, rustup is around 130 .rs files, but including its dependencies the total rustc invocations is only 320 or so. If we assume a wasted 35ms (the repeated manifest parse) of CPU time on my computer per rustc invocation and we generously assume that all that waste is reflected in the wallclock compile time at the same parallelism that the build occurs at, then of the 83s it took my computer to build rustup just now at 6.1x parallel, then that's 11375ms wasted which at 6.1x parallel is 1.864s of wallclock time or 2.2% of all build time in a debug build.
The same maths on a release build, 11375ms wasted for a total build time of 195s is only 0.95% of the total runtime of the build.
Considering the rustc --explain or rustc --help scenarios is also disingenuous. Most humans don't need to run those repeatedly in tight loops, and will take far longer than the wasted 50ms or so even noticing the screen updated let alone reading and grokking the output.
however
Having said all that, I do think it's worth correcting the double-load of the manifest since it's wasteful to do that anyway, and that shouldn't be too complex to do in a grungy way; or only medium hard to do nicely.
What would be of even more use though is working out how it takes 35ms on a 4GHz CPU core to parse the manifest, that's utterly abysmal.
The majority of the time spent parsing the manifest is in the toml load itself, i.e. before we apply any semantic understanding to the manifest. The channel is around 700k and will only get larger over time. Another possible improvement to do would be to trim out anything not relevant to the toolchain being installed when fetching the manifest. That ought not to be too complex to do and would speed up future operations since the manifest would be significantly smaller
I wrote a simple trimmer for the manifests to reduce the toml quantity in installed toolchains. Doing that results in the following (stable is untrimmed, beta is trimmed):
listless% echo stable > rust-toolchain
listless% hyperfine rustc
Benchmark #1: rustc
Time (mean ± σ): 69.8 ms ± 1.9 ms [User: 59.5 ms, System: 10.2 ms]
Range (min … max): 67.4 ms … 76.1 ms 42 runs
listless% echo beta > rust-toolchain
listless% hyperfine rustc
Benchmark #1: rustc
Time (mean ± σ): 19.6 ms ± 0.7 ms [User: 14.2 ms, System: 5.4 ms]
Range (min … max): 18.7 ms … 22.9 ms 128 runs
listless%
(for reference, directly invoking rustc (stable or beta) without rustup in the way is ca. 11ms.
Reacted by jyn, Matthias Krüger, Mikail Bagishov, Francis Lalonde and Jakub Beránek
So to ask the obvious first, do we need to load any metadata while in proxy mode? It's just to validate the components are installed right? Which feels more like the job of rustup proper so the proxies can just assume everything works.
So to ask the obvious first, do we need to load any metadata while in proxy mode? It's just to validate the components are installed right? Which feels more like the job of rustup proper so the proxies can just assume everything works.
@ChrisDenton Wait, IIRC ensure_active_toolchain() was exactly the call that we brought back in v1.28.1 (and later renamed in v1.28.2) to avoid the previous breakage (and the same call that made a whole bunch of query-only functions unnecessarily async), and in that very context, this call should be necessary...
Wait, IIRC ensure_active_toolchain() was exactly the call that we brought back in v1.28.1 (and later renamed in v1.28.2) to avoid the previous breakage, and in the very context of that breakage, this call should be necessary...
Ah right, that's a pain. So yeah persuading people to disable auto-install is the best call. 😉
My next thought would be to encode the essential information in a super simple binary format that's fast to load into memory.
Can we encode just the list of installed components in a binary file and only validate this file when running in proxy mode? And then leave any consistency checks that need to parse the full manifest to directly running rustup or when the proxy detects that a component is missing.
So to ask the obvious first, do we need to load any metadata while in proxy mode? It's just to validate the components are installed right? Which feels more like the job of rustup proper so the proxies can just assume everything works.
@ChrisDenton Wait, IIRC ensure_active_toolchain() was exactly the call that we brought back in v1.28.1 (and later renamed in v1.28.2) to avoid the previous breakage (and the same call that made a whole bunch of query-only functions unnecessarily async), and in that very context, this call should be necessary...
if !should_ensure {
return self.active_toolchain();
}
match self.ensure_active_toolchain(true, false).await { @Kobzol This makes me wonder, what will the graph look like if the new behavior is used instead (RUSTUP_AUTO_INSTALL=0)?
With that env flag, rustc takes ~22ms, instead of ~48ms, so the manifest loading seems to be removed in that case.
@Kobzol Thanks! So setting RUSTUP_AUTO_INSTALL=0 gives a reasonable workaround then.
It is also worth mentioning that with auto installation enabled, rustc --version can take up to minutes to complete (that is, when the active toolchain is missing), so it's absolutely bad news for anyone who wish this command could run faster.
Are there plans to flip the default in the future, or is it expected that the auto-install behavior will stay the default? If it is the latter, I think that it could still be worth it to optimize the manifest loading.
I personally would like to move towards it being the default but we tried that in 1.28.0 and it was not well received. So it'll likely we'll support at least some (perhaps more limited) form of auto-install for awhile yet.
So, speaking for myself, I think it may be worth optimising the happy path if someone wants to work on that. But other team members may have other opinions.
Are there plans to flip the default in the future, or is it expected that the auto-install behavior will stay the default? If it is the latter, I think that it could still be worth it to optimize the manifest loading.
I would say there are plans to flip the default, but I think none of us are currently very motivated to push the issue forward.
I tried to switch the loading to JSON, and with that running ./rustup which rustc takes ~20ms on my machine (vs ~40ms previously), out of which a large portion is building the regex for tracing-subscriber, which should be fixed with tokio-rs/tracing#3243 (to be backported in tokio-rs/tracing#3283). I noticed that there was already an attempt to optimize this in #2627, but it wasn't merged, so not sure how this will work out, but anyway - here's the PR: #4344.
#4350 should completely skip manifest parsing in most cases (unless there is a rust-toolchain.toml file). That reduces the overhead considerably, and with tokio-rs/tracing#3243 I think that it will be practically negligible for most rustup proxy invocations.
Reacted by jyn, Weihang Lo, rami3l, Mateusz Mikuła and stefnotch
Performance on my PC (64-bit Linux) with Rustup 1.29.0:
hyperfine "~/.cargo/bin/rustc" "~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc"
Benchmark 1: ~/.cargo/bin/rustc
Time (mean ± σ): 15.5 ms ± 0.5 ms [User: 4.8 ms, System: 11.6 ms]
Range (min … max): 14.2 ms … 17.3 ms 181 runs
Benchmark 2: ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc
Time (mean ± σ): 9.8 ms ± 0.3 ms [User: 2.6 ms, System: 7.5 ms]
Range (min … max): 8.7 ms … 10.9 ms 281 runs
Summary
~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc ran
1.58 ± 0.07 times faster than ~/.cargo/bin/rustc
So much better than the original number when this issue was created.
Activity
jyn514 commentedon Jan 1, 2021
@matthiaskrgr this is about the rustup wrapper, not the x.py rustc shim, right? If so I think this should be moved to https://github.com/rust-lang/rustup/.
matthiaskrgr commentedon Jan 1, 2021
Ah, ~/.cargo/bin/rustc is rustup? Yes, the issue should be moved then. :)
kinnison commentedon Jan 1, 2021
I can confirm that the difference in performance is around 6x:
kinnison commentedon Jan 1, 2021
Looking locally, ca. 15ms are before main() is entered (presumably due to dynamic link time for all the libraries rustup uses by dint of being linked to curl).
It looks like about 25ms is taken parsing manifest files, sadly this is also done twice, and thus accounts for the vast majority of the extra time. The toml parser must be quite inefficient.
It might be possible to knock one of those parse steps on the head since in theory we only need to parse once, but it'll involve quite a bit of refactoring.
If someone wants to begin to look at this, then the place to start is the config.rs file, around line 620 which is
find_or_install_override_toolchain_or_default(). In the sub-functioncomponents_exist()you can see it acquire the manifest (since it needs it) and then it lists the components in order to check that everything needed is installed. Thelist_components()function though ends up loading the manifest itself. Perhaps the distributable toolchain could cache the manifest having loaded it?rbtcollins commentedon Jan 1, 2021
To help triage this, how much does this contribute to typical build times? like, - sure we can put a chunk of time into making this snappy, and that would be good, but if a typical build is 1500ms ... ?
jyn514 commentedon Jan 1, 2021
@rbtcollins I wouldn't notice so much on builds, but this hurts things like
rustc --explain E0111orrustc --help.matthiaskrgr commentedon Jan 1, 2021
to addd to the list:
kinnison commentedon Jan 2, 2021
@matthiaskrgr
Comparing with C compilers is disingenuous since the number of rustc invocations for a similarly sized codebase will be many fewer. For example, rustup is around 130 .rs files, but including its dependencies the total rustc invocations is only 320 or so. If we assume a wasted 35ms (the repeated manifest parse) of CPU time on my computer per rustc invocation and we generously assume that all that waste is reflected in the wallclock compile time at the same parallelism that the build occurs at, then of the 83s it took my computer to build rustup just now at 6.1x parallel, then that's 11375ms wasted which at 6.1x parallel is 1.864s of wallclock time or 2.2% of all build time in a debug build.
The same maths on a release build, 11375ms wasted for a total build time of 195s is only 0.95% of the total runtime of the build.
@jyn514
Considering the
rustc --explainorrustc --helpscenarios is also disingenuous. Most humans don't need to run those repeatedly in tight loops, and will take far longer than the wasted 50ms or so even noticing the screen updated let alone reading and grokking the output.however
Having said all that, I do think it's worth correcting the double-load of the manifest since it's wasteful to do that anyway, and that shouldn't be too complex to do in a grungy way; or only medium hard to do nicely.
What would be of even more use though is working out how it takes 35ms on a 4GHz CPU core to parse the manifest, that's utterly abysmal.
kinnison commentedon Jan 2, 2021
Further investigation yields:
The majority of the time spent parsing the manifest is in the toml load itself, i.e. before we apply any semantic understanding to the manifest. The channel is around 700k and will only get larger over time. Another possible improvement to do would be to trim out anything not relevant to the toolchain being installed when fetching the manifest. That ought not to be too complex to do and would speed up future operations since the manifest would be significantly smaller
kinnison commentedon Jan 2, 2021
I wrote a simple trimmer for the manifests to reduce the toml quantity in installed toolchains. Doing that results in the following (stable is untrimmed, beta is trimmed):
(for reference, directly invoking rustc (stable or beta) without rustup in the way is ca. 11ms.
kinnison commentedon Jan 2, 2021
@matthiaskrgr Assuming #2627 goes CI-green, you might want to try the binary from there and see how it affects performance for you.
30 remaining items
ChrisDenton commentedon May 21, 2025
So to ask the obvious first, do we need to load any metadata while in proxy mode? It's just to validate the components are installed right? Which feels more like the job of rustup proper so the proxies can just assume everything works.
rami3l commentedon May 21, 2025
@ChrisDenton Wait, IIRC
ensure_active_toolchain()was exactly the call that we brought back in v1.28.1 (and later renamed in v1.28.2) to avoid the previous breakage (and the same call that made a whole bunch of query-only functions unnecessarily async), and in that very context, this call should be necessary...rustup/src/config.rs
Lines 534 to 538 in ac249c8
@Kobzol This makes me wonder, what will the graph look like if the new behavior is used instead (
RUSTUP_AUTO_INSTALL=0)?ChrisDenton commentedon May 21, 2025
Ah right, that's a pain. So yeah persuading people to disable auto-install is the best call. 😉
My next thought would be to encode the essential information in a super simple binary format that's fast to load into memory.
bjorn3 commentedon May 21, 2025
Can we encode just the list of installed components in a binary file and only validate this file when running in proxy mode? And then leave any consistency checks that need to parse the full manifest to directly running rustup or when the proxy detects that a component is missing.
Kobzol commentedon May 21, 2025
With that env flag,
rustctakes ~22ms, instead of ~48ms, so the manifest loading seems to be removed in that case.rami3l commentedon May 21, 2025
@Kobzol Thanks! So setting
RUSTUP_AUTO_INSTALL=0gives a reasonable workaround then.It is also worth mentioning that with auto installation enabled,
rustc --versioncan take up to minutes to complete (that is, when the active toolchain is missing), so it's absolutely bad news for anyone who wish this command could run faster.ChrisDenton commentedon May 21, 2025
Or
rustup set auto-install disablefor people who prefer not to use environment variables.Kobzol commentedon May 21, 2025
Are there plans to flip the default in the future, or is it expected that the auto-install behavior will stay the default? If it is the latter, I think that it could still be worth it to optimize the manifest loading.
ChrisDenton commentedon May 21, 2025
I personally would like to move towards it being the default but we tried that in 1.28.0 and it was not well received. So it'll likely we'll support at least some (perhaps more limited) form of auto-install for awhile yet.
So, speaking for myself, I think it may be worth optimising the happy path if someone wants to work on that. But other team members may have other opinions.
djc commentedon May 21, 2025
I would say there are plans to flip the default, but I think none of us are currently very motivated to push the issue forward.
Kobzol commentedon May 22, 2025
I tried to switch the loading to JSON, and with that running
./rustup which rustctakes ~20ms on my machine (vs ~40ms previously), out of which a large portion is building the regex fortracing-subscriber, which should be fixed with tokio-rs/tracing#3243 (to be backported in tokio-rs/tracing#3283). I noticed that there was already an attempt to optimize this in #2627, but it wasn't merged, so not sure how this will work out, but anyway - here's the PR: #4344.Kobzol commentedon May 27, 2025
#4350 should completely skip manifest parsing in most cases (unless there is a
rust-toolchain.tomlfile). That reduces the overhead considerably, and with tokio-rs/tracing#3243 I think that it will be practically negligible for most rustup proxy invocations.Kobzol commentedon Sep 3, 2025
#4405 should improve the speed of loading TOML files, even if you have a
rust-toolchain.tomlfile.[-]improve rustc wrapper startup time?[/-][+]improve `proxy_mode` (e.g. rustc wrapper) startup time?[/+]Kobzol commentedon Jun 15, 2026
Performance on my PC (64-bit Linux) with Rustup 1.29.0:
So much better than the original number when this issue was created.