Repository navigation
Conversation
The extra_dep target depends on the _extras target of a package with the requested extra set in a private build setting and the extra_select target selects the target of that extra and resets the setting, so that the targets of the extras are analyzed in the same configuration. The extras that the package does not provide fall back to the package itself, like pip.
The dependencies on the extras of other packages from the wheel METADATA, e.g. 'Requires-Dist: bar[socks]', used the '[socks]' target of the hub, which exists only if the extra is requested in the lock file, so the build failed if the lock file did not have the extra. Each whl repository now has an '_extras' target that selects the target of the extra requested via an extra_dep target, and the hub forwards it to the whl repository of the target platform. The extras that the package does not provide fall back to the package, like pip does.
…ries The whl repositories resolve the extras of their dependencies without the hub, so the config.bzl of the hub lists only the packages again and the missing_extra_error rule is removed. This means that the BUILD files of the whl repositories do not change when the extras in the lock file change.
…locked Also check that the extras that a package does not provide are ignored in the whl_library integration test.
Collaborator
Author
|
This draft PR was drafted by a Claude Code agent as a design proposal for wiring the extras of the dependencies without relying on the hub repositories or on the lock file containing the extras. The human author (@aignas) handles the review comments on this PR. This is an agent-generated comment, automatically generated. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With #4490 the dependencies on the extras of other packages from the wheel
METADATA, e.g.Requires-Dist: bar[socks], use the@pypi//bar:[socks]target of the hub. The hub only has the targets for the extras that are in the
lock file, so the hub passes the available extras to the whl repositories via
config.bzland the build fails if the extra is not locked. However, lockfiles often do not have the extras (e.g.
uv pip compileandpip-compilestrip them by default), even though the packages of the extras are locked,
and the BUILD files of all of the whl repositories change whenever the extras
in the lock file change.
The extras of a package are only known from its own
METADATA, so with thischange the whl repository of the package resolves them:
foothat depends onbar[socks]creates anextra_deptarget that depends on@pypi//bar:_extraswith a privaterequested_extrabuild setting set tosocksvia a transition.barcreates anextra_selecttarget named_extrasthat selects its
[socks]target based on the setting and resets thesetting on that edge, so that the targets of the extras and their
dependencies are analyzed in the same configuration as the rest of the
build. Only the
_extrastarget itself is in the transitionedconfiguration.
_extrasto the whl repository of the targetplatform, like the other per-package aliases, so it does not need to know
the extras that the packages provide.
itself, which is what
pipdoes, somissing_extra_errorand theavailable extras in
config.bzlare removed.Only the selected extra is analyzed, so the cycles are the same as with the
direct dependencies and the group members with extras from #4490 keep working
because the members are selected via the group library in the hub.
Considered alternatives:
[extra]targets in the hub for every extra that the packageprovides. This needs the
METADATAat hub creation time, i.e. fetching allof the wheels (or PEP 658 metadata) eagerly, including the ones for other
platforms.
provider keyed by the extra. This analyzes and fetches the dependencies of
every extra of every package reached through an extra and introduces
spurious analysis cycles.
reading the
METADATAof the dependency. This makes the whl repositoriesdepend on each other at fetch time and requires combining the markers.
missing_package_errorstyle message (the currentfeat(pypi): add targets for the extras of the packages #4490 behaviour) still requires the extras to be in the lock file.
The
[extra]targets of the hub for the extras requested in the lock file areunchanged.
Stacked on #4490.
Work towards #2786