Skip to content

feat(pypi): add targets for the extras of the packages - #4490

Draft
aignas wants to merge 23 commits into
bazel-contrib:mainfrom
aignas:feat.aignas.target-per-extra
Draft

aignas wants to merge 23 commits into
bazel-contrib:mainfrom
aignas:feat.aignas.target-per-extra

Conversation

@aignas

@aignas aignas commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

The requested extras of a distribution were passed to its whl repository
via the requirement line, so changing the extras refetched the repository, the
same wheel requested with different extras on different platforms needed
separate repositories, and the extras of the dependencies written in the
wheel METADATA (e.g. Requires-Dist: coverage[toml]) were ignored, silently
dropping their dependencies.

With this change the whl repositories create an [<extra>] target for every
extra in the Provides-Extra METADATA and the requested extras are no longer
passed to them. The hub exposes the extras requested in the lock file, which
follows the rules_pycross convention:

@pypi//google_api_core               # The package with the requested extras
@pypi//google_api_core:[]            # The package without any extras
@pypi//google_api_core:[async_rest]  # The package with the async_rest extra
@pypi//google_api_core:[grpc]        # The package with the grpc extra
  • The requirement macro accepts a single extra, e.g.
    requirement("google-api-core[grpc]"), and the new requirements macro
    returns a list of labels for any number of packages and extras, e.g.
    requirements("google-api-core[async_rest,grpc]", "numpy").
  • The same distribution requested with different extras on different
    platforms uses a single whl repository and the hub selects the extras per
    platform.
  • The dependencies use the extras from the wheel METADATA. If an extra is
    missing in the lock file, building the target fails with a message asking
    to add it to the requirements or the lock file.
  • The whl targets do not use the extras.
  • The members of experimental_requirement_cycles can be extras, e.g.
    foo[bar], so that only the targets of that extra are part of the group.
  • WORKSPACE pip_parse behaves the same as pip.parse.
  • The requirement line is only passed to the repositories that use pip.

Fixes #2786
Fixes #4168

@aignas
aignas requested a review from rickeylev as a code owner November 13, 2026 08:59
@aignas

aignas commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

This PR was written by an AI coding agent (Claude Code) working under the direction of @aignas, who reviewed the design and the changes. Review comments will be handled by @aignas, possibly with the help of the agent; any agent-drafted replies or follow-up commits are reviewed by @aignas before being posted. An agent monitors CI only to retry flaky network failures.

This comment was generated by an AI agent and reviewed by the user.

@aignas

aignas commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

FYI, AI is warning

BREAKING CHANGE: The dependencies on the extras of other packages from the
wheel METADATA are now honoured and the build fails with a helpful message
if such an extra is missing in the lock file, e.g. when the requirements are
compiled without --no-strip-extras. The whl targets no longer include the
dependencies of the extras.

However, I am sure that without --no-strip-extras we would fail, because the whl_library would not know what to do. So I am disregarding this as being too cautious.

@aignas aignas changed the title feat(pypi)!: add targets for the extras of the packages feat(pypi): add targets for the extras of the packages Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support extras and dependency_groups in the PEP508 env marker evaluation pip.parse drops extras declared on the wheel's dependency (package[extra])

1 participant