Skip to content

Harden the version and env marker handling against the specs and make it faster #19

Open

Description

@aignas

The PEP 440 version handling (python/private/version.bzl) and the env marker
evaluation (python/private/pypi/pep508_evaluate.bzl) are reimplementations of
what packaging does and they were only tested with hand written cases. A
differential check against packaging==26.3 found that neither followed the
specs for the less common inputs and that most of the evaluation time was
spent in the version parser.

This issue tracks the stack of PRs that fixes that.

The stack

The stack was rebuilt on the latest main and sent upstream, the fork PRs
#20 to #30 are closed. Each PR is stacked on the previous one:

  1. test(version.bzl): add conformance cases generated from pypa/packaging bazel-contrib/rules_python#4543: test(version.bzl): add conformance cases generated from pypa/packaging
  2. fix(version.bzl): compare versions as per the version specifiers spec bazel-contrib/rules_python#4544: fix(version.bzl): compare versions as per the version specifiers spec
  3. fix(version.bzl): reject invalid versions, normalize local segments bazel-contrib/rules_python#4545: fix(version.bzl): reject invalid versions, normalize local segments
  4. fix(pypi): return True for a satisfied partially evaluated marker bazel-contrib/rules_python#4546: fix(pypi): return True for a satisfied partially evaluated marker
  5. test: add benchmarks and a step counter for the Starlark code bazel-contrib/rules_python#4547: test: add benchmarks and a step counter for the Starlark code
  6. perf(version.bzl): parse the release-only versions without the parser bazel-contrib/rules_python#4548: perf(version.bzl): parse the release-only versions without the parser
  7. fix(pypi): evaluate the env markers like packaging does bazel-contrib/rules_python#4549: fix(pypi): evaluate the env markers like packaging does
  8. perf(pypi): parse the env markers once per requirement bazel-contrib/rules_python#4550: perf(pypi): parse the env markers once per requirement

What is tested

  • The inputs are harvested from the test-suite of the packaging sdist, which
    is a new dev dependency, and the expected values are either taken from there
    or computed with packaging itself. bazel run //tests/support/pypa_packaging:update regenerates the checked in
    packaging_cases.bzl files and up_to_date_test fails if they are stale.
  • Versions: 92 ordered versions (all pairs are compared), 20 invalid versions,
    101 normalization cases, 188 specifier cases and an operator matrix of
    243 valid operator and specifier combinations x 36 versions.
  • Markers: 24 evaluation cases, 9 invalid markers, a table of every
    variable x operator x value in both orientations (1080 rows, including the
    192 that are an error to evaluate) and 83 random and/or/parenthesis
    expressions.
  • Not checked in, but used to look for more bugs: random differential fuzzers
    against packaging for the versions and the markers and a round-trip check
    of the partial evaluation.

Behaviour changes to be aware of

  • version.is_eq(left, right) treats right as the specifier: the local
    version of left is only ignored if right does not have one and only
    right can use the V.* prefix matching.
  • version.key() returns a different tuple, it is still only meant to be
    compared with other keys. post and dev are ("post", N) and
    ("dev", N).
  • version.parse() (non-strict) no longer accepts .1, 1. or 1.*.* and
    returns None for the versions with placeholders.
  • platform_release is compared as a version and platform_version as a
    string, it was the other way around.
  • < and > on strings are always false and <= and >= check the
    equality.
  • Invalid markers fail instead of evaluating to something truthy. The
    variables that are not in the spec are invalid even if they are in the
    env. () is invalid.
  • The partial evaluation (strict = False) returns True instead of ""
    or () when the result does not depend on the missing variables.

Deliberate differences from packaging

  • The unary not <expr> is accepted.
  • Comparing two variables is an error, packaging compares the value of the
    left one with the name of the right one.
  • New lines in the strings are taken literally, packaging rejects them.
  • version.is_eeq compares the normalized strings, because the parsed version
    does not keep the original string. The marker evaluation compares the
    original strings.
  • Strings in Starlark are not unicode aware, so the unicode whitespace around
    the versions is not stripped.

Performance

See tests/perf/README.md in bazel-contrib#4549 for all of the
workloads. The steps are the executed lines of the .bzl files when they are
run with CPython, the time is measured with Bazel 9.1.1.

Workload Steps before Steps after Bazel before, µs Bazel after, µs
version_parse_release (3.11.4) 881 53 201.1 9.5
version_parse_pre_release (3.13.0rc1) 1016 1020 239.6 238.3
tokenize_long 4870 514 424.0 112.9
evaluate_version (python_version >= '3.8') 1975 283 432.7 56.4
evaluate_string (sys_platform == 'win32') 362 122 34.9 24.5
evaluate_long 11056 1974 2009.7 383.6
evaluate_long_parsed 11056 693 2009.7 114.1

Open design items

Activity

  1. self-assigned this
    on Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions