[tnazarew]

Stacked on #352 (base ci/pr-coverage, for the pytest setup); retarget to main after #352 merges. Independent of #353.

Why

Every job after the components runs with if: !failure(). One failed component (a Dataproc cluster that doesn't start, a jar staging error) blocks the report, releases.json and the site for everything else; the next scheduled run then re-runs everything that changed. The release check failed on 5 of its last 10 scheduled runs (4 in a row, 2026-10-28 → 31).

What

• Parts: a part is a (component, component_version, openlineage_version) triple (dataplex/scenarios: empty versions).
• Plan: initialize_workflow writes the parts it's going to test from the effective matrices (planned-parts artifact). Pairs without a valid scenario are left out, since they can never produce a report (e.g. dbt 1.12.6 today: csv_to_postgres is gated to 1.8.0).
• Pending: collect-and-compare-reports (new input track-pending, release check only) marks planned parts without a report as pending in generated-files/pending_runs.json, with failed runs in a row and first/last failed run, and lists them in the job summary. A part that reports again is dropped.
• Retry: get_release_matrices.sh adds pending parts to the next run's matrices, so a failed part is retried on every scheduled run until it passes, even when no new release appeared. Matrices are cross products, so this can re-run a few extra pairs.
• Partial publishing: collect, notify, update-repo-files, tables and site run when their inputs succeeded (!cancelled() + needs.<job>.result == 'success') instead of only when nothing failed. A failed notification doesn't block publishing.
• publish input (manual runs, default true): false skips issues, report and site, to test pipeline changes from a branch.
• fail-fast: false on the Hive and dbt matrices (release, PR and spec-change workflows), so one failed pair doesn't cancel the others. Spark already set it.
• Removes the update_repo_files move of updated-new-report.json, which collect never uploads.

The failed job itself still makes the run red (and is what the upcoming pipeline alarm reports); after 3 failed runs in a row a part will also be escalated (follow-up PR).

Verification

• pytest scripts/tests → 43 passed: pending_runs.py plan/update/summary, and get_release_matrices.sh run against the repo's versions.json with a temporary pending file (4 of those failed before the script change).
• Local smoke test: planned dbt 1.8.0 × {1.55.0, 1.55.99} + dataplex, reports only for dbt 1.55.0 and dataplex → pending dbt 1.8.0 × 1.55.99 (failures 1); the next run's matrix is exactly that pair; a second failure → failures 2, first run kept.
• actionlint: no new findings.
• Release check dispatched from this branch with publish=false, dbt only, dbt_matrix={"component_version":["1.8.0"],"openlineage_versions":["1.55.0","1.55.99"]} (1.55.99 isn't published, so that pair fails):
  • run 42217294534: the 1.55.99 failure cancelled the 1.55.0 job (default fail-fast), so both ended pending. That's what the fail-fast: false commit fixes.
  • run 42217693825, after the fix: dbt 1.55.0 and scenarios were collected; the pending list holds only dbt 1.8.0 × 1.55.99 (1 failed run); notify, update-repo-files and site were skipped.

🤖 Generated with Claude Code