Skip to content

feat(logs)!: require canonical episode identity and report coverage - #70277

Merged
pelikhan merged 7 commits into
mainfrom
pelikhan-glowing-adventure
Oct 19, 2026
Merged

pelikhan merged 7 commits into
mainfrom
pelikhan-glowing-adventure

Conversation

@pelikhan

@pelikhan pelikhan commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Require recorded canonical episode identity and distinguish recorded root claims from selected-run representatives. The canonical-only revision follows the explicit request to remove legacy episode implementation; this is no longer an additive compatibility-only change.

  • Group episodes only by nonempty recorded context.episode_id. Remove synthetic standalone/call IDs, heuristic workflow-call/workflow-run reconstruction, legacy bridging and inferred grouping. Runs without canonical IDs remain in .runs[] and overall usage totals but do not produce episode rollups or edges.
  • Remove episode-level root_run_id and primary_workflow. Use representative_run_id and representative_workflow for selected-data identity; use recorded_roots[] for distinct recorded root run_id, workflow_id, repository and run-level observed evidence. Missing roots are never silently replaced with descendants.
  • Require matching canonical episode IDs for recorded run-to-run edges. Preserve same-run reusable hops without creating self-edges or per-hop graph nodes. Retain all canonical episodes, including single-run episodes, in compact JSON.
  • Expose coverage with scope: selected_runs, status: unknown | partial, deterministic missing/invalid/conflicting/unverified-lineage reasons, sorted unique missing_parent_run_ids, and collection_truncated from existing continuation/count-limit inputs.
  • Wire verbose rendering and generated logs schema; recompute coverage from cached run context for each selection rather than caching selection-specific conclusions. Update cost-management, CLI and glossary documentation, changelog and a major changeset with migration guidance.

Coverage is deliberately never complete. Observing all recorded roots/parents, finding no gaps, or exhausting a cursor cannot establish that unselected descendants, missing context or same-run reusable hops are absent. Collection truncation describes bounded collection evidence, not proof of additional members in any particular episode. Lineage confidence remains independent of coverage. The default count is still 10 even with date filters. No crawler, per-hop graph, lifecycle store or outcome dependency is introduced.

ADR: ADR-70277: Episode Root Identity and Selected-Run Coverage Evidence

Breaking changes / migration

Replace episode-level root_run_id / primary_workflow lookups with representative_run_id / representative_workflow when selected-run identity is intended, or inspect recorded_roots[] for recorded logical-root claims. Consumers must no longer expect synthetic episode IDs or episode coverage for runs without recorded canonical IDs. Shared run-context fields and ordinary per-run/overall usage reporting remain available.

Review fixes

  • Preserve unlinked selected roots as unobserved and explicitly unverified, not conflicting. Same-run reusable callers now undergo repository validation without creating self-edges.
  • Compare GitHub repository identities case-insensitively while preserving original recorded root claims; retain and test workflow-only, repository-only and contradictory partial root evidence.
  • Use one collection-coverage authority for direct report builders and prepared reports. Scope continuations and count bounds by selected member repository/workflow pairs, including cross-repository episodes, cursorless limits and mixed local/remote targets. Only genuinely unattributed limits retain conservative unscoped behavior.
  • Finalize the canonical-only ADR, align cost-management/outcomes documentation, and integrate the concurrent remote coverage fixes without overwriting them.

Validation

  • make merge-main merged main through 6dffa9ce2e, rebuilt, recompiled all 344 workflows and formatted successfully.
  • make fmt and full make lint passed. Existing JavaScript warnings are non-blocking; Go lint reported zero issues.
  • make agent-report-progress passed its Go lint, schema and impacted-test stages. Its custom function-length diagnostic was fixed, then make agent-report-progress-no-test passed with a session-owned isolated TMPDIR, including all custom linters and all 344 workflow drift checks.
  • After integrating concurrent remote changes, go test ./pkg/cli -run 'Test.*(Episode|Logs|SuggestedRoute|GeneratedOutputSchemasAreCurrent|GeneratedSchemasValidateRealOutput|Continuation|MergeLogsTargetResults)' -count=1 and the no-test publication gate passed again.
  • Regression coverage includes missing root/middle, nil/uncorrelated canonical context, same-run and case-insensitive repositories, partial/contradictory root claims, bounded multi-target collection, count bounds without cursors, cross-repository membership, mixed local/remote targets, compact/verbose output and schema freshness.

Published revision: 88cf5dc6fe101101541ed1a834182c4a06f34927. Prior CI checks passed or were skipped before this push; CI on the published revision remains unverified and was not manually triggered.

Preserve legacy selected-run root fields while exposing recorded canonical root claims, representative aliases, and conservative coverage evidence. Retain useful single-run metadata in compact JSON and propagate collection bounds independently from lineage confidence.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review October 19, 2026 15:39
Copilot AI max review requested due to automatic review settings October 19, 2026 15:39
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

Remove heuristic episode reconstruction and legacy root aliases. Preserve recorded roots, representative identity and conservative selected-run coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · sonnet50 · 15.1 AIC · ⌖ 15.1 AIC · ◇ 6.56 AIC · ⊞ 12.6K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · 7.43 AIC · ⌖ 5.3 AIC · ⊞ 14.7K · ◷

Generated by Ponytail Reviewer for #70277

…nture

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan pelikhan changed the title feat(logs): expose episode root identity and coverage evidence feat(logs)!: require canonical episode identity and report coverage Oct 19, 2026
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (407 new lines in pkg/) but does not have a linked Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/70277-episode-root-identity-and-selected-run-coverage-evidence.md. Review and complete it before merging.

The draft records the decision to add representative_* aliases, recorded_roots[] and a selected-run coverage object (unknown | partial, never complete) without changing grouping, confidence or legacy root fields. It also lists the alternatives that were rejected: redefining root_run_id, crawling lineage, and folding gaps into confidence.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next
  1. Review the draft ADR committed to your branch. It was generated from the PR diff.
  2. Complete the missing sections: add context the AI couldn't infer, refine the decision rationale, confirm the alternatives and fill in the [TODO: verify] deciders.
  3. Commit the finalized ADR to docs/adr/ on your branch.
  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-70277: Episode Root Identity and Selected-Run Coverage Evidence

Once an ADR is linked in the PR body, this gate will run again and check that the implementation matches the decision.

❓ Why ADRs Matter

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors, including your future self, can find the reasoning behind the code.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context: What is the problem? What forces are at play?
  • Decision: What did you decide? Why?
  • Alternatives Considered: What else could have been done?
  • Consequences: What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number.

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · pi · opus55 · 30 AIC · ⌖ 40.4 AIC · ⊞ 1.9K · ◷
Comment /review to run again

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Coverage can misclassify lineage evidence and repository identities, making its machine-readable results unreliable.

3 open issues
What changed in this PR

Adds episode root identity and selected-run coverage evidence to gh aw logs while preserving existing grouping and legacy fields.

Changes:

  • Adds recorded roots, representative aliases, and coverage metadata.
  • Exposes evidence in compact JSON, verbose text, and the generated schema.
  • Adds regression tests and documents selection limits.
File Description
schemas/​logs.schema.json Defines new episode metadata.
pkg/​cli/​logs_report.go Marks truncation and retains useful episode evidence.
pkg/​cli/​logs_orchestrator_render.go Applies collection coverage signals.
pkg/​cli/​logs_format_compact.go Displays root and coverage evidence.
pkg/​cli/​logs_episode.go Integrates coverage into episode construction.
pkg/​cli/​logs_episode_rollup.go Populates representative aliases.
pkg/​cli/​logs_episode_coverage.go Computes root observations and coverage gaps.
pkg/​cli/​logs_episode_coverage_test.go Tests coverage, selection, and rendering.
pkg/​cli/​logs_cached_json_test.go Checks coverage from cached context.
docs/​src/​content/​docs/​setup/​cli.md Explains metadata and count limits.
docs/​src/​content/​docs/​reference/​cost-management.md Documents selected-run usage semantics.

Review effort: Max

Comment thread pkg/cli/logs_episode_coverage.go Outdated
Comment thread pkg/cli/logs_episode_coverage.go Outdated
Comment thread pkg/cli/logs_episode_coverage.go Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /codebase-design (primary) with /tdd as a secondary lens. Triage note: the pr-triage agent was not separately invoked; classification used the documented fallback heuristic against pr-meta.json/pr-diff.patch directly — this reads as a feature change (new representative_run_id, recorded_roots[], coverage{} fields), with logs_episode_coverage.go, logs_episode_coverage_test.go, and schemas/logs.schema.json as the highest-impact files by changed lines.

One actionable /codebase-design observation posted inline (duplicate collection-truncation trigger across two call sites — currently harmless due to addGap dedup, but a maintainability risk). Everything else here is solid, so approving with that single non-blocking note.

📋 Key Themes & Highlights

Key Themes

  • Minor duplication risk: markEpisodeCollectionTruncated is invoked from both buildLogsData (logs_report.go:706-708, gated on continuation != nil) and applyLogsCollectionCoverage (logs_episode_coverage.go:139-144, gated on len(continuations) > 0 || countLimitReached). Both fire on the same request via prepareLogsData. Not a bug today (dedup via slices.Contains in addGap makes it idempotent), but two independently-maintained trigger conditions for one flag is a navigability smell worth consolidating.

Positive Highlights

  • ✅ RepresentativeRunID/RepresentativeWorkflow are introduced as explicit, test-verified compatibility aliases of the legacy RootRunID/PrimaryWorkflow fields — a clean, non-breaking renaming strategy per /codebase-design.
  • ✅ logs_episode_coverage_test.go is thorough TDD-style coverage: root presence/absence/conflict, invalid/overflow run IDs, legacy-context compatibility, reusable same-run hops, missing-parent dedup/sort, and multiple collection-truncation shapes, plus compact/verbose JSON round-trips.
  • ✅ logs_cached_json_test.go confirms coverage fields are recomputed per-selection rather than served stale from cache — good regression guard.
  • ✅ Docs (cost-management.md, cli.md) introduce the new vocabulary ("recorded roots", "representative run", coverage scope/status/reasons) with clear field tables — good /grill-with-docs hygiene.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 153.4 AIC · ⌖ 16 AIC · ⊞ 10.1K
Comment /matt to run again

Comment thread pkg/cli/logs_episode_coverage.go Outdated
@github-actions

Copy link
Copy Markdown
Contributor

Comment Memory

Peek at saved memory (pr-code-quality-reviewer)
reviewed_at: 2026-10-19T15:54:32.362+00:00
review_event: REQUEST_CHANGES
top_themes:
  - collection coverage scoping
  - partial recorded root evidence preservation
files_reviewed:
  - pkg/cli/logs_episode_coverage.go
  - pkg/cli/logs_episode.go
  - pkg/cli/logs_episode_rollup.go
  - pkg/cli/logs_orchestrator_render.go
  - pkg/cli/logs_report.go
comment_count: 2

Note

This comment is managed by comment memory.

Expand the saved memory block to view or edit the persistent context for this thread.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 100 AIC · ⌖ 7.67 AIC · ⊞ 12.7K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes

The new episode coverage metadata still has two correctness gaps: multi-target continuations currently downgrade unrelated episodes, and partial recorded-root claims are being dropped from the JSON surface.

Blocking themes
  • collection_truncated is applied globally instead of only to the targets that actually returned a continuation.
  • recorded_roots loses workflow or repository evidence whenever root_run_id is absent, so the new machine-readable output can hide partial or conflicting lineage claims.

This was a static diff review only; I did not run the code or tests.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 100 AIC · ⌖ 7.67 AIC · ⊞ 12.7K
Comment /review to run again

Comment thread pkg/cli/logs_episode_coverage.go
Comment thread pkg/cli/logs_episode_coverage.go
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (pkg/cli/logs_episode_coverage.go:90): A selected root outside this episode is not necessarily contradictory. Dispatch roots normally have no inbound AwContext (generate_aw_info.cjs:181–221). Selecting root 100 and grandchild 300, but omitting caller 200, leaves the root in standalone:100; this branch then reports conflicting_root_lineage for an otherwise consistent root claim. The current "middle absent" fixture hides this by giving the root a canonical episode ID. Reserve the conflict reason for a repository mismatch or a different recorded canonical episode ID. Keep an unlinked root unobserved, and add a case with ... - feat(logs)!: require canonical episode identity and report coverage #70277 (comment)
  3. Review (pkg/cli/logs_episode_coverage.go:110): The same-run shortcut skips repository validation. A selected run 100 in org/repo with caller context {run_id: "100", repo: "org/other"} therefore keeps coverage unknown instead of reporting conflicting_parent_lineage. Without root claims, compact JSON also drops this evidence. Remove the shortcut so same-run callers use the membership/repository check too. Valid reusable-workflow hops still pass, and this does not create a self-edge. - feat(logs)!: require canonical episode identity and report coverage #70277 (comment)
  4. Review (pkg/cli/logs_episode_coverage.go:121): GitHub repository identities are case-insensitive, but this check treats recorded Org/Repo and selected org/repo as different repositories. Even when the root is selected in the episode, it becomes observed: false and coverage incorrectly reports conflicting_root_lineage. Use strings.EqualFold, as logs_cached_json.go:790 already does. The root-repository conflict set at lines 48–56 should likewise ignore case while preserving the original recorded claims. - feat(logs)!: require canonical episode identity and report coverage #70277 (comment)
  5. Review (pkg/cli/logs_episode_coverage.go:139): [/codebase-design] applyLogsCollectionCoverage marks episodes as CollectionTruncated based on len(continuations) > 0 || countLimitReached, but buildLogsData (logs_report.go:706-708) already calls markEpisodeCollectionTruncated under a different condition (continuation != nil). Both run on the same request in prepareLogsData (logs_orchestrator_render.go:76-78), so there are now two independent sources of truth deciding the same flag. - feat(logs)!: require canonical episode identity and report coverage #70277 (comment)
  6. Review (pkg/cli/logs_episode_coverage.go:144): This marks every episode as collection_truncated when any per-target continuation exists, so a multi-repo report can label fully collected repositories as partial just because one other target paged out. - feat(logs)!: require canonical episode identity and report coverage #70277 (comment)
  7. Review (pkg/cli/logs_episode_coverage.go:75): When root_run_id is empty this returns immediately, which drops root_workflow_id and root_repo evidence from recorded_roots and makes partial or contradictory root claims disappear from the new JSON surface. - feat(logs)!: require canonical episode identity and report coverage #70277 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 2ccb3c8
Sous-chef work: 22a0167d6169c60d26f250158452b93a7159d637420f1d8c74648d434925165b 5be4b51f62f31af31c3679ebb5979f6970462481641bbd23d545f2cec6c3148a 8a0490b226f0c50e3ae4d6b4d7ee4e0bfbcc293784fc96dfa1beb60d7765063a ad1a54566ab387166c272fb1e63bf54c8db2bb300fcc418fb8b64aefd99f5694 b0bd792e7836d72670aebd4fe7effc552dfc1baa550b2270826cfd0b805f5503 bb337cabbba84b310615989aef05a7e23b8ba0f421d03f06612819e1317bf110
Sous-chef state: 19ab09c425cbe711c3795b0d446e94f0bdfa128ed265cfa96c3953b87408674a

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 5.01 AIC · ⌖ 14.5 AIC · ⊞ 1.1K · ◷
Comment /souschef to run again

pelikhan and others added 2 commits October 19, 2026 09:45
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
… targets

Validate same-run parent repositories and compare repository identities case-insensitively. Centralize collection coverage with scoped continuation and count-limit evidence, including mixed local and remote targets. Add regressions and finalize the canonical-only episode ADR.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve the concurrent case-insensitive repository matching and regression tests while retaining scoped selected-member collection bounds, direct-builder consistency and explicit unverified-root evidence.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator Author

In response to the design gate status: the substantive ADR request is addressed by the finalized ADR-70277 and its link in the PR description. The ADR records the current canonical-only behavior rather than the superseded compatibility-only proposal.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

In response to the skills review status: no actionable change was requested by this status notice, so no additional implementation change is needed for it. The concrete correctness findings from the other reviews are handled separately in this batch.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

In response to the test-quality status: I did not treat reported_no_action as validation evidence. The published batch adds focused regression coverage and passes the local formatting, lint, impacted-test and workflow-drift gate.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

In response to the Ponytail review status: this notice contains no actionable finding, so no separate code change is needed. The batch is validated locally rather than relying on the review's no-action outcome.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

In response to the code-quality review status: the substantive findings are addressed by target-scoped collection coverage and preservation of partial recorded-root claims. Each detailed inline thread has its own response and resolution.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Accepted the ADR requirement. Finalized ADR-70277 and linked it in the PR body, including the explicit canonical-only breaking decision, alternatives, consequences and conservative coverage limits.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

In response to the saved review themes: both underlying findings are addressed, with collection bounds scoped to affected targets and partial root claims retained in recorded_roots. The managed comment-memory block is unchanged.

@pelikhan

Copy link
Copy Markdown
Collaborator Author

Accepted the outstanding-work request: merged the latest main with make merge-main, addressed the six inline findings, and pushed the locally validated batch. Each listed review thread is answered and resolved; no workflow run was manually triggered and the PR was not merged.

@pelikhan
pelikhan merged commit d5ecd72 into main Oct 19, 2026
80 checks passed
@pelikhan
pelikhan deleted the pelikhan-glowing-adventure branch October 19, 2026 17:18
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.92.6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants