Skip to content

Agent Persona Exploration - 2026-10-18 #69944

Closed as not planned
Closed as not planned

Description

@github-actions

Persona Overview

  • Agent: agentic-workflows custom agent (via persona-evaluator sub-agent)
  • Personas This Run: Product Manager, Program Manager, QA Tester
  • Scenarios Tested: 4 of 6 generated (PM2-3, PRG-3, QA-3, QA-4)
  • Average Quality Score: 4.85/5.0

Key Findings

  • All 4 scenarios received strong, production-ready recommendations (no score below 4 on any dimension).
  • The agent consistently defaults to least-privilege read-only permissions and empty network.allowed: [] even when not explicitly asked.
  • For "visibility over time" asks (milestone risk, test-health trend), it correctly reframes them as scheduled digests rather than forcing an event trigger.
  • It proactively designs dedup keys (close-older-issues + per-entity title prefixes) to prevent duplicate tracking issues across runs.
  • It independently flagged a cross-scenario collision risk (two scheduled report workflows using the same create-issue/close-older-issues pattern) and specified distinct dedup keys to avoid interference.

Top Patterns

  1. Triggers: pull_request (scoped to specific types, same-repo only) for inline review tasks; schedule + workflow_dispatch for periodic digests; workflow_run with a precise if: gate (run_attempt > 1) for rerun-based detection.
  2. Tools: github MCP in gh-proxy mode with the minimal toolset needed (actions: read only when reading CI artifacts/logs); cache-memory: true for cross-run trend baselines.
  3. Security practices: read-only permissions scoped per-field (contents, issues, pull-requests, actions), no network access granted for analysis-only tasks, explicit noop conditions defined for every scenario.
View High Quality Responses (Top 3)
  • PM2-3 (Product Manager, PR automation): Resolves the roadmap issue via closing keywords, diffs PR scope against the issue's acceptance-criteria checklist, and posts a single rolling comment (hide-older-comments: true) instead of spamming the thread on every push.
  • QA-3 (QA Tester, issue automation): Uses workflow_run gated on run_attempt > 1 as a precise, low-noise trigger for flaky-test detection, and dedups tracking issues by test name (not commit) so recurring flakiness consolidates into one issue.
  • QA-4 (QA Tester, scheduled): Uses cache-memory to persist the prior week's baseline for trend comparison, and correctly distinguishes "no data" (report with a note) from "noop" (zero qualifying runs) — avoiding an over-eager noop that would hide a real reporting gap.
View Areas for Improvement (Top 2)
  • No weak responses were observed this run, so there is less adversarial signal than usual — all 4 scenarios tested were already well-aligned with authoring guidance, which may indicate the subset chosen (PR/schedule/workflow_run triggers) is already a "solved" pattern space for the agent. Future runs should include an issue-triggered or on-demand scenario to probe less-tested trigger types.
  • The recurring "scheduled digest + create-issue + close-older-issues" pattern appeared independently in both PRG-3 and QA-4; while the agent self-corrected with distinct dedup keys, this pattern recurring unprompted across unrelated scenarios suggests it's worth codifying explicitly rather than relying on the agent to re-derive it each time.

Recommendations

  1. Add the "recurring digest" pattern (schedule + create-issue with close-older-issues: true and a dedup key scoped to <entity>:<time-window>) as a named, citable pattern in .github/aw/create-agentic-workflow.md, since it now appears independently across multiple unrelated personas/scenarios.
  2. Document the workflow_run + if: run_attempt > 1 gating technique for rerun/flakiness-detection workflows in .github/aw/github-agentic-workflows.md, as a reusable low-noise trigger condition.
  3. Next run, deliberately select at least one issues-event or workflow_dispatch-only scenario per persona to diversify trigger coverage beyond the PR/schedule/workflow_run space this run concentrated on.

Generated by 🎭 Agent Persona Explorer · claude · agent · 111.1 AIC · ⌖ 0.742 AIC · ⊞ 6.7K · ◷

Activity

  1. github-actions commented on Oct 20, 2026

    @github-actions
    ContributorAuthor

    This issue is being closed as outdated. A newer issue has been created: #70544

    View newer issue


    This action was performed automatically by the Agent Persona Explorer workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions