Skip to content

Add Copilot Centralization Optimizer workflows#39394

Merged
mnkiefer merged 9 commits into
mainfrom
prompt-analysis-workflows
Jun 15, 2026
Merged

Add Copilot Centralization Optimizer workflows#39394
mnkiefer merged 9 commits into
mainfrom
prompt-analysis-workflows

Conversation

@mnkiefer

@mnkiefer mnkiefer commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds two new gh-aw agentic workflows that together form a Copilot Centralization Optimizer pipeline. The pipeline mines cross-user Copilot agent task data to surface repeated prompt patterns that should be standardized into first-class workflows, shared prompts, or playbooks, then drills each candidate down into a concrete, reviewable draft artifact.

What changed

New: copilot-centralization-optimizer workflow

A scheduled (weekly, Monday) + workflow_dispatch workflow that:

  1. Collects agent task data across the repository for the past 30 days (up to 300 tasks) via the /agents/repos/{repo}/tasks API. A single consolidated API call per task avoids duplicate fetches (addressed in a87bd1afa).
  2. Precomputes two jq-derived datasets — intent-bucket clusters and exact-repeat groups — with overall stats (total tasks, distinct creators, avg sessions/task, fallback rate).
  3. Passes the precomputed data to an AI agent phase (max 250 credits / 1 000 daily) that recommends one of three centralization kinds per candidate:
    • continuous_workflow — automate as a scheduled or event-driven gh-aw workflow
    • shared_prompt_or_chatops — create a reusable shared prompt or ChatOps command
    • keep_ad_hoc_but_standardize — document as a playbook for consistency
  4. Publishes recommendations as a GitHub issue (title prefix [copilot-centralization], labels report, ai-optimization, 30-day expiry, closes older issues).
  5. Persists long-lived trend snapshots in a repo-memory branch (memory/copilot-centralization-optimizer) for cross-run comparison.

Safe-output constraints: no bot mentions, no external GitHub references, strict: true.


New: copilot-centralization-drilldown workflow

A workflow_dispatch-only companion workflow that takes a single optimizer candidate and produces a concrete draft:

Input Purpose
candidate_title Short name for the repeated task / skill
target_kind workflow, shared_prompt, playbook, or auto
recommendation_kind Optimizer output kind (drives auto resolution)
sample_prompt Representative prompt text
evidence_summary Compact behavior description
candidate_json Full compact JSON object from optimizer (overrides fields above)

auto resolution logic maps optimizer kinds to concrete targets:

  • continuous_workflowworkflow
  • shared_prompt_or_chatopsshared_prompt
  • keep_ad_hoc_but_standardizeplaybook

Candidate strength is classified as strong (sample ≥ 24 chars, evidence ≥ 40 chars, or full JSON provided) vs weak to guide the AI draft quality.

The AI step (max 120 credits / 500 daily) produces a full draft artifact, published as a GitHub issue (title prefix [copilot-centralization-draft], labels report, ai-optimization, workflow-design, 30-day expiry). Issues are intentionally not auto-closed to allow iterative refinement.


Action pin updates

File Pin added
pkg/actionpins/data/action_pins.json github/gh-aw-actions/setup@v0.79.6
pkg/workflow/data/action_pins.json github/gh-aw-actions/setup@v0.79.6, github/gh-aw/actions/setup-cli@v0.79.8

Stale upload-artifact@v4 pin was also corrected during recompile (c28f49d9a).

Motivation

Copilot agent tasks across a repository frequently contain near-identical or structurally similar prompts submitted by different users. Without tooling, these patterns remain invisible and continue consuming ad-hoc AI credits. This pipeline automates discovery and standardization, reducing redundant usage and improving prompt quality organization-wide.

Design notes

  • Two-phase design (optimizer → drilldown) keeps concerns separate: the optimizer runs at scale on raw task data while the drilldown focuses deeply on one candidate at a time with more AI budget per artifact.
  • jq preprocessing before the AI phase reduces token consumption and improves reliability by feeding structured, deduplicated signal rather than raw API payloads.
  • No duplicate API fetches: the optimizer uses a single gh api call per task id and pipes directly, avoiding the double-fetch pattern identified in review (a87bd1afa).
  • Both workflows use strict: true and conservative safe-output settings (no external mentions, bounded bot mention count).

Breaking changes

None.

Generated by PR Description Updater for issue #39394 · 269.4 AIC · ⌖ 13.2 AIC · ⊞ 19.9K ·

@mnkiefer mnkiefer self-assigned this Jun 15, 2026
Copilot AI review requested due to automatic review settings June 15, 2026 15:19
@mnkiefer

Copy link
Copy Markdown
Collaborator Author

@copilot Recompile the new workflows and do nothing else

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds two new gh-aw workflows to mine repeated Agent Task prompt patterns (“optimizer”) and to expand a single candidate into a concrete, reviewable draft (“drilldown”), plus updates the action lockfile to support the new workflow dependencies.

Changes:

  • Introduces a scheduled “Copilot Centralization Optimizer” workflow that collects Agent Task metadata, precomputes repetition/trend datasets, and publishes a recurring report.
  • Adds a “Copilot Centralization Drilldown” workflow_dispatch workflow to turn one optimizer candidate into a single draft artifact published via create-issue.
  • Extends .github/aw/actions-lock.json with additional pinned action references used by the compiled lock workflows.
Show a summary per file
File Description
.github/workflows/copilot-centralization-optimizer.md New optimizer workflow definition + deterministic data prep for centralization analysis and reporting.
.github/workflows/copilot-centralization-optimizer.lock.yml Compiled/pinned workflow for the optimizer.
.github/workflows/copilot-centralization-drilldown.md New drilldown workflow for expanding one candidate into a single draft artifact.
.github/workflows/copilot-centralization-drilldown.lock.yml Compiled/pinned workflow for the drilldown.
.github/aw/actions-lock.json Adds pinned entries for actions referenced by the new workflows.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/5 changed files
  • Comments generated: 6

Comment thread .github/workflows/copilot-centralization-optimizer.md Outdated
Comment thread .github/workflows/copilot-centralization-optimizer.md Outdated
Comment thread .github/workflows/copilot-centralization-optimizer.md Outdated
Comment thread .github/workflows/copilot-centralization-optimizer.md Outdated
Comment thread .github/workflows/copilot-centralization-drilldown.md
Comment thread .github/workflows/copilot-centralization-drilldown.md Outdated
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>

Copilot AI commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

@copilot Recompile the new workflows and do nothing else

Done. Recompiled both workflows in commit db8a0e7.

One minor fix was also needed: the optimizer markdown referenced actions/upload-artifact@v4, which caused a stale v4 pin entry to be synced into action_pins.json and break the TestGetActionPinWithData_SemverPreference tests. Updated the reference to @v7 and cleaned the stale entry from actions-lock.json before recompiling.

mnkiefer and others added 4 commits June 15, 2026 18:47
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ smoke-ci: safeoutputs CLI comment + comment-memory run (27561935911)

Generated by 🧪 Smoke CI for issue #39394 ·

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
@mnkiefer

Copy link
Copy Markdown
Collaborator Author

@copilot Address all remaining Copilot review comments and recompile workflows.

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
@github github deleted a comment from github-actions Bot Jun 15, 2026
@github github deleted a comment from github-actions Bot Jun 15, 2026
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>

Copilot AI commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

@copilot Address all remaining Copilot review comments and recompile workflows.

Done in 282bb36. I addressed the remaining Copilot review comment and recompiled the workflows.

@mnkiefer mnkiefer merged commit e6f1181 into main Jun 15, 2026
1 check failed
@mnkiefer mnkiefer deleted the prompt-analysis-workflows branch June 15, 2026 17:25
Copilot stopped work on behalf of mnkiefer due to an error June 15, 2026 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants