Skip to content

[WIP] Add support for custom agents in waza check#314

Merged
spboyer merged 2 commits into
mainfrom
copilot/feat-waza-check-support-custom-agents
Jun 15, 2026
Merged

[WIP] Add support for custom agents in waza check#314
spboyer merged 2 commits into
mainfrom
copilot/feat-waza-check-support-custom-agents

Conversation

Copilot AI commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.


This section details on the original issue you should resolve

<issue_title>feat: waza check support for custom agents (.agent.md)</issue_title>
<issue_description>## Summary

waza check validates SKILL.md readiness (compliance score, token budget, agentskills.io spec compliance, advisory checks), but there is no equivalent readiness/lint command for custom agents (.agent.md). Waza already treats .agent.md as a first-class evaluation target for waza run (per the custom agents guide and #225), so it would be natural to extend waza check to lint agent files too.

Problem

Today waza check is hardcoded to require a SKILL.md. As a result, teams that ship custom agents have to maintain a separate, homegrown validator for .agent.md frontmatter and conventions, duplicating logic that waza check already provides for skills.

Proposed solution

Teach waza check to recognise and validate .agent.md files, mirroring the skill flow:

  • If a target directory (or path) resolves to an .agent.md instead of SKILL.md, run an agent check rather than erroring.
  • Reuse the existing check framework (compliance scoring, token budget, advisory checks) with an agent-aware spec checker.

Suggested agent-specific checks (based on the fields documented in the custom agents guide):

  • Frontmatter present & parseable, with required name and description.
  • name matches filename (e.g. security-reviewersecurity-reviewer.agent.md), consistent with how the guide describes agent naming.
  • Allowed/known frontmatter fields: name, description, tools, model, handoffs, mcp-servers, agents — warn on unknown keys.
  • tools shape validation (array of tool names) when present.
  • Token budget / body-size check for the agent system prompt (the markdown body), analogous to the SKILL.md token budget, with .waza.yaml override support.
  • Advisory checks for description quality, etc.

Open question / options for the maintainers:

  1. Auto-detect agent vs. skill from the file/dir contents within waza check (preferred — keeps one command), or
  2. Add a dedicated subcommand/flag (e.g. waza check --agent <path> or waza check agent <path>).

Why this matters

  • Removes the need for downstream homegrown agent validators.
  • Gives agent authors the same pre-submission confidence skills already get.
  • Keeps agent and skill governance consistent under a single waza check entry point/CI step.</issue_description>
  • Closes feat: waza check support for custom agents (.agent.md) #313

<agent_instructions>Ensure we also update tests, docs, and the public site. Be sure that tests are added/updated and pass. Create a PR with the changes </agent_instructions>

Comments on the Issue (you are @copilot in this section)

Copilot AI requested review from Copilot and removed request for Copilot June 11, 2026 16:09
@spboyer spboyer marked this pull request as ready for review June 15, 2026 17:06
Copilot AI review requested due to automatic review settings June 15, 2026 17:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.

@spboyer spboyer enabled auto-merge (squash) June 15, 2026 17:06
@spboyer spboyer merged commit bfb23d4 into main Jun 15, 2026
7 checks passed
@spboyer spboyer deleted the copilot/feat-waza-check-support-custom-agents branch June 15, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: waza check support for custom agents (.agent.md)

3 participants