New extension: Microsoft.Testing.Extensions.GitHubActionsReport (groups, annotations, step summary)

[Evangelink]

Summary

Add a new Microsoft.Testing.Extensions.GitHubActionsReport extension package (sibling of the existing Microsoft.Testing.Extensions.AzureDevOpsReport) that emits GitHub Actions-native workflow commands so test runs on GitHub Actions produce a first-class experience:

1. Per-assembly log groups — ::group:: / ::endgroup:: so the runner UI collapses them by default.
2. Failure annotations — ::error file=…,line=…,title=Test failed::FQN so failures appear in the PR Files changed gutter and the workflow Annotations tab.
3. Job summary — write a markdown summary to \ rolling up totals, failures, slowest tests.
4. Slow-test notices — ::notice:: for tests crossing the slow-test threshold from Silence-driven progress heartbeat renderer for SimpleAnsi/NoAnsi terminal modes #9139.

Hooks into the IProgressEnricher surface introduced by #9139.

Motivation

• GitHub Actions is now one of the most common CI hosts for .NET projects. There is currently no first-party MTP integration.
• \ is a uniquely powerful GH surface — a markdown blob that appears on the workflow run summary page. Test runners that use it provide a massively better PR review experience than those that don't.
• Failure annotations on the diff gutter let reviewers jump straight to the failing assert in code review without scrolling logs.

Proposed behaviour

Log groups

On OnAssemblyStart(asm) → ::group::Tests: {asm.Name} ({asm.TargetFramework}).
On OnAssemblyEnd(asm) → ::endgroup::.

Failure annotations

On OnFailure(testResult):

::error file={file},line={line},col={col},title=Test failed: {fqn}::{firstLineOfMessage}

Multiline error messages: prefer ::error followed by the message body via %0A URL-escaped newlines, or fall back to a single-line snippet.

Slow-test notices

On OnSlowTestEmit(test, currentDuration):

::notice file={file},line={line}::{fqn} still running after {currentDuration}s

Job summary

Append a markdown block to the file pointed to by \ (per Microsoft docs):

## ✅ Test Run Summary

| Assembly | Passed | Failed | Skipped | Duration |
|---|---:|---:|---:|---:|
| MSTest.UnitTests (net9.0) | 845 | 0 | 2 | 1m 12s |
| MSTest.Acceptance.IntegrationTests (net9.0) | 312 | 1 | 0 | 4m 37s |

### ❌ Failures (1)
- `MyOther.OtherTest` — Expected: 42 but was: 41 ([MyOther.cs:88](…))

### ⏱ Slowest tests
- `SlowTest.IntegrationFoo` — 4m 12s
- `SlowTest.Foo` — 2m 03s

Concurrency-safe append (per GH docs) since multiple matrix legs may write concurrently — though each leg gets its own file, so this is mostly a non-issue.

Knobs

• --report-gh-groups — on/off, default on.
• --report-gh-annotations — on/off, default on.
• --report-gh-step-summary — on/off, default on.
• --report-gh-slow-test-notices — on/off, default on.

Auto-detection: extension activates iff GITHUB_ACTIONS=true; otherwise no-ops silently. Discoverable via a single master switch — TBD whether --report-gh (similar to --report-azdo-progress) or auto-on when the env is detected. Recommendation: auto-on when env detected, with each individual knob still respected.

Touchpoints

• New package: src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/ (mirror the AzureDevOpsReport folder layout).
• GitHubActionsReporter.cs — implements IProgressEnricher.
• GitHubActionsReportCommandLineProvider.cs — the four options above.
• GitHubActionsEscaper.cs — escape special chars in titles/messages per GH workflow commands docs.
• Resource strings + dotnet msbuild .../GitHubActionsReport.csproj /t:UpdateXlf.
• Help/info acceptance test expectations (alphabetically sorted):
  • test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs
  • test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuild.KnownExtensionRegistration.cs
• Unit tests in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/ — verify escape behaviour, group emission, step-summary markdown shape.

Open questions

1. Auto-on (when GITHUB_ACTIONS=true) vs explicit --report-gh-* opt-in master switch. AzDO precedent is explicit opt-in via --report-azdo-progress; symmetric choice would be --report-gh master switch.
2. Should \ be byte-capped to GitHub's 1 MiB limit? (Yes, with truncation note.)
3. Should we emit ::warning:: for skipped tests or only for failures? (Probably only on opt-in.)
4. How to surface the file/line for failure annotations when running on Linux runners but the test assembly was built on Windows (path-separator translation).

Out of scope

• Other CI hosts (GitLab, Jenkins, Buildkite, TeamCity) — each is its own package.
• Uploading test result artifacts (use the existing actions/upload-artifact action + the platform's existing TRX/JUnit/HTML reports).
• PR review comments (would need pull-requests: write token, separate concern).

Related

• Add CI-friendly textual progress heartbeat to MTP terminal reporter (raw log progress signal) #9125 — parent design discussion (see "Building blocks" S1 grouping + B7 annotations).
• Silence-driven progress heartbeat renderer for SimpleAnsi/NoAnsi terminal modes #9139 — silence-driven heartbeat + IProgressEnricher hook (prerequisite).
• Microsoft.Testing.Extensions.AzureDevOpsReport — sibling extension; aim for symmetric command-line shape where it makes sense.