Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 5, 2026

Tests were accumulating orphaned resources (repos, teams, secrets) from failed runs, causing rate limiting and conflicts. Each test context repeatedly created/destroyed resources independently, multiplying API calls across parallel execution.

Changes

New Scripts

  • tests/BeforeAll.ps1 - Pre-test cleanup of stale resources across all 7 auth scenarios
  • tests/AfterAll.ps1 - Post-test safety net for missed cleanup
  • tests/README-SETUP-TEARDOWN.md - Implementation documentation

Resource Cleanup Pattern

# Before: Each of 15 test files × 7 auth scenarios creates/destroys resources
# Now: Cleanup runs once before/after all parallel test jobs

foreach ($authCase in $authCases) {
    Connect-GitHubAccount @authCase.ConnectParams
    
    # Remove orphaned test resources by prefix
    Get-GitHubRepository | Where-Object { $_.Name -like "$testPrefix*" } | 
        Remove-GitHubRepository -Confirm:$false
        
    # Similar for teams, secrets, variables, app installations
}

Benefits

  • Significantly fewer API calls → reduced rate limiting
  • Clean slate per test run → eliminates resource conflicts
  • Safety net → catches missed test cleanup
  • Zero test file changes → fully backward compatible

Coverage
Cleans up resources for all test prefixes: RepositoriesTests, TeamsTests, EnvironmentsTests, SecretsTests, VariablesTests, AppsTests, ArtifactsTests, ReleasesTests, PermissionsTests, MsxOrgTests, GitHubTests

Process-PSModule v5 automatically detects and executes these scripts before/after the test matrix.

Original prompt

This section details on the original issue you should resolve

<issue_title>🩹 [Patch]: Consolidate test setup/teardown tasks</issue_title>
<issue_description>### Describe the change

Now that Process-PSModule supports BeforeAll and AfterAll scripts, that run once before and after all the parallel tests, we should move some of the setup/teardown tasks to these. This will hopefully speed up the testing process, and result in fewer failures due to ratelimiting.

Current Test Structure

Each test file currently performs repeated setup/teardown in individual Context blocks:

  • Creates unique test resources (repos, teams, orgs) per Context
  • Connects/disconnects GitHub accounts multiple times
  • Cleans up resources after each Context

Proposed Changes

tests/BeforeAll.ps1 - Infrastructure Setup

Create shared test infrastructure that ALL tests can use:

  • Test Organizations - Pre-create test organizations for different auth scenarios
  • Test Repositories - Pre-create base repositories for testing
  • Test Environments - Pre-create deployment environments
  • Test Teams - Pre-create teams with different permission levels
  • Cleanup old artifacts - Remove stale resources from previous failed runs

Important: This does NOT replace the BeforeAll blocks in test files, which are still needed for:

  • Authentication (Connect-GitHubAccount, Connect-GitHubApp)
  • Context-specific setup
  • Loading test data (AuthCases.ps1)

tests/AfterAll.ps1 - Infrastructure Teardown

Clean up all shared test infrastructure:

  • Remove all test organizations created in BeforeAll
  • Remove all test repositories
  • Remove all test teams and environments
  • Generate cleanup report

Test Files - Keep Existing BeforeAll/AfterAll Blocks

Test file Context blocks should continue to:

  • BeforeAll: Connect to GitHub with appropriate credentials (this stages the runner)
  • AfterAll: Disconnect GitHub sessions
  • Use the pre-created infrastructure instead of creating their own

Benefits

  1. Speed: Infrastructure created once instead of repeatedly per Context
  2. Rate Limiting: Significantly fewer API calls during test execution
  3. Reliability: Consistent test infrastructure across all test runs
  4. Cost: Fewer parallel operations hitting API limits
  5. Maintainability: Centralized infrastructure definitions

Implementation Strategy

  1. Analyze current test patterns to identify common infrastructure needs
  2. Create tests/BeforeAll.ps1 with infrastructure provisioning
  3. Create tests/AfterAll.ps1 with cleanup logic
  4. Update test files to consume shared infrastructure instead of creating their own
  5. Maintain existing authentication logic in test file BeforeAll/AfterAll blocks

Test Files Affected

All 15 test files will need updates to use shared infrastructure:

  • Apps.Tests.ps1
  • Artifacts.Tests.ps1
  • Emojis.Tests.ps1
  • Enterprise.Tests.ps1
  • Environments.Tests.ps1
  • GitHub.Tests.ps1
  • GitHubFormatter.Tests.ps1
  • Organizations.Tests.ps1
  • Permissions.Tests.ps1
  • Releases.Tests.ps1
  • Repositories.Tests.ps1
  • Secrets.Tests.ps1
  • Teams.Tests.ps1
  • Users.Tests.ps1
  • Variables.Tests.ps1</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 4 commits January 5, 2026 23:04
Co-authored-by: MariusStorhaug <17722253+MariusStorhaug@users.noreply.github.com>
Co-authored-by: MariusStorhaug <17722253+MariusStorhaug@users.noreply.github.com>
Co-authored-by: MariusStorhaug <17722253+MariusStorhaug@users.noreply.github.com>
Co-authored-by: MariusStorhaug <17722253+MariusStorhaug@users.noreply.github.com>
Copilot AI changed the title [WIP] Consolidate test setup and teardown tasks Add BeforeAll/AfterAll scripts for centralized test resource cleanup Jan 5, 2026
Copilot AI requested a review from MariusStorhaug January 5, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

🩹 [Patch]: Consolidate test setup/teardown tasks

2 participants