Skip to content

[OCPERT-210] Add OpenShift Tests Extension (OTE) integration by using the automated migration OTE tool #705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ming1013 wants to merge 2 commits into
base: master
Choose a base branch
from
Open

[OCPERT-210] Add OpenShift Tests Extension (OTE) integration by using the automated migration OTE tool #705

ming1013 wants to merge 2 commits into from
+30,171 −0

Conversation

@ming1013
Copy link

@ming1013 ming1013 commented Dec 15, 2025 •
edited
Loading

Tested by:

$ tests-extension/router list | grep -F '"name": "[sig-' |wc -l
7742

./tests-extension/bin/router-tests-ext run-test "[sig-network-edge] Network_Edge Component_Router Author:hongli-LEVEL0-Critical-47344-check haproxy router v4v6 mode"
I1226 16:01:10.352767 75868 test_context.go:567] The --provider flag is not set. Continuing as if --provider=skeleton had been used.
Running Suite: - /home/minl/router

Random Seed: 1766736070 - will randomize all specs

Will run 1 of 1 specs

[sig-network-edge] Network_Edge Component_Router Author:hongli-LEVEL0-Critical-47344-check haproxy router v4v6 mode
/home/minl/router/tests-extension/test/e2e/haproxy-router.go:1185
STEP: Creating a kubernetes client @ 12/26/25 16:01:10.6
I1226 16:01:19.932636 75868 client.go:293] configPath is now "/tmp/configfile2442011323"
I1226 16:01:19.932714 75868 client.go:368] The user is now "e2e-test-router-env-9rvfn-user"
I1226 16:01:19.932752 75868 client.go:370] Creating project "e2e-test-router-env-9rvfn"
I1226 16:01:20.358323 75868 client.go:378] Waiting on permissions in project "e2e-test-router-env-9rvfn" ...
I1226 16:01:21.979722 75868 client.go:407] DeploymentConfig capability is enabled, adding 'deployer' SA to the list of default SAs
I1226 16:01:22.388446 75868 client.go:422] Waiting for ServiceAccount "default" to be provisioned...
I1226 16:01:23.309156 75868 client.go:422] Waiting for ServiceAccount "builder" to be provisioned...
I1226 16:01:24.232013 75868 client.go:422] Waiting for ServiceAccount "deployer" to be provisioned...
I1226 16:01:25.152388 75868 client.go:432] Waiting for RoleBinding "system:image-pullers" to be provisioned...
I1226 16:01:25.971406 75868 client.go:432] Waiting for RoleBinding "system:image-builders" to be provisioned...
I1226 16:01:26.790745 75868 client.go:432] Waiting for RoleBinding "system:deployers" to be provisioned...
I1226 16:01:28.327035 75868 client.go:465] Project "e2e-test-router-env-9rvfn" has been fully provisioned.
STEP: Get ROUTER_IP_V4_V6_MODE env, if NotFound then v4 is using by default 12/26/25 16:01:28.327
STEP: Get ROUTER_IP_V4_V6_MODE env, if NotFound then v4 is using by default @ 12/26/25 16:01:28.327
I1226 16:01:30.096362 75868 util.go:587] the result of router pod name: router-default-df59fff4f-dkvbc
I1226 16:01:34.280665 75868 client.go:1086] Error running oc --kubeconfig=/home/minl/openshift/auth/kubeconfig/kubeconfig exec -n openshift-ingress router-default-df59fff4f-dkvbc – bash -c /usr/bin/env | grep ROUTER_IP_V4_V6_MODE:
StdOut>
Error from server (BadRequest): pod router-default-df59fff4f-dkvbc does not have a host assigned
StdErr>
Error from server (BadRequest): pod router-default-df59fff4f-dkvbc does not have a host assigned

I1226 16:01:34.280896 75868 util.go:606] the matched Env are:
NotFound
I1226 16:01:35.715528 75868 haproxy-router.go:1190] the cluster IP stack type is: ipv4single
I1226 16:01:36.084249 75868 client.go:681] Deleted {user.openshift.io/v1, Resource=users e2e-test-router-env-9rvfn-user}, err:
I1226 16:01:36.449548 75868 client.go:681] Deleted {oauth.openshift.io/v1, Resource=oauthclients e2e-client-e2e-test-router-env-9rvfn}, err:
I1226 16:01:36.813026 75868 client.go:681] Deleted {oauth.openshift.io/v1, Resource=oauthaccesstokens sha256~LfImyEEoz0_PaQokGvtQWjNXP8Ukeja3xixghkTi6qw}, err:
STEP: Destroying namespace "e2e-test-router-env-9rvfn" for this suite. @ 12/26/25 16:01:36.813
• [26.636 seconds]

Ran 1 of 1 Specs in 26.636 seconds
SUCCESS! – 1 Passed | 0 Failed | 0 Pending | 0 Skipped

Part of automated migration OTE tool log:

Migration Complete! 🎉

I've successfully migrated your router component to the OpenShift Tests
Extension (OTE) framework. Here's what was done:

What Was Created

A new tests-extension/ directory in your router repository containing:

  • 23 test files from openshift-tests-private/test/extended/router/
  • 74 testdata files from
    openshift-tests-private/test/extended/testdata/router/
  • Complete OTE integration with:
    • cmd/main.go - Extension entry point
    • go.mod - Properly configured with all dependencies
    • Makefile and bindata.mk - Build automation
    • test/testdata/fixtures.go - Testdata helper functions

Key Features

  • 223 router component tests discovered (out of 7,742 total including k8s
    tests)
  • Platform filters automatically applied based on test labels
  • Testdata embedded using go-bindata
  • Build verified and working

Critical Fix Applied

I discovered and fixed a Ginkgo version incompatibility issue by adding the
OpenShift Ginkgo fork replace directive:
github.com/onsi/ginkgo/v2 => github.com/openshift/onsi-ginkgo/v2
v2.6.1-0.20241205171354-8006f302fd12

Quick Start

cd tests-extension

Build the extension
make build

List all tests
./router list

Run tests
./router run

The complete migration summary with all details, troubleshooting tips, and
customization options is provided above. You're ready to integrate this into
your CI/CD pipeline!

Directory structure:

/home/minl/router/ # Working directory
└── tests-extension/ # Isolated test extension directory
├── cmd/
│ └── main.go # OTE entry point
├── test/
│ ├── e2e/ # Test files
│ │ └── *.go (23 files)
│ └── testdata/ # Testdata files
│ ├── bindata.go # Generated
│ └── fixtures.go # Wrapper functions
├── go.mod # Single module
├── go.sum
├── Makefile # Build targets
└── bindata.mk # Bindata generation

@ming1013
Copy link
Author

ming1013 commented Dec 15, 2025

Need more testing
/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 15, 2025
@openshift-ci openshift-ci bot requested review from ironcladlou and rikatz December 15, 2025 08:18
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Dec 15, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign rikatz for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ming1013
Copy link
Author

ming1013 commented Dec 15, 2025

cc @rioliu-rh @LuboTerifaj @jhuttana @tomasdavidorg

rioliu-rh
rioliu-rh reviewed Dec 15, 2025
View reviewed changes
rioliu-rh
rioliu-rh reviewed Dec 15, 2025
View reviewed changes
rioliu-rh
rioliu-rh reviewed Dec 15, 2025
View reviewed changes
@rioliu-rh
Copy link

rioliu-rh commented Dec 15, 2025 •
edited
Loading

the changes in root Makefile is missed, i.e. compile router test extension in a builder stage, compress it with gzip and copy it to router image

@ming1013 ming1013 changed the title Add OpenShift Tests Extension (OTE) integration by using the automated migration OTE tool [OCPERT-210] Add OpenShift Tests Extension (OTE) integration by using the automated migration OTE tool Dec 15, 2025
@ming1013 ming1013 mentioned this pull request Dec 15, 2025
Open
3 tasks
@ming1013
Migrate router tests to OTE framework using single-module strategy
dcd39a0 
  - Created tests-extension/ directory structure
  - Migrated 23 test files from openshift-tests-private
  - Migrated 74 testdata files to tests-extension/test/testdata/router/
  - Replaced compat_otp.FixturePath() with testdata.FixturePath() (variadic)
  - Generated OTE main.go with platform filters and cleanup hooks
  - Added Makefile and bindata generation support
  - Updated .gitignore to exclude generated files

  The extension binary can be built with:
    cd tests-extension && make build

  Tests can be run with:
    ./tests-extension/bin/router-tests-ext list
    ./tests-extension/bin/router-tests-ext run-test <test case name>
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Dec 26, 2025

@ming1013: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/verify dcd39a0 link true /test verify
ci/prow/okd-scos-images dcd39a0 link true /test okd-scos-images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ironcladlou ironcladlou Awaiting requested review from ironcladlou

@rikatz rikatz Awaiting requested review from rikatz

1 more reviewer

@rioliu-rh rioliu-rh rioliu-rh left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ming1013 @rioliu-rh