Bug 2060532 - LSO e2e tests are run against default image and namespace
Summary: LSO e2e tests are run against default image and namespace
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.10
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.11.0
Assignee: Jonathan Dobson
QA Contact: Wei Duan
URL:
Whiteboard:
Depends On:
Blocks: 2075346
TreeView+ depends on / blocked
 
Reported: 2022-03-03 17:36 UTC by Jonathan Dobson
Modified: 2022-08-10 10:52 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-08-10 10:52:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift local-storage-operator pull 346 0 None open [WIP] Bug 2060532: LSO e2e tests are run against default image and namespace 2022-04-12 02:13:25 UTC
Red Hat Product Errata RHSA-2022:5069 0 None None None 2022-08-10 10:52:41 UTC

Description Jonathan Dobson 2022-03-03 17:36:01 UTC
In bug 2030776 I updated openshift/release to build and deploy a CI image of LSO. I can see from the CI in https://github.com/openshift/local-storage-operator/pull/327 that this image does get built and deployed:

https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_local-storage-operator/327/pull-ci-openshift-local-storage-operator-master-e2e-operator/1498661877167689728/artifacts/e2e-operator/gather-extra/artifacts/pods.json

                "name": "local-storage-operator-778b5c876f-hck6h",
                "namespace": "openshift-local-storage",
                "ownerReferences": [
                    {
                        "apiVersion": "apps/v1",
                        "blockOwnerDeletion": true,
                        "controller": true,
                        "kind": "ReplicaSet",
                        "name": "local-storage-operator-778b5c876f",
                        "uid": "f399c537-b8c7-4b4f-9528-3dacbfbfde53"
                    }
                ],
                "resourceVersion": "29968",
                "uid": "118bac8a-a9a0-4cd4-8e9f-aea58ac7891e"
            },
            "spec": {
                "containers": [
                    {
                        "args": [
                            "--leader-elect"
                        ],
                        "command": [
                            "local-storage-operator"
                        ],
                        "env": [
                            {
                                "name": "WATCH_NAMESPACE",
                                "valueFrom": {
                                    "fieldRef": {
                                        "apiVersion": "v1",
                                        "fieldPath": "metadata.namespace"
                                    }
                                }
                            },
                            {
                                "name": "DISKMAKER_IMAGE",
                                "value": "registry.build01.ci.openshift.org/ci-op-0x7p6lit/pipeline@sha256:d2707d124a6a714a9a7973c199b8229ae3d07988f9970c8b9910918b3f078fa2"
                            },
                            {
                                "name": "KUBE_RBAC_PROXY_IMAGE",
                                "value": "quay.io/openshift/origin-kube-rbac-proxy:latest"
                            },
                            {
                                "name": "PRIORITY_CLASS_NAME",
                                "value": "openshift-user-critical"
                            },
                            {
                                "name": "OPERATOR_CONDITION_NAME",
                                "value": "local-storage-operator.v4.11.0"
                            }
                        ],
                        "image": "registry.build01.ci.openshift.org/ci-op-0x7p6lit/pipeline@sha256:a0d36c7382267e0b2789066af0fa4de792bbcf64a09f1374a22f62e19889a364",
                        "imagePullPolicy": "IfNotPresent",
                        "name": "local-storage-operator",


However, there is something else going on when running hack/test-e2e.sh as part of e2e-operator. Wei pointed out some events like this, showing the default image (and in the default namespace) still:

https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_local-storage-operator/327/pull-ci-openshift-local-storage-operator-master-e2e-operator/1498661877167689728/artifacts/e2e-operator/gather-extra/artifacts/events.json

        {
            "apiVersion": "v1",
            "count": 1,
            "eventTime": null,
            "firstTimestamp": "2022-03-01T15:05:29Z",
            "involvedObject": {
                "apiVersion": "v1",
                "fieldPath": "spec.containers{local-storage-operator}",
                "kind": "Pod",
                "name": "local-storage-operator-577fddbf77-69792",
                "namespace": "default",
                "resourceVersion": "31181",
                "uid": "2b52eecb-a9e0-46a8-8c8e-15e8dfade6e2"
            },
            "kind": "Event",
            "lastTimestamp": "2022-03-01T15:05:29Z",
            "message": "Pulling image \"quay.io/openshift/origin-local-storage-operator\"",
            "metadata": {
                "creationTimestamp": "2022-03-01T15:05:29Z",
                "name": "local-storage-operator-577fddbf77-69792.16d84a17c51ae7ed",
                "namespace": "default",
                "resourceVersion": "31204",
                "uid": "b894832a-05b0-4a95-89ac-1e2bef4a61b6"
            },
            "reason": "Pulling",
            "reportingComponent": "",
            "reportingInstance": "",
            "source": {
                "component": "kubelet",
                "host": "ip-10-0-203-120.us-west-2.compute.internal"
            },
            "type": "Normal"
        },


And from the test logs I see that test-e2e.sh is actually using a separate manifest.yaml that still deploys the default image to the default namespace:

https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_local-storage-operator/327/pull-ci-openshift-local-storage-operator-master-e2e-operator/1498661877167689728/artifacts/e2e-operator/test/artifacts/manifest.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: local-storage-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      name: local-storage-operator
  template:
    metadata:
      annotations:
        target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}'
      labels:
        name: local-storage-operator
    spec:
      serviceAccountName: local-storage-operator
      containers:
        - name: local-storage-operator
          image: quay.io/openshift/origin-local-storage-operator
          ports:
          - containerPort: 60000
            name: metrics
          command:
          - local-storage-operator
          args:
          - --leader-elect
          imagePullPolicy: IfNotPresent
          env:
            - name: WATCH_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: DISKMAKER_IMAGE
              value: quay.io/openshift/origin-local-storage-diskmaker
            - name: KUBE_RBAC_PROXY_IMAGE
              value: quay.io/openshift/origin-kube-rbac-proxy:latest


The substitutions here update the manifest that gets deployed by bundle.Dockerfile:

https://github.com/openshift/release/blob/c52cc269b7e7657725266bf60ca17fe974caa267/ci-operator/config/openshift/local-storage-operator/openshift-local-storage-operator-master.yaml#L48-L51

But it does not affect the images in test-e2e.sh which apparently builds a separate manifest to test:

https://github.com/openshift/local-storage-operator/blob/a9d477a312fa3a7411366f8d025dec14b0aee8ae/hack/test-e2e.sh#L36-L40

We need to fix this so that the CI image is what actually gets tested by test-e2e.sh.

Comment 2 Wei Duan 2022-04-19 08:45:40 UTC
In the latest CI, the test manifest.yaml was removed and I did not see the default image used in the default namespace.
Move the status to VERIFIED.

Comment 4 errata-xmlrpc 2022-08-10 10:52:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069


Note You need to log in before you can comment on or make changes to this bug.