Base image question using playwright when performing visual comparison

22 Views Asked by At

How do one save the base images of pages when running on a CI/CD platform like Gitlab This is my Gitlab YMl file setup

stages:
   - uat

test-uat:
  rules:
      - if: $CI_PIPELINE_SOURCE == "trigger"
        when: always
      - if: $CI_PIPELINE_SOURCE != "trigger"
        when: manual
      - if: $CI_COMMIT_BRANCH == "master"  
  stage: uat
  image: 
    name: mcr.microsoft.com/playwright:v1.41.1    
    pull_policy: [always, if-not-present]
  allow_failure: false
  script:
      - npm ci
      - ENV=UAT npx playwright test 
  artifacts:
    paths:  
      - playwright-report/
      - test-results/
      - results.xml/
    when: always
    expire_in: 120 days  #artifacts will purged after 120 days of test run
    reports:
        junit: results.xml

I get this error messgae in the log

A snapshot doesn't exist at /builds/test-automation/playwright-portal-test-automation/base-images/chromium/support.png

I can fix this by running the same test again if I were running this on my machine to create the base image. On Gitlab, I can't because the image will not be saved in the repository. I have tried to save the image on my windows machine and then push it to the Gitlab(Linux) remote repository, but I get failures due to pixel differences because of how the browser renders the page on different operating systems.

I created a docker image of the project by copying the existing base images captured in the Windows machine and still ran into the same issue

# Get the latest version of Playwright
FROM mcr.microsoft.com/playwright:v1.41.1

# Set the working directory
WORKDIR /app

# Copy package.json and package-lock.json
COPY package.json package-lock.json ./

# Install dependencies
RUN npm ci

RUN npx playwright install

# Copy the rest of the application files
COPY . .

EXPOSE 3000

# Set the entry point for the container
CMD ["sh", "-c", "ENV=UAT npx playwright test --reporter=html "]

I want to run the test in an environment that will be consistent thereby eliminating the pixel issues. I don't want to compensate by setting maxDiffPixels, threshold, or maxDiffPixelRatio.
How do you approach this?

Thanks

0

There are 0 best solutions below