Using Docker to Run Visual Regression Tests with Jest and Puppeteer

This is a follow-up to my last article about about how to create Visual Regression Tests with Jest and Puppeteer for Vue.js applications. Today we’ll explore how we can use Docker to run our tests in a standardized environment in order to prevent false positives because of differences between operating systems. For a detailed explanation of how to set up Jest and Puppeteer please take a look at the previous article.

You can find the full code of a demo application on GitHub.

Diff showing Visual Regression

Why Docker?

What our Visual Regression Tests do is run a real browser, take a screenshot of our application, and compare that screenshot to a reference image. As we all know (due to many painful experiences), websites render slightly differently on different operating systems or even just different versions of operating systems. By using Docker, we can ensure that the screenshots created by Jest and Puppeteer always look the same, no matter what host operating system is used, whether it is MacOS, Windows or Linux.

Setup

Before we get started you need to install Docker. You can read more about how to get started with Docker on the official website. Next we can proceed with creating our Dockerfile.

# Dockerfile
FROM buildkite/puppeteer:latest
RUN mkdir /app
WORKDIR /app
COPY package.json yarn.lock ./
RUN yarn && yarn cache clean
COPY . .

We use the buildkite/puppeteer Docker image because it has everything we need to run our Puppeteer powered Visual Regression Tests out of the box. Other than that this is a pretty standard Dockerfile for running Node.js applications. In order to optimize caching of the Docker container we copy the package.json and yarn.lock files first before installing all of our dependencies with Yarn and copying the rest of the application into our /app working directory.

# .dockerignore
node_modules/

We don’t want to copy the node_modules directory so we add a new .dockerignore file which excludes this directory.

# docker-compose.yml
version: '3'
services:
  visual:
    build:
      context: .
    volumes:
      - "./src:/app/src"
      - "./test:/app/test"
    command: sh -c "npm run serve & wait-for-it.sh localhost:8080 -- npm run test:visual"

Next we add a docker-compose.yml file which makes it easier to start our Docker container with some predefined settings. In order to sync changes between the Docker container and the host system at runtime we link the src and test directories as volumes. We use the command property to specify which command should be triggered when starting the container. In our case we serve our application with npm run serve and then use wait-for-it.sh, which is built into the buildkite/puppeteer Docker image, to wait until localhost:8080 is up so we can start our Visual Regression Tests by running npm run test:visual.

 const DEBUG_MODE = process.argv.includes('--debug');

+const debugLaunchOptions = DEBUG_MODE ? {
+  headless: false,
+  slowMo: 100,
+} : {};
+
 module.exports = {
-  launch: DEBUG_MODE ? {
-    headless: false,
-    slowMo: 100,
-  } : {},
+  launch: {
+    args: [
+      // Required for Docker version of Puppeteer.
+      '--no-sandbox',
+      '--disable-setuid-sandbox',
+      '--disable-dev-shm-usage',
+    ],
+    ...debugLaunchOptions,
+  },
};

Because unfortunately Puppeteer does not work inside of a Docker container out of the box, we have to set some special launch arguments inside of our jest-puppeteer.config.js configuration file.

Running the tests

After we have everything set up we can start our Visual Regression Tests by running docker-compose run visual. If we have done everything right, our tests should now run inside of a Docker container.

Wrapping it up

The only thing worse than having no tests at all is having unstable tests. By making sure our Visual Regression Tests always run in the same Docker environment, we’re able to perform the tests on different machines and even in our CI pipeline without having to worry about minor rendering differences on these different machines.

References

• Docker Puppeteer Example
• Alan Foster, Running react unit tests within docker