Front-End Testing Part 2: Cross-Browser Acceptance Tests with TestCafe, BrowserStack, and npm Scripts
In the first part of this two-part series about front-end testing, we explored how to write JavaScript unit tests. In this article, we build acceptance tests powered by TestCafe, BrowserStack, and npm scripts. After setting up local testing, we configure Travis CI to automatically run our tests after pushing new code to a Git repository.
Setup
First of all we’re creating the directory structure for our test code. In the first part of this series we’ve already created a test
directory containing a sub directory for our unit tests now we add an additional acceptance
directory for our acceptance test files.
.
└── test
├── acceptance
└── unit
Next we need to install the tools required to run acceptance tests.
npm install --save-dev http-server testcafe
We’re installing TestCafe locally, therefore we need to add an npm script to our package.json
file to conveniently run our tests with a simple command.
"scripts": {
"test:unit": "ava test/unit/**/*.test.js",
"test:acceptance": "testcafe chrome,firefox test/acceptance/ --app 'http-server demo/ -p 1337 -s'",
"test": "npm run test:acceptance"
}
The test:unit
npm script you can see here was created in the first part of this article series about front-end testing. The test:acceptance
script is the one we’re going to focus on in this article. By providing chrome,firefox
in the command line arguments for the testcafe
command we’re telling TestCafe to run tests in the locally installed browsers Chrome and Firefox. The next argument test/acceptance/
is telling TestCafe were to look for test scripts.
By specifying the --app
option we can tell TestCafe to start a new HTTP server using the http-server
npm package, serving the contents from the demo/
directory, using port 1337
and suppress (HTTP server) log messages by providing the -s
(silent) option. The site which is served by http-server
is the site we’re going to test.
Writing tests
The page we’re going to test is the project website of an open source project of mine: perfundo – a pure CSS lightbox. You can find the code of the site and also all the test code on GitHub.
Usually you’ll create a separate test file for every page of your website. In some cases – if you want to test very complex pages – you might also create separate files for every feature on a page. In our case there is only one page, the homepage, which we’re going to test.
// test/acceptance/index.js
import { Selector } from "testcafe";
fixture("Index").page("http://localhost:1337/");
In the example code above you can see that we’re importing Selector
from the testcafe
package – Selector
is a helper function to make it possible to select DOM elements. Next we’re defining a new fixture
, fixtures in TestCafe are a way of categorizing your tests. In this case we’re naming our test Index
and defining the page we want to test. The port we use here, must be the same you’ve specified in the npm script in your package.json
file.
test("The hero section is visible.", async (t) => {
const heroSection = Selector(".c-hero");
await t.expect(heroSection.exists).ok();
});
Our first test is a rather simple one. What we’re doing here is that we try to select an element with a .c-hero
selector and check if it exists on the page.
test(`Click on hero image opens lightbox.`, async (t) => {
const heroImageLink = Selector(`.c-hero .perfundo__link`);
const heroImageOverlay = Selector(`.c-hero .perfundo__overlay`);
await t
.expect(heroImageOverlay.visible)
.notOk()
.click(heroImageLink)
.expect(heroImageOverlay.visible)
.ok();
});
Now the fun begins. In the test code above we’re selecting the first element with a .perfundo__link
class inside an element that matches the .c-hero
selector. And we’re also selecting the first overlay element inside the hero section.
The expect
function checks if the heroImageOverlay
element is visible and we expect it to be not visible at first by using notOk()
. Next we trigger a click()
on the heroImageLink
element. After that we expect that the visibility state of the heroImageOverlay
element has changed and that it is visible by now.
By running npm run test:acceptance
we can run our tests locally. Assuming Chrome and Firefox are installed we can see how TestCafe is opening those browsers and we can watch while the provided tests are running.
If you want to see more test examples you can go to the perfundo GitHub repository.
Configuring Travis CI
In order to automatically run tests when pushing new code to GitHub (or some other system with Travis CI integration) we have to add a .travis.yml
configuration file and activate our project in the Travis CI dashboard. You can read more about adding a new project to Travis CI in the official documentation.
language: node_js
node_js:
- "node"
script:
- npm run lint
- npm run test
sudo: required
addons:
firefox: latest
apt:
sources:
- google-chrome
packages:
- google-chrome-stable fluxbox
before_script:
- "export DISPLAY=:99.0"
- "sh -e /etc/init.d/xvfb start"
- sleep 3
- fluxbox >/dev/null 2>&1 &
In the Travis CI configuration file you can see above, we’re telling Travis CI to run the lint
and test
npm scripts. sudo: required
is needed to install Firefox and Chrome via apt
. In the addons
section we’re instructing Travis CI to install Firefox, Chrome and the Fluxbox window manager which is needed to run TestCafe tests in Firefox and Chrome. The commands which we’re specifying in the before_script
section are necessary to configure Fluxbox correctly for our use case.
That’s it. After enabling your project in the Travis CI dashboard and adding a .travis.yml
configuration file to your project you’re able to automatically run your TestCafe powered acceptance tests on Travis CI every time you push new code.
Cross-browser testing with BrowserStack
Automatically running your acceptance tests in Chrome and Firefox is nice but we all know that there are several other browsers we have to take care of. BrowserStack makes it possible to boot up an instance of basically every relevant browser on the market. And furthermore BrowserStack provides an automation API which makes it possible to automate cross-browser testing.
Luckily TestCafe makes integrating BrowserStack as easy as it gets by providing an npm package to connect with the BrowserStack automation API.
npm install --save-dev testcafe-browser-provider-browserstack
All we have to do is to install the testcafe-browser-provider-browserstack
npm package, specify the browsers we want to test in the test:acceptance
npm script and make our BrowserStack credentials available via environment variables.
export BROWSERSTACK_USERNAME=perfundo
export BROWSERSTACK_ACCESS_KEY=1234567supersecret
First of all we’re creating a new .env
file which is exporting the BrowserStack configuration environment variables and add it to our shell configuration file (e.g. .bashrc
or .zshrc
depending on which shell you’re using) by adding the following line source ~/.env
. Restart your shell afterwards for the changes to take affect.
"scripts": {
"test:acceptance": "testcafe 'browserstack:ie@10.0:Windows 8,browserstack:ie@11.0:Windows 10' test/acceptance/ --app 'http-server demo/ -p 1337 -s'"
}
As you can see in the example above I have replaced chrome,firefox
with the BrowserStack browser identifiers for Internet Explorer 10 and 11. To get a list of all available browser identifiers you can run testcafe -b browserstack
(assuming you’ve installed TestCafe globally). If you’re getting an error stating that authentication fails you’ve either provided wrong BrowserStack credentials or there is a problem with loading the .env
file which exports the BrowserStack environment variables.
While your tests are running you can watch their status in the BrowserStack Automate dashboard. And after the tests are finished you can watch a video of your test.
Re-configure Travis CI for BrowserStack cross-browser testing
If you want to test only in BrowserStack browsers, you can cleanup your .travis.yml
configuration file.
language: node_js
node_js:
- "node"
script:
- npm run lint
- npm run test
You can remove all the additional settings which were required to install Firefox, Chrome and Fluxbox.
Configuring ESLint to work with TestCafe
I personally had some troubles to get up and running with linting TestCafe files using ESLint. To save you the headaches here are some things I’ve learned about using ESLint with TestCafe (following the JavaScript syntax described in the TestCafe documentation) I wish I had known earlier.
{
"env": {
"browser": true,
"node": true
},
"globals": {
"fixture": true,
"test": true
},
"parserOptions": {
"ecmaVersion": 8
}
}
What you can see above is my .eslintrc
ESLint configuration file. By defining fixture
and test
as globals, ESLint ignores the usage of those variables without defining them. Setting the ecmaVersion
to 8
makes it possible to use the latest JavaScript syntax which is also used in the TestCafe documentation code examples. There is also an official TestCafe ESLint configuration package you can use, but the only thing it does is to add the fixture
and test
globals.
Wrapping it up
The combination of TestCafe, BrowserStack and Travis CI makes it possible to run all of your acceptance tests on all of your supported platforms automatically whenever you push new code to your repository.
You can save a lot of precious time and money by making sure your new feature runs in all relevant browsers without manually running every test yourself. And even more importantly automated tests can give you the peace of mind that everything still works after a major code overhaul or after implementing a new feature which affects other parts of your site.
With powerful tools like TestCafe and BrowserStack there is no reason not to write acceptance tests for at least the core features of your site.