Running unit tests on a continuous integration service like Travis CI or Wercker

Introduction

This is the last article in a series of three in which I explain how I setup my JavaScript unit testing with code coverage and getting it to run on a CI Server. First I described how to add unit testing and then I showed how to configure Istanbul and Grunt to generate code coverage reports. In this third step I will explain how I configured the project to use a continuous integration (CI) server like Travis CI or Wercker to run the unit tests when changes are pushed to the Git server.

Series overview

This series consists of the following articles:

Part 1: JavaScript unit testing with Mocha and Grunt
Part 2: Generating code coverage reports using Istanbul
Part 3: Running unit tests on a continuous integration service like Travis CI or Wercker

The demo project on GitHub

The files of the demo project are available on GitHub. For every step in getting the project setup there is separate branch in the repository. Whenever there is a branch with the current state of the code in the article I will make a mention of it in the text.

Why Travis CI and Wercker?

It is a question which can easily be answered: they can both be used for free. In the case of Travis CI that is only true for public repositories but so is GitHub it depends on. Because I frequently use BitBucket instead of GitHub there was a need to look at some other service as well. Wercker was one of the few services I found that could be enjoyed for free. Right now it doesn't seem to have any subscription plan but according to a post in this Quora thread they plan to ask for money once they're out of beta.

Common configuration

Before I explain the Travis CI and Wercker specific setups there is a bit of common ground between the two of them. For Node.js projects, the type of project this is, both will run the command npm test by default. Right now the command npm test will not have any effect when executed in the project's root folder. It needs to be configured, this is something that is done in the package.json file.


{
    // 1: The required fields to make this a valid package.json file.
    // 2: The packages we need for this project, these have been added by
    //    executing "npm install --save-dev <package>" in the first two parts.
    // 3: This is the part to wire up "npm test" to run the grunt task. This is
    //    done by adding the key "test" and give it the command
    //    "grunt test:testonly". Now "npm test" will run the unit tests without
    //    generating code coverage reports.
    "name": "unit-test-demo",     /* [1] */
    "author": "Thijs Busser",     /* [1] */
    "version": "1.0.0",           /* [1] */
    "devDependencies": {          /* [2] */
        "expect.js": "^0.3.1",
        "grunt": "^0.4.5",
        "grunt-mocha": "git+https://github.com/jonbretman/grunt-mocha.git",
        "istanbul": "^0.3.2",
        "load-grunt-config": "^0.13.1",
        "mocha": "^1.21.4",
        "requirejs": "^2.1.15"
    },
    "scripts": {                  /* [3] */
        "test": "grunt test:testonly"
    }
}

The package.json for the project

Because the unit tests are run using a Grunt task it is necessary to make sure the CI will have the grunt-cli available before starting the tests. This is something to remember when configuring the task for the CI server.

For the next part it is necessary for the code to be hosted on either GitHub or BitBucket. In order to use Travis CI it will be necessary to use GitHub as there is no support for any other service.

Travis CI

For Travis CI it is necessaryto add a configuration file in the root of the project, named .travis.yml. It is a fairly straight forward configuration for this demo project. Just make sure not to push the file to GitHub before going into Travis CI and have it monitor the repository.


# 1: This tells Travis CI it is a Node.js project, we need this to run our
#    packages.
# 2: It is possible to test your project on several versions of Node.js on each
#    run. Version 0.10 is the lastest stable release and is good enough for this
#    project.
# 3: Before Travis CI runs the "npm install" command it will run whatever we
#    specify here. Because we need the grunt-cli to be available so we can
#    start the unit tests we can instruct Travis CI to install it now.
language: node_js                 # [1]
node_js:
  - "0.10"                        # [2]
before_install:
  - "npm install -g grunt-cli"    # [3]

The .travis.yml file for the project

Tell Travis CI to monitor the project

Before pushing the .travis.yml file to GitHub you need to tell Travis CI to start watching the project for changes. For the sake of brevity I will assume you have a Travis CI account and that it has access to your public GitHub repositories. Telling Travis CI to start monitoring the project is as simple as flicking a switch. Just to go the Travis CI profile page and turn the switch on.

The switch to let Travis CI know it should monitor the repository

Adding a status image to the project README.md

To let people looking the project on GitHub know the project passes the unit tests it is common to add a README.md to the project with a status button. To get the Markdown code for the button you need to enter the Travis CI page for your project, this can be done by clicking on the wrench in front of the on/off switch as seen in image 1.

At the top right of the project page you will see the current status which should read "build unknown" (box 1 in the image below), click on this and a dialog appears with a select box. Select the markdown option (box 2 in the image below) and it will show the Markdown code for the project's status image.

Screenshot of the Travis CI project page showing which element to click to get the Markdown code for the status image

See Travis CI at work

Once Travis CI is configured you can push the .travis.yml file to the repository on GitHub. If all went well Travis CI will detect the new commit and schedule the project in for a run. Once the project reaches the top of the queue the unit tests will be executed and the status should change to "build passing".

Screenshot of the Travis CI project pages with a status indicator showing the build passes the unit test

Wercker

The setup for Wercker is not much different from Travis CI. It also needs a configuration file to let it know how to setup the project. For Wercker the file is called wercker.yml and needs to be in the root folder of the project.


# 1: This tells Wercker the project is a Node.js project.
# 2: This is a custom install step which we need to install the grunt-cli
#    package. We need this to run our unit tests.
# 3: This will run the "npm install" command, it will the Node.js packages we
#    need for the project.
# 4: This will run the command "npm test", we set it up in the package.json file
#    to run the command "grunt test:testonly".
box: wercker/nodejs                             # [1]
# Build definition
build:
  steps:
    - script:                                   # [2]
        name: install grunt-cli
        code: sudo npm install -g grunt-cli
    - npm-install                               # [3]
    - npm-test                                  # [4]

The wercker.yml file for the project

Unlike Travis CI you can already push the file to the repository on GitHub or BitBucket. You can also wait until after the next step, this gives the opportunity to get the code for the Wercker build status image and push the configuration file together with a README.md with the current Wercker status.

Tell Wercker to monitor the project

Just like Travis CI Wercker needs to be told to monitor the project. In the Wercker interface you do this by adding an app. It will let you pick between adding a GitHub or BitBucket repository and once Werkcer has the necessary permission to access your repositories on the selected service you can select the repository you want it to monitor.

Screenshot of the Wercker website showing the interface for selecting a project to monitor

The rest of the setup process for public GitHub and Bitbucket repositories is just straight forward. Basically keep clicking next till you hit finish.

Adding a status image to the project README.md

Wercker also has a status image that can be displayed to show the status of the project. On the application overview page for the project there is a button "Badge" which will summon a dialog (Box 1 in the image below). Here you can pick the version you want and have the markdown code copied to your clipboard (Box 2 in the image below).

Screenshot of the Wercker project page showing where to find the Markdown code for the status image

See Wercker at work

Now that everything is configured Wercker should run the unit tests when changes are pushed to the remote repository. If everything went well Wercker should show something similar to the image below.

Screenshot showing the build for a commit passes the unit tests

Wercker and private BitBucket repositories

Wercker can also be configured to work with a private BitBucket repositories. This is not that complicated to setup but it is easy to make the wrong choice in the setup wizard for a new app. Be sure the use the option Manually add the deployment key.. Upon selecting this option you will be presented with a SSH key. You need to copy this key, and go to your account settings on BitBucket. From the menu on the left select SSH keys (Box 1 in the image below) and use the Add key button (Box 2 in the image below) to paste the key copied from Wercker.

Screenshot showing the BitBucket account management page with the SSH Keys menu item opened

Wrapping up

That concludes this series. I've explained how I have added unit tests to my project using Mocha and Express.js. In addition to unit tests I've shown how to add code coverage reports using Istanbul. In this part I've demonstrated how to setup the project to run the unit tests on a CI server like Travis CI or Wercker. This covers our bases and ensures unit tests are an integral part of the development process.

They way the Grunt tasks are setup they will work just as well in a larger project as they did in this small example. They way I have it setup it requires a manual action to run the tests during development but it shouldn't be a problem to have them run automatically using a Grunt file watcher or something else.