Detecting Web Application Regression with PhantomJS

22 Jan 2017

For those people who are developing web applications/websites then every time a change is made it should be going through a QA. This is for functional testing of the change but also to detect any regression which may have occurred.

PhantomJS is a tool that should be considered for this regression checking, with only simple simple scripting it can be put into your Continuous Integration system and you have a way of catching regression early and before it goes for QA.

There are alternative ways, such as Selenium driving with test suites, which are great but require a lot of work and maintenance. Using PhantomJS is something you could do today and start getting the benefits immediately. But how?

There is an example script supplied by PhantomJS that takes a list of URLs and renders them out as images. This could be called from Jenkins, TeamCity, etc. with the URL of the server that is being deployed to (as part of Continuous Deployment) and the images can be compared to a known state by scripting a tool like Beyond Compare. The output of Beyond Compare (or alternative) can then be output in the CI task and the differences can either be flagged or quickly checked against the known state. Even without a comparison to a known state, the screenshot can be put right in the CI job (and email, etc.) so that the person can see that the check-in caused Tests to pass, linting to pass, was deployed and exactly what the main pages look like on deploy.

This obviously isn't a cross browser test, but that's not the point of it, it renders the site and ensures it loaded and the output is valid in the "happy case". You could have a PhantomJS script which screenshots using multiple viewport sizes to do a quick test across the main device sizes for your responsive design.

This last point is a great one, I wrote a script to allow checking a single URL against a list of 50 top devices based on UA, width, height and pixel density that only takes less than a minute to run. It's not perfect, it's not the actual browser or device but if a developer can run a script in less than a minute as a first step then why not do it? It can catch things early and lower that feedback cycle.

As an alternative to screenshot-ting, phantomJS can do a check against the text and elements on the page to quickly check the main areas and text are present. A quick and simple test that avoids needing to setup a Selenium project, grid, etc.

For those of you with restful APIs, then you can run a request simply using PhantomJS as well to check the basics.

If you are a TDD team and already (or are planning to) have a large test framework (including UI testing) then it might not be for you, but the ability to very quickly extend a CI job (or ran manually) with some quick regression/smoke tests should not be dismissed. Lowering the feedback cycle is very important for a development team.

Posted by makit
Last revised 22 Jan 2017 11:54 AM

makit / Martyn Kilbryde

Professional software developer, ponderer and eccentric.

flickr

Github

LinkedIn

Stack Overflow

Twitter