As we work almost exclusively with WordPress, websites often consist of a hodgepodge of ready-made third-party solutions. This often means that it is very difficult to know if something will go wrong when introducing changes, no matter how small the updates are. For features, it is possible to do automated functional tests with for example Selenium & Unit Tests. However, it is very time-consuming (i.e. expensive) and does not work to ensure what is often very annoying for our customers – those little things. In the past, it has been very difficult to be accurate in deliveries, and if you change people, it is very difficult to transfer knowledge from one person to another. Furthermore, the quality of the work has mainly been based on feelings and hunches rather than solid knowledge, which has meant that sometimes things have gone really well and sometimes not so well.
Visualising differences
The theory behind visual comparison tests is simple; In the environment that is open to the outside world (the production environment), things look as they should, but when we introduce changes – an update or a new feature – we don’t really know if there will be regression errors elsewhere, that is, if things that “shouldn’t” break break break. We therefore make a copy of the production environment database in a separate environment, but with the changes we want to introduce.
From there we let the tool do its job. The tool goes into the test site and goes through page by page, while doing the same in the production environment. Once this is done, it presents a result. This result is a calculation where it has made a visual comparison of the two environments, highlighting the difference.
The result shows both the resolution it has tested in (we test in 5 different resolutions), and presents exactly how much the pages differ. For this example, we have not synchronised the databases, so the example shows that the difference is only that there are different records in the “Recent records” column on the right.
What problems does this solve?
The problems solved are many. To mention the main ones;
- Shows if the visual impression of something changes in different resolutions
- Shows if permalinks and page structures still work
- Calculates and highlights anomalies rather than relying on the person having time to spend hours examining the page in detail
- Provides greater confidence as more pages can be reviewed at a very high speed.
The current tool is built based on Wraith, but to reach our goal where we have automated most of the tool and where the tool sits nicely with all of our processes, we realise that we need to do a total redesign. But; until then, we have this awesome tool that helps us and our customers feel safe when performing installations and updates.