On Monday morning, a new ticket comes through to Jamie. They have to fix a bug impacting some users, and locking them out of their account.
Jamie looks into the bug, they can replicate it. “Aha!”, Jamie says under hushed breath, the bug has been spotted.
Jamie changes some code, checks the user journey again. Voilà! The impacted user accounts can now get back in. Jamie commits the code, and it’s accepted into the production branch and instantly in use by thousands of users.
But wait, now customer service’s phones are lighting up, and hundreds of more people can’t log in. What happened? It turns out Jamie just treated a symptom of the bug - not the cause - and now it’s morphed into something far more powerful because Jamie didn’t regression test well enough.
However, Jamie did check the broken journey - so they shouldn’t get all the blame right? After all, the code was accepted into the codebase with no issue, suggesting that others weren’t doing their due diligence either.
In all honesty though, who wants to go through all the previous tests they’ve done and ensure editing one function hasn’t broken something anywhere else? It’s tedious and could be a huge waste of time looking for a problem that doesn’t exist.
These things don’t have to be manual. In fact, most of the time they shouldn’t be manual. There is another way.
With automated testing, we can cut a lot of the doubt, time and effort from testing. We can write test scripts that describe exactly what we want to happen with no ambiguity, and have the results almost instantly.
What is testing?
Testing is a way in which we can ensure our expectations of the software are being met, and that the application is working as intended. It can help us find potential issues with the application, or identify problems before implementing anything.
It's fairly well agreed that all testing is sampling - that is, you take a few samples of the application's code, design your tests to gain some form of coverage, and then test those samples.
If testing produces failures, it’s safe to assume that other unsampled areas have issues too, and so you need to continue sampling and testing. However, if you meet a certain threshold (e.g. no bugs found in 90% of tests in all samples), then you could feel confident that things are working correctly and release the product.
Manual vs. automated testing
Manual testing is important, and there is definitely a time and place where something will need to be tested manually. For example, it would probably be overkill to start implementing automated tests if your app is small, or you have made a marketing site with only a handful of pages.
However, if you have a dedicated tester on your team, the likelihood is that your application or site is way too big to manually test everything.
The issue with manual testing is that once you have completed a whole round of testing, the code may need to be updated, and you'll now have a product with very different behaviour - untested behaviour.
So you have three choices:
I tested it before and it looks like the problem is fixed, let's just roll it out.
I'll run every test case manually again, and hope I don't have to make any changes and repeat this whole process again.
I'll make these tests automated, that way we can change whatever we want, and I can rerun the tests in a matter of seconds.
Out of all three, the third option looks pretty tempting, right?
With automated testing, we can cut a lot of the doubt, time and effort from testing. We can write test scripts that describe exactly what we want to happen with no ambiguity, and almost instantly have the results inform us of whether the app or web design is working smoothly.
Choose the right automated testing framework
There are many automated testing languages and frameworks out there, like Cucumber, Jasmine and EarlGrey. No single framework is a silver bullet, so you should choose one according to the platform you’re developing for, and any specific frameworks you are already building in.
Easy and efficient automated design testing is becoming fairly straightforward nowadays with front-end frameworks shipping with tools built-in to help.
The beauty of this approach to testing is that as your tests continue to mount up, you can become more and more sure that your application is working correctly. In addition, you don’t have to worry about the application regressing when you add new features or edit functionality, as you will be alerted to the issue within a matter of seconds.
When things are constantly changing, it’s nice to have the added confidence that your code is still working as expected.
You may wonder why we’d worry about the application regressing when you’re writing additional code. Well, bugs come from somewhere. When you’re adding new code to the repository there’s a fair chance that you might also - unintentionally - be introducing a fault.