Weighing up the benefits

Manual vs. automated testing

Jason Silberman

Automated testing can speed up the testing process, but manual tests can help quickly familiarise testers with the product’s features. Is one approach better than the other? Jason Silberman investigates.

The use of manual and automated testing in the application lifecycle has been much debated. In general, developers are quick to defend the one they normally use and are most familiar with. If manual testing is what teams know, they will go with that whenever possible while those with the tools and automation experience often attempt to automate everything.

But quality assurance teams and developers will tell you that the reality is more nuanced, and you actually need both manual and automated testing. Each approach has its role at different times in the testing practice, and both contribute to the ability of a team to deliver high quality releases. Instead, the question should be when to use one versus the other?

What are the benefits of manual testing?

Manual testing provides a way to quickly evaluate a product and provide testers with a familiarity of the features during the development process. Testers create test cases based on their ability to determine whether or not requirements are met. But initially these test cases have to be executed manually, to both verify the necessary steps, and also to be able to record test scripts for automation in the future.

Additionally, manual testing includes exploratory testing, which enables testers to learn more about the application, while also identifying areas that potentially need more test cases to fully understand weaknesses and risk. Lastly, manual testing is especially valuable if it is used early in the development of features and the user interface, as layouts and controls can change daily in response to design considerations and user feedback. It can be more time-consuming to maintain and change automated scripts than to execute tests manually.

What are the benefits of automated testing?

Test automation also has multiple roles in the development process. As testers grow more confident in their manual tests, automation saves the time and effort of repeating the same tests over and over again. Recording these tests produces scripts that automated testing tools can execute at any time, often with different sets of data.

Automation brings an important aspect to testing practices: consistency. Specifically, it assures consistency in how a test is performed. Once the validity of a given test is established, automating it provides a defined benchmark of how the test is run on a regular basis. The steps of the test aren’t subject to interpretation by individual testers, and are conducted with the same timings and inputs throughout.

Automation is also essential for regression testing, which is the practice of executing already-passed tests throughout the development process, to ensure that already-implemented features aren’t broken as development continues. Regressions suites typically contain hundreds or even thousands of tests that can help maintain find issues throughout the development lifecycle.

What is the influence of agile?

Agile processes play a role in the debate. Agile requires as much testing as possible to be automated in order to meet the timing requirements of sprints. Recording automation scripts as quickly as they are validated enables an agile team to focus on giving new code and features in sprints the most individual attention possible.

Of course, automation is a defined practice in both unit testing and smoke testing. Developers typically use an automated unit testing framework such as JUnit or NUnit to perform basic checks on individual parts of their code. And whatever approach the team takes to building and integration, executing a small suite of functionality tests at the end of the process ensures the integrity of the build and provides the team with a known level of quality to begin further work.

Lastly, automation is essential to certain types of testing. With most types of performance and load testing, it’s simply not feasible to have many users manually exercise an application at the same time. Instead, performance testing tools create virtual users that execute scripts emulating actual activities that the software is supposed to perform. In many cases, teams can use existing functional test scripts as a basis for performance and load testing activities. Both manual and automated testing are important parts of any development effort. Knowing when to use each enables a team to leverage the strengths of both in the pursuit of quality, no matter what the methodology or schedule.

Jason Silberman
Jason Silberman works at BlazeMeter, a PaaS that offers a cross-enterprise test automation framework for technical teams throughout the product development lifecycle.

Inline Feedbacks
View all comments