Sauce Labs, Inc. and Dimensional Research’s survey is out

Testing trends in 2018: Value of automated testing, automation challenges and more

© Shutterstock / 10 FACE  

Testing trends, automation challenges and more. The 2018 survey on testing trends conducted by Dimensional Research under the sponsorship of Sauce Labs, Inc. is out and draws our attention to the main testing trends of 2018 as well as the main challenges faced by development and testing professionals.

In early March, Sauce Labs, Inc. announced the results of the “Testing Trends in 2018: A Survey of Development and Testing Professionals”.

The survey was conducted by Dimensional Research with over 1,000 individual participants as well as featuring a wide range of company sizes, roles, vertical industries, and geographies. The survey was conducted online with development and testing professionals.

The key findings as presented in the introductory section of the survey report include the following trends:

  • Trend #1: Testing is conducted on more browser types, increasingly on only the most recent versions
  • Trend #2: A mix of real devices and simulators becomes the norm for mobile testing
  • Trend #3: The value of automated testing is clear
  • Trend #4: Agile and DevOps adoption increases, but bugs are not getting fixed faster
  • Trend #5: The desire to deploy even more quickly appears to be slowing

The highlights

According to the survey, the agile development trend grows steadily. More specifically, according to the 2015 survey, 82% of software developers said that their organization had adopted an agile development methodology while this year the number grew to 91%.

SEE ALSO: Testing your frontend code: Integration testing

However, what is most interesting is that, apparently, the depth of the agile adoption is inversely proportional to the company size.

As the survey suggests, “any transformation typically takes longer at a large organization, and agile is no exception”.

Source: Testing Trends in 2018: A Survey of Development and Testing Professionals, p. 8

When it comes to Continuous Integration, the overwhelming majority reported that their development organization practices CI. What’s more, the survey predicts that “as more companies adopt CI chains to increase the speed and quality of releasing code as well as to find bugs in the software development process itself, it will be extremely critical to include continuous testing in these conversations”.

DevOps, as a more recent trend than agile, was introduced in the survey series as late as 2017.  The 2018 survey suggests that the DevOps trend adoption is mostly still in progress. The survey identifies an increasing level of adoption.

SEE ALSO: Testing your frontend code: Visual testing

According to the 2017 report, only 10% reported fully embracing DevOps across the entire company. In 2018, this number grew to 17%. Nonetheless, the results show that the largest groups have either “just started with DevOps or have a few teams fully immersed”.

Source: Source: Testing Trends in 2018: A Survey of Development and Testing Professionals, p. 9

“Teams are adapting to move faster, and a rising adoption of DevOps and a commitment to automated testing reflects how modern methodologies are becoming more and more critical to successful application development.”

Diane Hagglund, principal researcher at Dimensional Research

Main challenges

In addition to the main trends for 2018, the survey also identifies the main challenges.

Most importantly, the survey identifies an alarming lack of progress in the speed of bug-fixing which acts as a break in the further adoption of agile and DevOps trends. As suggested in the survey, “debugging code quickly remains a barrier to reaping the full benefits of agile and DevOps.”

Source: Source: Testing Trends in 2018: A Survey of Development and Testing Professionals, p. 10

What’s more, it seems that automation is not the default mode yet, even though management is typically supportive towards automated testing.

Source: Source: Testing Trends in 2018: A Survey of Development and Testing Professionals, p. 7

Last but not least, the vast complexity of software and its testing is pictured in the survey as “testers now need to consider multiple platforms, devices, browsers, version of operating systems”.

Overall, the survey presents a growing adaptation of the trends like agile and DevOps and the increasing value of automated testing. Nonetheless, the challenges identified indicate obstacles that need to be overcome if a company needs to achieve a higher level of automation and efficient testing.


Eirini-Eleni Papadopoulou
Eirini-Eleni Papadopoulou was the editor for Coming from an academic background in East Asian Studies, she decided that it was time to go back to her high-school hobby that was computer science and she dived into the development world. Other hobbies include esports and League of Legends, although she never managed to escape elo hell (yet), and she is a guest writer/analyst for competitive LoL at TGH.

Inline Feedbacks
View all comments