Article published in DevOps.com by Jim Azar, Orasi Senior VP and Chief Technology Officer.
The COVID-19 pandemic in 2020 created a pressing need to give people additional ways to do their jobs and handle the tasks of day-to-day life via mobile devices and forced the growth of app innovation at a never-before-seen rate. Thanks to new application development technologies and agile processes along with open source and embedded libraries, organizations are delivering greater functionality, faster, and for multiple devices (think web, mobile, tablet, etc.). And while a never-ending stream of new apps and updates has been great for consumers, it’s proven to be a pain point for many enterprises whose testing processes aren’t up to speed, literally and figuratively.
While test automation is an essential component of agile development, it’s not addressed in published agile methodologies such as Scrum and Kanban. Of course, testing tools and methodologies have changed along with modern development practices, but they haven’t always kept up with today’s rapid-fire pace of AppDev, and the past 18 months have only served to elevate the need for improved testing. For companies to keep up with and move beyond their competition, testing must be conducted in parallel with development.
Automated and Continuous Testing: Analogous, not Synonymous
There are many ways to test software, including manual, user interface (UI), automated and continuous testing (CT), but most enterprises today employ some form of automated testing and, increasingly, continuous testing. All too often, people throw the terms automated and continuous around as if they are synonymous, but they aren’t. So, before we delve into the merits of one over the other, it’s important to have a clear understanding of what each of the terms means.
Automated testing refers to conducting software tests programmatically with little to no human interaction. It measures actual outcomes in relation to predicted outcomes, and, as it’s less time-consuming to execute than manual testing, has become the norm for modern enterprises. Continuous testing, meanwhile, uses automated tests to provide uninterrupted feedback at every stage of the software development life cycle and, therefore, plays a key role in DevOps.
Automated and continuous testing work hand-in-hand to ensure software quality, cost-savings and speed time to market and should be conducted in lockstep with development. Whereas it’s possible to have one without the other, it’s far from advisable, however.
The Upside of Continuous Testing
Continuous testing takes automation to the next level and, as such, holds several advantages over automated testing alone.
- Early and actionable feedback at each development stage means issues can be addressed sooner—before they evolve into larger, more complex issues.
- Agile teams’ velocity is measured by how much they can complete in a sprint. Because of this, a team’s testing focus is narrowed to the sprint they are immersed in, meaning that problems affecting functionality previously deployed often aren’t discovered until later in the development life cycle, or they are discovered by users after deployment.
- An automated and continuous approach means more problems are identified and fixed before deployment.
- Release cycles are accelerated as organizations can respond to market changes faster and in real-time.
- Correcting problems early reduces rework, which translates into cost and time savings.
- Continuous integration requires continuous testing.
- CT tests not only the parts of the application that have changed but also how those updates work within the app’s entire functionality.
- Unlike automated testing, CT requires no human intervention.
What’s the Holdup?
Despite numerous benefits, CT isn’t as widespread as one would hope. One of the reasons for the disconnect comes down to investment, whether it’s hiring the right people, training or tools.
Perhaps the greatest stumbling block comes at the hiring stage. CT requires a specific set of technical skills coupled with a natural aptitude for test development and a mindset focused on resiliency. Beyond investing in staff, there are upfront (and ongoing) automation costs to consider. Unfortunately, it’s not as if you can implement CT and never touch it again—CT becomes its own software delivery pipeline that must be developed, maintained and enhanced to ensure it doesn’t become obsolete.
Continuous Testing: What Could go Wrong?
As the pace of AppDev quickens, so does the imperative to integrate it into DevOps. Failure to do so has serious implications, the most obvious of which is that without CT, companies fail to test as much and as frequently as they should. Regression tests aren’t done as deeply, for instance, so when a new app or update is released, it’s more likely that something will break. Further, without the right skill sets, teams have a tendency to examine whatever is new and fail to see how it fits into the whole. Moreover, without proper testing, you may inadvertently introduce bugs or discover too late that functionality doesn’t work as planned. Relying on manual and slow, fragile UI testing will only hinder progress in organizations that don’t adapt.
Negative consequences go beyond the confines of an agile team or enterprise as poor software design and implementation feed into the all-important issue of customer (dis)satisfaction. As everyone knows, unhappy customers aren’t loyal customers, and that translates to a lower bottom line. Another equally serious (perhaps more serious, depending on your perspective) ramification is the potential impact on data security, exposing organizations to a very different but significant risk posture.
Thanks to DevOps, ideas are moving from deployment to business value at lightning speed. It stands to reason, then, that testing also must be faster than ever.
When you consider that what you see today is the least amount of change you’re going to experience and that tomorrow is guaranteed to bring more, it should be abundantly clear why continuous testing is so important: The less testing you do, the more problems will occur downstream.
Businesses develop software for a reason—to get business value. If your app isn’t working, you aren’t reaping its full value. So, instead of asking yourself if you should implement continuous testing, you need to be asking yourself why you haven’t done it already. By failing to invest in CT now, you will undoubtedly pay for it later.