Does Your Test Automation Strategy Need an Overhaul?

"If it ain't broke, don't fix it." It's an old adage, however, in the case of your test automation strategy, it may not always ring true. In the midst of juggling looming deliverables, servicing the needs of corporate stakeholders, and maintaining the status quo, it's easy to push test strategy reviews into the long-term planning category. However, it only takes one small issue to expose gaps in a test automation strategy. For this reason, it's worth proactively asking the question, "How well is your test automation strategy working for you?"

Building a test automation strategy takes time, resources, and consensus. This is why maintenance and periodic reviews of that strategy help ensure all that valuable work is being efficiently leveraged.  When considering a review of your test automation strategy, it’s critical that you ask some probative questions:

Have there been significant changes in your team or in stakeholders?

Your strategy was ideally built based on input from a core team of knowledgeable professionals with experience and skills that speak to the goals of the product, system, or service being tested. Whether a stakeholder has a role that makes them directly responsible for some aspect of test strategy, or they are just impacted in some way by the strategy, it is important to make sure the right subject matter expertise is represented.

In order to assess your test strategy's performance, you'll also need feedback from those who are instrumental in deploying and measuring the strategy. If those who are most familiar with crafting the strategy have changed, it is critical that current stakeholders understand the strategy and have an opportunity to provide input.

Are you meeting the goals your organization has established?

Whether your goals were speed to market, improved performance metrics, streamlined communication, enhanced user experience, and/or cost savings, you need to discern whether you have met your measureable goals and how and where you have improved. It's not enough to say that QA has been improved; your stakeholders will want to understand how it has improved. For example, has the introduction of automated tests resulted in a significant reduction in development time or technical debt? Are your automated tests primarily unit-driven, or are behavior-driven tests also included? It is important to consider both quantitative and qualitative (but still measurable!) goals when determining the effectiveness of your test automation strategy.

Have your organization’s QA needs changed?

With the introduction of new products, functionalities, and improvements, it’s worthwhile to evaluate whether any of these needs have required your QA processes to change. Have your QA processes been updated to address new changes or needs in your environment? Your test automation strategy should be able to effectively support the requirements for your developmental and product needs.

Don’t be afraid to put your strategy to the test! Consider complex test environments and whether some of the scenarios you previously thought were edge cases are now a bit less “edgy”. Think through how user behaviors may have evolved since your previous test automation strategy had been created. Are all business-critical paths being considered?

Automated testing solutions such as Cycle can support the creation and maintenance of a comprehensive test automation strategy by serving as a “single source of truth” for stakeholders.

Whether you need to update or overhaul your test automation strategy, always ensure you have the right mix of people, resources, and tests to ensure the right level of coverage and optimize your strategy.

This post was written by:

Jillian Ketterer
Scrum Master, Product Development

Jillian is the scrum master for the product development team that builds Cycle continuous testing software. She has more than 10 years of experience in project management, training, and process improvement in the information technology sector.