Busting Test Automation Myths In Agile Projects: Part 1

Raluca Vartic | April 28, 2021 | 5 Min Read

Agile teams often put testing automation on the back burner. This is due to several myths floating around among some agile teams. In part one of this two part series, Macadamian's Senior Test Automation Engineer, Raluca Vartic, tackles a couple of these myths.

Agile teams house some of the most tech savvy and quick-witted people, who know how to get things done.

These teams excel at collaboration, which is needed to refine product requirements, implement new features, and release improved versions of a product.

But there’s one glaring flaw. Software changes all the time, and the way we are testing it today is not scalable.

Still, agile teams often don’t automate their tests.

Essentially, test automation is about verifying that your product works as intended, and continues to do so as it evolves, with no human interaction involved. It verifies the requirements of a product.

Test automation could look like, for example, running a piece of software to check that unauthorized users cannot access sensitive data, or to verify the logic and the algorithms that determine the information the user will see on the screen.

It’s a key component to developing a successful product. Yet, agile teams tend to either skip test automation entirely, or relegate it to another team. There are several myths that have a hold on agile teams when it comes to test automation.

In this first of a two-part series, I’ll look at a couple of myths that live rent free in the minds of agile team members.

Myth 1: Test Automation Slows Down Already Busy Agile Teams

One of the most common reasons agile teams push away test automation is due to the concern that it will slow down an already busy team.

It doesn’t.

Reality is, automation tests can catch issues early, regressions in particular, and prevent defects. If we want automation test suites to flag product issues early, the tests need to run quickly and early on.

These tests also need to be executed whenever code changes are being made, and stay relevant.

When coding issues are caught really early, the impact is minimal. The issue can be fixed immediately—long before it becomes a defect.

It saves everyone a lot of time and heartache.

On the other hand, when issues are caught later, for example, in the next sprint by a regression team or in production, a defect is typically created.

This defect then needs to be prioritized, and it might take weeks or even months to fix.

And if that’s not challenging enough, resolving a defect involves added communication channels between teams. And that has the tendency to create a broken-telephone effect when tackling a defect. You could end up with a product release that has defects and unhappy users.

No one wants that.

The myth that agile teams are too busy to take time out for test automation stems from a belief that setting up and maintaining an automation framework is time consuming and costly.

What this fails to consider is the cost of bugs, added communication channels, and manual regression testing needed when test automation is passed off to another team.

Myth 2: Test Automation Isn’t Reliable

Despite the concept of the testing pyramid having existed for more than a decade, many teams still automate testing at the User Interface (UI) level only.

The test pyramid is illustrated through a hierarchy with End to End tests (UI, Mobile) up top, Integration tests in the middle, and Unit Tests at the bottom of the pyramid.

Not all tests are built the same. But some companies associate test automation solely with time-consuming UI tests.

The test pyramid is a great way of visualizing how different automated tests should be conducted. You should conduct more bottom-level tests (Unit Tests) than top-level tests (UI Tests).

 

Why?

Higher level tests, involving browsers or devices, take a long time to execute. It is unreasonable to expect all tests to be executed for every code change. High level tests also don’t clearly point to the cause of the failure.

Understanding the root cause of a new test failure may require complex investigation, and that takes time.

Besides, failures could be caused by factors outside of the product, like browser updates, different screen resolutions or intermittent network conditions.

What it boils down to is that maintenance of UI tests can be costly. Even if regressions are being caught by the automation framework, it can be difficult for teams to react quickly, because there is too much noise, and tests are not executed frequently.

The solution is to move away from using UI tests only for regression, and to develop an automation strategy that heavily relies on more bottom-of-the-pyramid tests.

Test Automation Leads To Great Products

In order to optimize test automation, teams need to collaborate when discussing the test plan for a feature, or for reducing bugs.

Such collaboration is easier when everyone involved has visibility and good understanding on what is being tested at each level.

Teams can be successful at reducing higher level tests only if there is awareness and a clear understanding of what is being tested at each level, and lower-level tests are visible and trusted by everyone working on ensuring the product’s quality.

Test automation can do wonders for the product your agile team is working on. Don’t let the myths have you think otherwise.

Get Email Updates

Get updates and be the first to know when we publish new blog posts, whitepapers, guides, webinars and more!

Suggested Stories

Applications of Voice Assistants in Healthcare

Discover how organizations across the continuum of care can leverage the growing consumer demand for voice-enabled devices to achieve an extensive list of objectives from increased patient engagement to improved outcomes and lowered care costs.

Read More

Structuring Multidisciplinary Software Teams

5 strategies we've learned from working with the biggest names in software for structuring multidisciplinary software teams to get amazing software out the door fast.

Read More

Guide to Creating Engaging Digital Health Software

This guide shares our knowledge and insights from years of designing and developing software for the healthcare space. Focusing on your user, choosing the right technology, and the regulatory environment you face will play a critical role in the success of your application.

Read More