4 Automated Testing Pain Points (and How to Solve Them)

Posted Oct 2nd, 2018

If you’re like most QA teams, you’ll discover that adopting automated testing requires clearing some hurdles. That is because like most innovations, test automation creates some challenges that organizations need to overcome before they can reap the full benefits of an automated testing strategy.

Let’s take a look at some of the most common pain points that arise when QA teams begin implementing an automated testing strategy, along with tips for addressing them.

1. Lack of Automated Testing Expertise

Perhaps the most common pain point that QA teams tend to experience when they move to an automation-based testing strategy is a lack of expertise with test automation.

This challenge is to be expected. If your QA engineers were previously accustomed to performing testing by hand, they likely lack experience in writing automated test scripts or using automated testing frameworks, like Selenium and Appium.

The obvious solution to this challenge is to educate your QA engineers, of course. But you don’t have to limit your approach to forcing your QA team to take tutorials on writing test scripts or pore over Selenium documentation.

You might also think about how to leverage the coding skills that your organization already possesses to help your QA team learn to write automated tests faster and more effectively. Ask your developers to assist in the automated testing learning process. The coders you have on staff may not have backgrounds in QA engineering, but they should be able to wrap their heads easily around writing test scripts and learning a new scripting framework. By involving developers in the automated-testing migration process, you can speed your QA engineers’ ability to gain the skills they need in order to take full advantage of automated testing.

2. Lack of Automated Testing Infrastructure

Another common challenge that organizations face when moving to automated testing is a lack of infrastructure for executing their tests.

To solve this problem, you have two options. The first (and the one to which you might be most tempted to turn) is to build your own automated testing environments by setting up software like Selenium server on your own on-premises infrastructure.

That approach works, but it’s less than ideal, for two reasons: First, setting up automated testing software will take time, especially if your team doesn’t have prior experience configuring these tools. Second, an on-premises test infrastructure is difficult to scale. If you plan to increase the number of automated tests that you run, you’ll have to add more infrastructure, which will be costly and take time.

The other, better approach is to use a testing cloud, where the environment is already set up to let you run tests using any framework you choose. An automated testing cloud is also highly scalable; you can use as much infrastructure as you need to run as many tests as you want at the same time.

3. Not Knowing Which Test Framework to Use

If you’re new to test automation, you may wonder which automated testing frameworks to use. There are a number of frameworks out there, including both open source and proprietary options, and their functionality overlaps to a great extent.

Obviously, I can’t tell you which frameworks are best for you. You’ll need to do some research and probably experiment with different frameworks before deciding.

What I can recommend, however, is adopting a test infrastructure that will allow you to switch between different testing frameworks as needed. This flexibility is another advantage of cloud-based testing. If you set up a testing environment based on a certain framework on your own infrastructure, you’ll have to overhaul everything in order to switch to a different framework. But when you use a public cloud that supports multiple frameworks, you can switch to a new framework easily. You may have to modify your test scripts, but you won’t need to set up a whole new infrastructure to run them.

So, as you determine which testing frameworks to use, make flexibility a priority. The framework that you settle on at first may not prove ideal. Or you may want to adopt additional frameworks in the future—there is no rule that says you can only use one automated testing framework at once.

4. Deciding Which Tests to Automate

Realistically speaking, virtually no organization automates all of its software tests. You’ll almost certainly always perform some tests manually.

That means that you need to identify which tests are the best ones to automate, and prioritize them. When making this determination, consider the following factors:

  • Which tests do you perform most often? These are the best to automate.
  • Which tests involve purely technical elements (such as whether an application starts as expected in a given browser), as opposed to tests that are subjective in nature (such as how users interact with a UI)? Purely technical tests are easy to automate; those that have subjective elements may need to be performed manually in many cases. (This is not to say that you can’t automate UI tests (you certainly can—but in some cases, you may want your QA engineers to perform UI tests manually in order to pick up on nuances that automated test scripts would miss).
  • Which tests take the longest to perform manually? Automating the most time-consuming tests will deliver the greatest benefits in efficiency.

Your mileage will vary, of course, but in general, these pointers are a good place to start when you’re deciding which tests are the best candidates for automation.

For more tips on choosing what to automate, check out this presentation by Angie Jones.

Chris Riley (@HoardingInfo) is a technologist who has spent 15 years helping organizations transition from traditional development practices to a modern set of culture, processes and tooling. In addition to being an industry analyst, he is a regular author, speaker, and evangelist in the areas of DevOps, BigData, and IT. Chris believes the biggest challenges faced in the tech market are not tools, but rather people and planning.

Written by

Chris Riley


Automated Testing