Automated testing enables software testing teams to test faster, but these tests are not always perfect. The challenge for companies then is, how do you quickly analyze those failed tests and eliminate any bottlenecks in testing? Sauce Labs is addressing this problem with its newly released Sauce Labs Test Analytics, a new solution that expands data analytics and enables teams test faster, smarter, and more efficiently.
Sauce Labs Test Analysis is the latest addition to the Sauce Labs Automated Testing Cloud, and it’s designed for developers and QA teams, as well as managers and business leaders.
For developers and QA teams, Sauce Labs Test Analytics gives them the ability to narrow down builds and tests that are error-prone. It also lets them identify platforms with high failure rates, and improves parallelization efficiently for their tests. Teams can use this solution to analyze test trends, and see what’s happening with particular browsers over a long period of time, instead of what is just happening during testing that day.
“Maybe [teams] find that there is a browser operating system combination or there is a group of testers or testers in particular geo that aren’t testing as efficiently as other testers,” said Sauce Labs testing expert Ken Drachnik. “Management can go back in and say why are they testing differently than other people, and make sure that essentially, testing is done uniformly across everybody within the organization.”
Developers are also able to drill down on the dashboard and identify what particular tests are failing, why they are failing, and if they are failing repeatedly. Developers can use this information to troubleshoot tests faster and get to the root of the cause quickly, said Drachnik.
For managers or vice presidents of engineering, for example, Sauce Labs Test Analytics gives them visibility into tests coverage per business requirement. It also can reduce errors and failures per team.
According to Drachnik, managers really want to know how their teams are doing. They want to know the quality of the code, and they aren’t as interested in knowing how many tests pass or fail or how quickly it takes to identify an error. The real data that managers need, according to Drachnik, is how soon can they ship software, and is that software conforming to the requirements.
“I think what a lot of people want to know, [especially] if I am a large company with teams around the world, is how do I compare testing by teams,” said Drachnik. “You are not there on a day-to-day basis to evaluate [testing teams], or see if different groups have different efficiency ratings, different levels of parallelization in their tests, or if they are doing things differently. Sauce Labs Test Analytics is a way to identify those areas.”
Sauce Labs Test Analytics also comes with benefits that can help enterprise teams test faster and ensure the quality of both the tests and the applications. There is a powerful REST API that lets users aggregate analytics data into their dashboard or test analytics tools. In addition, teams can drill down into specific problem areas with multi-dimensional filters. The filters provide quick insight into the failure or success rates of the build, platform, browser, department, or user. Teams can also expand the visualization of test metrics with the interactive click and drag GUI of Sauce Labs Test Analytics, according to Drachnik.