Testing is a crucial piece of the software life cycle but QA teams often can’t produce enough test coverage quickly enough because they are bogged down by test maintenance. Test maintenance often takes more than 50% of a QA team’s time.
testRigor, a plain English-based testing system, is easing the pain points of QA teams by reducing grunt work and overhead, and accelerating the speed of delivery with its test maintenance and test generation solution.
testRigor was designed to autonomously generate tests and dramatically reduce the need to maintain those tests. “Our tool is a functional end-to-end testing tool for web and mobile designed to automate away the work of manual testers,” said Artem Golubev, CEO of testRigor.
Test maintenance doesn’t have to be time consuming
When the company set out in 2015, it had a mission to help customers expand their test coverage and generate thousands of tests, but the company quickly found test maintenance as the number one problem it needed to solve. “We realized we needed to help our customers maintain our tests, otherwise those tests would be completely useless and thrown away,” said Golubev.
What the company saw was QA teams spending man-years building out hundreds of tests and then being completely bogged down maintaining the code. If there was a major change to their solution such as changing the checkout process from just clicking “buy” to a flow involving clicking “add to cart” and “Cart,” then a majority of the tests would fail and the company would a need to invest more man-months adapting thousands of tests to the new flow — only to resort to manual testing only in the meantime.
“You have a huge test suite with tons of useless code that almost all fails. Do you want to spend years fixing it or do you want to try to figure something else out? So people figure something else, and basically get back to manual testing because that is the only way they can fix them,” Golubev said.
This results in QA teams having to use valuable time to maintain regression tests, or the business decides it’s not worth the time and falls back to manual regression. In fact more than 50% of QA engineering time is spent on test maintenance. As a result, according to Golubev, about 70% of functionality is still tested manually. “It’s as if you’d need to stop and fix your car every mile on the road,” Golubev added.
“That is a big problem for companies. We want to help companies solve that in a meaningful way,” Golubev continued. According to Golubev, testRigor can help manual QA testers build tests 20 times faster than with Selenium and reduce test maintenance by 200 times vs Selenium.
On top of the plain English, testRigor provides a browser plugin that can be used to record tests while testers are performing their manual regression tests that further speeds up the test creation.
Test generation that ensures you’re covered
Another problem QA engineering teams face is making sure they have created enough tests that cover all the functionality within a solution. Test creation can be hard, expensive and inefficient if not done properly, Golubev explained.
With testRigor, customers can not only build up to 1,000 tests anywhere between two weeks and two months, but it uses artificial intelligence to constantly learn from end users and create tests based on most frequently used end-to-end flows from production. Because it is not dependent on white-box information like XPaths, tests are also more stable and adaptable.
“Tests are automatically created based on mirroring how your end-users are using your application in your production plus tests which are produced to map your most important functionality out of the box. It is achieved by using our JavaScript library in your production environment to capture metadata around the functionality & flows your users are taking,” according to testRigor’s website.
This helps also to ensure that the tests that are generated are actually helpful and reflect the most important areas, eliminating the question of what needs to be covered. According to Golubev, legacy test approaches usually struggle to provide more than 30% of test coverage, while testRigor provides more than 90% of click-through coverage out of the box.
The solution leverages the same low-code, low-maintenance, plain English-based platform that customers can use to build their tests manually. Its JavaScript library and browser plugin ensures tests cover the most frequent and business critical functionality and flows, and tests are run in parallel so prioritization doesn’t become an issue.
Additionally, because testRigor uses plain English-based tests, users can easily see what happened, and explore the tree of paths to find out which paths are covered without having any coding experience.
All the tests can be run on a branch, test or production environment within minutes by running tests in parallel.
“It’s like if you had to pedal your bike to go faster. It is a known problem for you unless you invent an engine. We are that engine for pedaling your bicycle,” Golubev added.
A new era of testing
With the test maintenance, creation and automation issues solved, testRigor is hoping to usher in a new era of intelligent testing.
For instance, many automated testers look to the testing pyramid as an example on how to create a balanced testing strategy. Typically, at the top of the pyramid, you have end-to-end testing, followed by integration testing and unit testing. The wide-spread belief is that you can’t have a lot of end-to-end tests because they are flaky, slow to create and pain to maintain.
testRigor wants to take the testing pyramid and mold it into a testing hourglass, expanding the end-to-end testing portion. The reason why end-to-end testing is just a small portion of the testing pyramid is because typically end-to-end tests are too hard to build and too hard to maintain, according to Golubev. With testRigor, Golubev explained it has solved the stability and maintainability aspects of end-to-end testing as well as the ability to generate tests based on actual end-user behavior in production, paving the way for more end-to-end testing.
“We are allowing customers to have a lot of end-to-end tests because we believe this is exactly how systems should be tested otherwise you don’t get actual proof that stuff actually works on behalf of your end users,” Golubev said. “In 2020 and beyond, it is paramount to be able to move faster. You can’t resort back to manual testing anymore. People that move faster end up not only with less issues, but also have positive business impacts.”
testRigor’s four principles for any testing system in 2020 (and beyond) are:
- No setup required, eliminating painful onboarding and test environment creation
- The ability to generate tests when possible
- Never failing without a good reason. Flakiness is not a good reason for test failure.
- Ability to maintain tests easily.
“We believe that a great software testing system should help you to move faster, not slower. And this is why and how we built testRigor,” the company wrote in a post.
The testRigor system is particularly good at acceptance-level functional UI-level regression tests. It is complementary, not a replacement, for unit and API tests, and supports calling APIs and APIs testing. Being able to perform exploratory tests and regression testing faster can be a huge differentiation factor between your competitors, Golubev explained.
“Rather than dedicating testing resources to this type of tedious, repetitive & time-consuming work, you can offload that testing to testRigor and re-deploy your testers to your core testing needs,” the company wrote.
Other features include SMS, phone call, audio, email, and downloaded files testing. It can also provide tests for systems teams don’t control the underlying code for such as Salesforce, MS Dynamics, SAP implementations and RPA scenarios.
Learn more at testrigor.ai
