Software testing no longer stands alone

Long gone are the days when a tester would sit on one side of the wall and wait for the developers to throw code over. Today’s testers are not only wearing their testing hats, but also the developer’s hat, the security team’s hat and IT’s hat.

“Testers need to be able to pick up any hat depending on what they need to do to get the current sprint done,” said Caleb Billingsley, senior vice president of performance, data and delivery at Orasi.

Being a software tester is no longer about preventing your team from shipping bugs, but it is becoming a much more strategic function, according to Steve Hazel, CTO of Sauce Labs. “In the modern development shop, quality is everyone’s concern because of the impact of poor quality on the entire development pipeline,” he said. “The modern day tester is an engineer. They have development experience, or can think in terms of development and code.”

(Related: How testing works in an agile world)

But as testers are adjusting to their new roles and responsibilities, they need to remember that their part on the team is becoming more important than ever, not less. Just because their roles might start to look more like a developer, their obligation as a tester doesn’t diminish, according to Jason Hammon, director of product management at TechExcel. “The need for testing is higher than ever because as the release cycles are getting shorter, as the number of devices are getting larger and as platforms are getting larger, the amount of software issues that could affect people’s personal data gets higher and higher,” he said.

Even as developers start picking up more testing responsibilities, and everyone on the team is more cross-functional, testers are still going to be the ones who can provide better insight and more experience with software tests.

“It is a specialty,” said Billingsley. “If you spent your whole career testing, when you get into the critical phase of finding defects or relevant problems that are going to impact users, that’s the software testers’ world. That’s what they live in. It is just built into their DNA. They love to find problems.”

Today’s testers are more of a facilitator of quality, according to Tom Lounibos, CEO of SOASTA. They can bring their expertise and skills to provide advice and information to developers about how they can be better. “Testers are actually becoming performance engineers more than they are test engineers, and that is a much higher-value solution,” said he said.

Baking testing into the software development process is a huge competitive advantage for companies, according to Hammon. “By finding issues earlier in the life cycle through continuous testing and better planning on the testing side, you are able to identify issues early and allow the development team to fix them while you are still in your normal development cycles, as opposed to having to delay a release,” he said.

Traits employers are looking for in today’s modern testers
New responsibilities come with new skillsets. Here are what the experts say today’s software testers need to have in their toolbox:

The ability to write code: Testers are increasingly becoming more accountable for development responsibilities, and a good understanding of how code is written will help them understand the development process better and help them write better test cases, which will result in better user experiences, according to Rod Cope, CTO of Rogue Wave.

“It is becoming more and more common that testers are able to actually write code,” he said. “They will [have to] write low-level code in the same language as the developers do in order to test at a very intimate level with the code that is being written.”

More technical skills: “Quality teams are gaining a higher profile, and they have more input into making software testable earlier in the process,” said Sauce Labs’ Hazel. “With that comes the necessity for greater technical sophistication. QA will need to focus more on creating solid test automation infrastructure and building a testing strategy, rather than on the execution of tests.”

Traditional qualities: Just because they don’t look like the traditional tester anymore doesn’t mean the qualities they possessed as a traditional tester went away. They still need to be detailed, able to manage a complex inventory of test assets, and have a good sense of where their focus should be turned to, according to Hazel.

The ability to automate: If testers aren’t learning how to code, they at least need to learn how to automate tests in whatever tool they are using. Test automation involves a certain level of programming skills. Without the ability to automate, testers are going to have to perform a lot of time-consuming manual tests to gain insights across their different browsers, platforms and environments, according to Cope.

A focus on the business value: Testers need to understand whether or not they are testing software that is irrelevant to the company, according to Billingsley. “Instead of a flawless, ‘We shall test everything’ mantra, test budgets can be cut by as much as 60% if testers can test based on risks,” he said.

“Testers need to make sure they are hitting the areas that have the most risks or the most activities from a development standpoint,” said TechExcel’s Hammon.

A mindset of the end user: How the end user is going to be interacting with your software is sometimes overlooked, according to Hammon. “Testers need to have a real understanding of the software that they are going to be testing, and a passion about how that software is going to be used,” he said.

Also, testers are going to be able to focus more on strategy and exploratory testing, so they need to think like the user in order to possess good product sensibility, according to Hazel.

Diverse experience: If a tester has been using the same tool for more than a decade, that can look like that they aren’t open to new processes or methodologies, according to Brad Johnson, vice president of marketing and business development at SOASTA. Today’s employers want to see that testers are open to taking new approaches to testing.

The notion of continuous testing: Continuously delivering and continuously testing speeds up development, and speed is a competitive advantage, according to SOASTA’s Lounibos. “You have to continuously test because there are constant changes in your infrastructure and your network by your consumers using different kinds of platforms,” he said. “As you are constantly testing, you are constantly optimizing your application, and that means continuous quality, continuous optimization.”

Where software testing can improve
Software testing has benefited with the creation of new tools and technologies, as well as the introduction of automation and real-time insights, but the space is not done growing yet. “There’s still a lot to do. There’s so much room for better tools to improve software quality,” said Hazel.

One area that still needs a lot of work is how to automate the detection of visual problems, he said. “Today, your user interface can be rendering completely wrong in a way that makes it unusable for humans. But as long as the components are all working underneath the broken visual layer, your automated tests will tell you everything’s fine.”

Applying advancements from other technologies can also help testing go a long way, according to Orasi’s Billingsley. Today’s cars include computers that send a report to the driver if it detects a problem with itself, but having the same thing built into software testing can have real value, he explained.

“If we come to a point where we start building self-diagnostics into software, and where there are standards instead of everyone having their own way of doing that or not at all, I think it will really revolutionize how testing is done,” said Billingsley. “It would allow the tester to really focus in on the system and tell us where it is not healthy.”

Another major area testing needs to address is the ability to unite descriptions of features with test cases, according to Hazel. “Things like feature requirements or user stories capture most of what’s unique and important about the intended functionality, so it makes sense to look for ways that they can be more easily translated into automated test cases,” he said.

Aside from tools, TechExcel’s Hammon believes that as teams start to get smaller and more connected, they lose sight of great test planning, and that’s where organizations can improve. Test plans need to consider the multiple platforms the apps are going to run on, how they are going to solve the problems that arise with the different platforms, and how to get adequate coverage on them, he explained.

The software testing industry is advancing, but it isn’t finished yet; this is only the beginning, according to SOASTA’s Lounibos. “Ultimately all of us are dramatically changing the technologies and the approaches of delivering quality user experiences,” he said. “The focus is all about the user experience; it is all about the customer experience. That is the biggest driver of all because you have so much revenue associated with getting it and doing it right. We are only in the first inning of this whole new evolution of how we test applications.”

Staying out of the headlines
Almost every month or every week, a new headline comes out about the latest software vulnerability that sends security researchers into a frenzy. But software security is just as much of a responsibility to the software testing team as it is to the security team, according to SOASTA’s Johnson.

“One of the challenges I have had with the testing community is that mentality of ‘Oh, it’s not my problem at the end of the day,’ but it is,” he said. “It is important for the growth of the testing industry for testers to really take ownership that they work for a company that is selling or delivering a service.”

First and foremost, testers need to be aware of known vulnerabilities and run security tools against every build looking for them, according to Orasi’s Billingsley. “There is no excuse for a SQL injection hack,” he said. “That is so well known and there are so many good tools to find that. It should be a thing of the past.”

But going beyond known vulnerabilities is where making sure software is secure becomes tricky. When it comes to security, testers need to put their black hats on and think like a hacker, according to Rogue Wave’s Cope. To think like a hacker, they need to understand how they hack. He explained that testers should take a look at the stories that come out about hacks, try to figure out the root causes, and then try to reproduce them.

“Don’t necessarily listen to these developers who say, ‘Well this part of the code is good,’ ” said Cope. “Assume it is all vulnerable all the time, and try to be malicious with the code.”

As hackers get more and more intelligent, companies should also set aside some time for testers to stay on the cutting edge of what a hacker is up to, according to TechExcel’s Hammon. He explained that being aware of what is going on the outside of an application and being aware of new security issues is an important part of being a software tester.

“Make sure that if something is changing the way a browser works, and you are using plug-ins, that may not be caught by the development team,” said Hammon. “Hopefully the QA team has educated themselves to make sure that they are catching those issues.”

And as new technologies such as the Internet of Things (IoT) emerge, the surface area for vulnerabilities and hack attacks is going to become a lot larger, and it is going to be a lot harder to test everything, according to Cope. He suggested testers trying to tackle IoT should take a modular approach to testing and start with the most vulnerable areas in each device.

“Over time you will see developers and testers working together to create tested components that are safe, and they can reuse those modules so that they are not reinventing the wheel with every new device and having to reinvent the wheel with testing every new device,” he said.

Even with all these practices, Cope says that there will always be headlines because hackers are continuously motivated to break in, while the victims only have to slip up once. But that’s no excuse to push security to the side.

“You don’t want to be the next company on the evening news or on the front page of the newspaper,” said Cope. “So more time, more effort put into security is going to be one of the key answers.”

Testing as a microservice
Many organizations are turning to microservices in order to break down their monolithic applications into smaller components to achieve a faster delivery life cycle. But others are finding the same benefits in turning their testing services into microservices. SOASTA is bringing microservices to its testing solutions in order to shorten the testing process.

“We are moving testing into being these small incremental tests, not these five- to six-hour tests or six weeks of doing tests, but these micro-tests that literally take seconds,” said Tom Lounibos, CEO of SOASTA. “Developers can be running thousands and thousands of tests on a daily basis going through their system, and that is a whole different way of doing things.”

Other than speed, he noted that moving to a micro-testing approach also improves quality because with the amount of tests that can be done in a short amount of time, organizations will have a much better understanding of what is going on in their software. “Doing thousands of tests means different angles, different views,” he said. “You are constantly testing, and the more you test, the more it delivers a much better user experience for your consumers, and the more you will be able to anticipate problems better.”

It’s also important to mention that applications will not need to be built on a microservice architecture in order to utilize micro-tests, according to Lounibos. And while he believed micro-testing will become more common in the future, there will still be a need for more traditional testing approaches.

“It will not be the only type of testing performed,” said Lounibos. “Functional, load, stress, performance testing on apps will always be important to delivering quality user experiences. However, [micro-tests] will most likely be the most frequently used form of testing of Web, mobile and IoT apps in the future.”

What are the biggest trends in the software testing industry you are seeing?
As the way we develop software is constantly changing, testing also has to change to keep up. SD Times spoke with some experts to talk about the trends they see around software testing and what we can expect next.

Brad Johnson, vice president of product marketing and development at SOASTA: Testers who are focusing on a career in technology need to be more technical, period. There are fair arguments for less tech skills for user acceptance testing or perhaps some aspects of exploratory testing. But because the gap is closing between developers and testers, more testers are creating value by helping developers with technological approaches that help offload the early testing efforts—using continuous integration, for instance—and introduce concepts around performance testing and earlier-stage functional test automation. Testers are aspiring to become cross-functional quality advocates that span the delivery life cycle.

Additionally, performance is becoming everybody’s business, so testers who don’t consider how fast or performant components and functionality are, or how to measure and report on that, are finding themselves left behind as DevOps teams keep moving faster.

Caleb Billingsley, senior vice president of the professional service organization at Orasi: We do see kind of an inversion of where companies used to do, let’s say, 80% of user interface testing, and now they are basically doing a kind of true agile inversion of that. They are doing 10% to 20% user interface testing, and then they are trying to put that 80% down in the middle tier and the API tier, so you see a huge push for that, especially with agile. Not all companies are true agile, but they are adopting an agile testing paradigm and testing much more of the API in the middle tier, the Web services tier, and trying to get uncoupled from so much of the user interface automation.

The challenge of user interface automation is very time consuming and costly to maintain. There is value there, but every time the user interface changes, you have to adapt it, so that is definitely a trend that we see pretty much across the board with any company that is looking to gain efficiencies.

Jason Hammon, director of product management at TechExcel: There is a lot more mobile applications and multi-platform testing that has to happen just because so many people are using their phones or tablets in addition to the traditional PC or Mac OSes. You have to make sure you are supporting these other formats that people are interacting with your applications on, and just in general development cycles are getting shorter and shorter with some mobile application providers even running every single day with new builds, so there are lots of challenges in that regard. Testers have to make sure that they are planning well enough to accommodate the fact that they are going to have to test on these multiple platforms.

Rod Cope, CTO of Rogue Wave: Over time you are going to see testers using more development-type tools to get their jobs done because they are actually starting to write more code. Maybe we will see in the future debugging tools for testers and static code analysis tools for testers, things like that.

And then when things like the Internet of Things come, testing will become more important because security is getting more important. All these things are starting to talk to each other without as much human intervention and involvement. There is so much new surface area for a hacker to get in. All he has to do is slip in one little crack, like your tire pressure monitor, and he can take over your car, or open your garage door and get into your house. I think you are going to see a lot more tool use, more scrutiny around security, more needing to prove compliance of standards and more testing all that to make sure these devices are actually safe in the real world.

Steve Hazel, CTO of Sauce Labs: Software testing is undergoing a period of dramatic change in the wake of the adoption of agile methods and DevOps. Practices like Continuous Integration and [Continuous] Delivery are driving an unprecedented demand for the reliability and speed of automated tests. More and more organizations are going to make the move not just to test automation, but to view tests as a key part of the codebase.

At the same time, testing is moving earlier in the pipeline, so that bugs are resolved more cost-effectively and progress is more reliable.

Finally, all of this means that the skill set for testers is evolving. QA roles are changing and going towards [quality engineering].

A guide to software testing tools
Applause: Applause provides in-the-wild testing services that go beyond the traditional QA lab to create a great experience for app users where they live, work and play. With functional, usability, localization, load and security testing, Applause services span the entire app life cycle. Through a global community of more than 175,000 professional testers in more than 200 countries and territories, Applause empowers companies to understand how apps will work in the real world, with real people on real devices, across locations and use cases that match actual users.

Coverity: Coverity provides advanced static analysis algorithms designed to detect critical defects specific to each programming language. Test Advisor—Development Edition finds and fixes more issues earlier in the life cycle, improves efficiency of automated testing, and reduces the risk of costly software failures. Test Advisor—QA Edition helps QA teams improve manual and automated testing efforts for .NET and Java apps. Code Advisor detects quality and security defects in the development workflow.

LogiGear: LogiGear provides a variety of testing solutions that range from test automation and mobile testing to code-level testing. With its TestArchitect products, users can rapidly create and automate large-scale tests for desktop, mobile and Web apps. Features include the ability to test multiple platforms or versions with a single test case, the ability to monitor performance as tests are executed, and the ability to create reusable test cases.

Neotys: Neotys is a leading innovator in load testing and performance monitoring solutions for Web and mobile applications. Neotys’ products NeoLoad and NeoSense enable development, QA and IT operations to quickly and efficiently test and monitor the quality, reliability and performance of their applications. More than 1,500 organizations globally have selected Neotys’ solutions because they are agile, easy to use and support all RIA and Mobile technologies.

Orasi: Orasi is a leading provider of software testing services, utilizing test management, test automation, enterprise testing, Continuous Delivery, monitoring, and mobile testing technology. The company is laser-focused on helping customers deliver high-quality applications, no matter the type of application they’re working on and no matter the development methods or delivery processes they’ve adopted. In addition to its end-to-end software testing, Orasi provides professional services around testing, processes and practices, as well as software quality-assurance tools and solutions to support those practices.

QASymphony: QASymphony offers a suite of testing tools built from the ground up for agile. Through its visual-based tools, QAS provides insight into exploratory, automated and scripted testing details. qTest eXplorer records and documents exploratory testing, and also saves time when performing traditional manual testing. QASymphony also provides qTest, a scalable test-management solution that allows organizations to coordinate, track and manage multiple software testing projects and teams across multiple locations in real time. QASymphony’s newest innovation, qMap, allows management to visualize high-level information about the health of the release as development is occurring in a mind-map-like tool.

Rogue Wave: Rogue Wave offers a number of solutions that make it easier to test complex code. OpenLogic helps ensure the quality of open-source software with open-source management and governance, in addition to a security vulnerability update service. Klocwork allows testers to easily detect security, safety and reliability issues in real time with static code analysis, and it also provides reports and metrics to understand what issues should be prioritized. TotalView Debugger gives software testers a better understanding as to what is going on in their apps by providing control over processes and thread execution along with deep visibility into the application and data.

Sauce Labs: Sauce Labs provides a cloud-based platform for automated functional and JSUnit testing. Optimized for use in CI and CD environments, and built with an emphasis on security, reliability and scalability, Sauce Labs helps organizations of all sizes eliminate the time and expense of maintaining test infrastructure. Users can run tests written in any language or framework using Selenium or Appium, both widely adopted open-source standards for automating browser and mobile application functionality. Videos, screenshots and HTML logs help pinpoint issues faster, and Sauce Connect allows you to securely test apps behind your firewall.

SOASTA: SOASTA’s cloud-based platform delivers key testing capabilities uniquely suited to the next generation of Web and mobile applications. It provides real-time analytics, visual test creation, cross-cloud grid provisioning, and the ability to start, stop, pause and restart tests. CloudTest allows testers to build, execute and analyze performance tests of any scale inside the dev lab, all the way to live production. TouchTest delivers mobile functional test automation capabilities for multi-touch, gesture-based mobile applications. mPulse connects and analyzes 100% of the performance data from every Web and mobile user, and it correlates this data with critical business metrics—all in real time.

TechExcel: TechExcel takes quality to the next level with DevTest. With DevTest, testers get full control over every detail of their testing process, from planning and managing to analyzing. It gives insights into what has already been tested and what still needs to be tested, increases team productivity, and ensures accountability. Features include comprehensive test coverage management, automatic defect submission, built-in reports and analytics, test scheduling and assignment, a Web-based architecture, and automated testing integrations.

Telerik: Test Studio and Mobile Testing are sets of test automation tools that help teams be more efficient in regression testing, improving test coverage and reducing the number of bugs that slip into production. The company offers automation solutions for GUI, load and performance testing for desktop, Web, hybrid and native apps running on any device (Android, iOS or Windows Phone). Tests can be authored in C#, VB.NET or JavaScript.