New Maturity Model outlines practices for development testing

To help companies integrate testing into their DevOps cycle, tool provider Coverity recently announced the Development Testing Maturity Model, a guide for implementing testing best practices.

Coverity’s Development Testing Maturity Model works in conjunction with the Coverity Development Testing Platform, but it isn’t solely for users of that platform. Rather, it’s a set of practices for organizations no matter what testing platform they’re using. “Between our open-source experience as well as having worked with a variety of enterprise customers, we’ve come up with this best practices model that the whole industry can use,” said Jennifer Johnson, VP of marketing at Coverity.

The Development Testing Maturity Model outlines a phased-in approach to development testing adoption. “We’re not telling people to just throw in a technology and figure it out or do everything all at once,” Johnson said. “Take incremental steps. That’s how you’re going to be successful.”

The maturity model helps companies find quality and security issues as code is being developed. “We are not saying that we’re going to replace QA testing and security audits, absolutely not. You still need to do them,” said Johnson. “But what this does is it helps remove a lot of the defects in the front end that, right now, QA testers and security auditors waste time trying to fix, like basic bug detection and fixing.”

This helps accelerate the DevOps process, Johnson said, because if you can find the majority of defects in development and fix them before they ever get to a QA tester, then that QA tester can focus on validating that the code works and that it can scale, as well as do load testing. She said it also helps the security auditor to focus on exactly what they were meant to do, which is to look at the software from a security standpoint to make sure that it meets compliance requirements.

“You’ll have more reliability in what you’re actually pushing out into operations. You’re going to have less of a troubleshooting escalation loop back to development once you’re in the field,” Johnson said. “This is because you’re helping everybody eliminate more defects so you get less out in the wild, so to speak. Plus you get more products faster to market, and you get better quality on the back end.”
#!
The Development Testing Maturity Model
Level 1: Automated defect detection: At this level, you take a snapshot of where you are today, use that as your baseline, and then put a policy in place so that you will not introduce any new defects. Johnson recommended that you do this with your nightly builds.

Level 2: Identification of residual risk: This level brings unit testing into the equation. Unit testing, Johnson said, is like a version of the functional testing that QA testers do. But with unit testing, it’s the developers who do the tests. When developers are coding, she said they should be writing a script that helps them automatically test the unit. If developers want to start doing unit testing, Coverity will help them understand where the risk is in their codebase.

Level 3: Developer workflow optimization: This level is where the testing platform is more integrated with the systems that developers are already using in their environment, such as source control, bug tracking and IDEs. Because of this integration, Johnson said developers’ workflows become more automated. They receive automatic notifications when defects are introduced, those tickets are automatically sent to their bug-tracking system, and their source-control-management system lets them more deeply analyze the code changes they need to make.

Level 4: Code governance: This level applies to any organization that has a supply chain. First, you set code policies and principles for your in-house development team. Then you hold your suppliers accountable to those same policies and principles. Johnson said your supply chain can include any packages of open-source software you may be using, offshored or outsourced development teams you may have, or any other third-party suppliers that are giving you code.

Level 5: Enterprise code assurance: This is the level at which you go back and select the defects that you want to fix. In some cases, legacy defects are not so urgent, Johnson said, but as you write new code and you have new functionality, it could be that code dependencies make a legacy defect worse. Here your build will fail if you have any legacy defects that conflict with new code that you have introduced, she said.
#!
Five New Coverity Verification Services
Coverity also recently announced five new code-verification services, which can work in conjunction with Coverity’s Development Testing Maturity Model.

1. Coverity Supply Chain Audit Service: In the Maturity Model, Level Four is Code Governance, where you get your supply chain vendors to meet your policies. “This is kind of the same thing, but it’s a point-in-time snapshot,” said Johnson. “So if you were earlier on in the security curve, like at Level One or Level Two, but you had a supplier you thought might be problematic, you could have this service done. We’ll give you a report of what’s in [their] code.”

2. Mergers and Acquisition Due Diligence Audit Service: This is virtually the same service as the Coverity Supply Chain Audit Service, but it’s done for a different purpose. “If you’re looking at acquiring a company and you want to understand the quality of their software before you decide to purchase them, we’ll do a one-time scan and give you a report of the quality,” Johnson said.

3. Security Service: This is a service where Coverity looks at the OWASP Top 10 and CWE Top 25 and tells you what kinds of automated technology you could bring into the development cycle to help you meet the security issues or defects that are relevant to your application. “People will say you have to meet all OWASP Top 10 and CWE Top 25, but actually it depends on the application,” Johnson said.

4. Food & Drug Administration (FDA) Product Implementation Validation Service: This service is for medical device companies. “As part of the FDA approval process, not only do they need to show that the quality of their software is high, but they need to use static analysis as part of the FDA approval process,” Johnson said.

5. MISRA Service: The Motor Industry Software Reliability Association (MISRA) is a standard for quality for the automotive industry. “The MISRA Service is a custom deployment where we’ll look at an organization’s application and the pieces of MISRA that are applicable to them,” Johnson said. “We’ll develop customized checkers to find those types of issues on an automated basis.”