DevSecOps has come to be known by many as the shifting left of security, making it a key part of software development while code is being written, as opposed to trying to put security onto the application after it’s completed.
This follows the trends of DevOps, which moved operational considerations for applications into development, as well as software testing — though the term DevTestOps hasn’t really caught on.
And DevSecOps, like many initiatives in their early stages, has awareness but often is not well understood. “People recognize the term DevSecOps and have a general notion of what it means,” said Jeff Williams, co-founder and CTO at Contrast Security. “It means shifting left and automating security somehow, but in practice we’re really just at the very early stages of this. I think most folks don’t have a very well-formed idea of exactly what they need to do DevSecOps. In fact, I think most people get it dramatically wrong.”
Shifting left, of course, puts even more responsibility on developers, who have been trained to write code, and even find and correct errors in code, but who largely have not been trained on security best practices.
“People have this idea that shifting left is taking the things you’re currently doing and pushing them on to the developers,” Williams said. “I call it shitting left. It doesn’t work. The way that security works today is largely built around experts, using expert tools. You can’t just take those same tools and shove them onto developers who don’t have the skills or background to be effective using them, and expect great results. All you’re going to do is create a lot of alienated developers who don’t do good security. You’re probably going to end up hurting security overall.”
That could be problematic for organizations where security is a priority, as 73 percent responding to a recent Forrester study said it is. Trusting developers alone with security is the wrong approach, as developers are often the ones introducing insecure code into their applications through the use of open-source components.
The Forrester report found there were 17,308 vulnerabilities published in 2018, up 23 percent from a year earlier. It also indicated that a best security practice would be to have developers reduce the attack surface in their applications as they code. “Today’s reality is that developers don’t code securely,” the report stated. “When measured against major industry vulnerability standards, 70 percent of applications fail security testing on the first scan.”
None of this is to say developers are at fault here. Forrester noted that the top 40 computer science programs in the United States do not require secure coding or application design in their curricula.
Yet Williams cautioned against taking too much of a developer-centric view of DevSecOps, and noted that many people trying to do it correctly simply forget about the ‘extending right’ part of DevSecOps.
Particularly in application security, Williams said most organizations don’t have any idea of who’s attacking them, or what attack vectors they’re using to go after them, or what systems they’re targeting. Large organizations, he said, are blind at that level and they don’t have a way of stopping those attacks or using intelligence of how they’re being attacked to drive their security strategy. “It’s that kind of feedback loop that’s one of the real characteristics of DevOps,” he explained. “Someone attacks you using a new attack, that should instantly drive changes in the product. But we don’t have that feedback loop, so really, when I think about DevSecOps, it’s about continuing to do what you do now — generate assurance — but extend left and extend right. This idea of shifting left is dumb, and dangerous. It’s unfortunate, but those [advocating shift left] are people who haven’t thought it through very well.”
Doing DevSecOps effectively
Today, many organizations are just taking the DevSecOps name and pinning it on trivial modifications of what they’ve been doing, and it’s not really that, Williams said. “DevSecOps is a fundamental transformation of security the way that DevOps is a fundamental transformation of the way we build software,” he explained. “My friend at Comcast runs their security program and says, ‘Vendors are putting DevSecOps lipstick on a traditional security pig, because they’re not fundamentally changing how their products work; they’re just kind of taping them onto a DevOps pipeline and going, ‘Yep, we’re DevSecOps! Look!’ ”
Williams went on to say the organizations that are doing DevSecOps effectively are being smart about security across the entire software life cycle. “In dev, what that means is you empower developers to find and fix their own vulnerabilities, fix their own code, and check in clean code,” Williams said. “Seems pretty straightforward, and automation is a big part of that. It’s got to be accurate, because developers don’t have time or skills to deal with inaccuracies. If we can achieve that in dev, there are some really good downstream benefits from that. In CI/CD, both traditional and what we might call QA, in that stage, I think the goal has to be to generate assurance; that what you’re pushing into production has been thoroughly tested and is free of vulnerabilities.”
Traditionally, this assurance came from a big test after the application was complete. So by pushing all that to the left, that final assurance is lost. But, if security has been factored in earlier in the process, the big test should find nothing because tests were done along the way and any found vulnerabilities would have been remediated.
There is no assurance, though, that an effort to do DevSecOps effectively will succeed, because — like Agile, DevOps and Value Stream — the methodologies are not prescriptive. Organizations are usually left to their own devices to determine how they are going to realize the benefits.
“There’s some real value in DevSecOps and I don’t want to see the term get watered down to apply to anything that’s security,” he said. “I think it really does mean something. When I go back to the fundamental principles of DevOps, things like breaking down the work into small pieces to create flow, creating tight feedback loops and creating a culture of innovation and learning, those three things, if you interpret them for security, that’s DevSecOps. So that means breaking security work down to small pieces to create flow; it means creating tight security feedback loops, and it means creating a culture of security innovation and learning.
“Those are the three ways of DevOps and I think they’re essentially the same for security,” he added. “But very few organizations are really focused on that. They’re focused on let’s buy some new tool and plug it into our CI/CD pipeline, and bam! We’re DevSecOps. But that is not how it works. You’re not going to achieve a transition overnight. You’re gonna have to do it piece by piece over the course of years.”
DevSecOps, by its very definition, encompasses the entire stack, from coding, to UI, to the infrastructure it’s running on, and Williams added that the whole stack is turning into software. “If you’re deploying into the cloud, you’ve got a container on top of that, maybe you’ve got an app server running in the container, you’ve got libraries in the app server in the container, and you’ve got trusted code running on top of the app server.. but it’s all really software.”
Inside out, outside in, perpetual change
To have an effective DevSecOps practice, you have to approach security at each layer of the stack. If you’re running containers, you’ll need to create rules to ensure that the container has no vulnerabilities, is built with the proper defenses, and that it’s being monitored at runtime.
“The old way we used to do that is with what I’ll call an outside-in approach,” Williams explained. We used to put a firewall around it and scan the shit out of it, and try to see if the whole thing is secure. The problem is, modern architectures are much too complicated for that. I think the effective approach today is to get inside the thing we’re trying to secure. If you’re trying to secure a container, you need to be inside the container asking those questions about security. If you’re trying to secure an app server, you need to be inside the app server. If you’re trying to secure custom code, you need to be inside that custom code. That’s where you have all the information to make a smart decision about whether something is secure or not.”
What Williams described is an instrumentation-based approach to security. Contrast Security, he said, doesn’t do container or cloud security. What Contrast does is instrument the application layer so vulnerabilities can be found and so the team can prevent vulnerabilities from being exploited at runtime.
“If you zoom out and look at that, you can imagine instrumenting each layer of the stack with the right products, and then that stack is secure. It secures itself. And then you can put that stack wherever you want. If you want to put it internally, in an internal data center, great. If you want to put it in the cloud, great. The security goes with the code. For me, we’re talking about securing everything, and that’s a very DevOps/Cloud/Container kind of way of doing security. It doesn’t matter if you’re rolling out tons of elastic servers or you’re spinning up containers all over the place, because the security goes with the code. Trying to do that kind of protection with an outside-in approach is impossible, because you can never keep the walls up around everything, and you can never scan everything from the outside, because what’s in there keeps changing, moving.”
As Williams said, automation must play a big role in DevSecOps, because automation is what creates the guardrails around your development pipeline, to ensure no bugs or vulnerabilities sneak into the code and gets pushed into production. “So you have this automated pipeline that does all that work; that optimizes for the developer the ability to create new functionality and push it into production quickly,” Williams said. “All security, especially application security, has massive scale problems. There’s just not enough people to do the work the old way, so you have to automate. Most big organizations, they’re really only doing effective application security on 10 percent of their applications. They only secure the public-facing stuff, or the ones they deem to be critical. They’re not securing all their applications, and it’s a huge risk. The only way to fix that problem is we’ve got to change the economics. We’ve got to figure out a force multiplier, and I believe that is DevSecOps. By empowering developers, we can use the big machinery of software development to do the security work.”
