Serverless computing puts more power in the hands of the developer. Rather than developing an application and sending it off to IT for deployment, developers can deploy a serverless application themselves, without having to wait on IT operations or their fellow developers.
But serverless as a term can be kind of vague. When people refer to serverless, they’re usually referring to functions-as-a-service, explained Ben Sigelman, CEO and co-founder of APM company LightStep. Popular examples of serverless providers include Amazon Lambda, Google Cloud Functions and Microsoft Azure Functions. Functions are an individual piece of code or programming logic.
According to Sigelman, the basic driver for serverless comes from management. Companies are recognizing that if they are to exist and succeed, they have to be built around software. “There’s certainly a lot of evidence for what happens to companies that don’t become digital companies,” Sigelman said. “If you’re going to develop software that involves hundreds of developers, which I think is also going to be necessary if you want to actually win in that sort of digital economy, you cannot have hundreds of developers working on one piece ofsoftware. They have to be split up into smaller pieces. That’s where the desire for microservices and serverless comes from.”
A benefit of serverless is that it allows developers to deploy code separately, which results in faster time-to-market. “The move to serverless is kind of an extreme way of allowing individual developers to deploy their code separately from each other, and the units of deployment are very, very small because a serverless function is almost the smallest possible thing that you could deploy separately,” said Sigelman. The functions are also, by design, stateless. This means that if they’re deployed correctly, they won’t disrupt your database as they’re being deployed, said Sigelman.
Another benefit that comes from developers deploying code separate from one another is that they don’t have to waste time worrying about other developers’ breakages as they’re trying to deploy their software. This is another reason why monoliths are slow to deploy; everyone in the organization needs to have a clean codebase before something can be deployed, Sigelman explained.
But for all the benefits of having code broken up into small functions, this also causes one big problem: it’s hard to get a global view of the system. Looking at the functions on their own doesn’t tell you much about how the whole system is behaving, Sigelman explained.
“You’ve intentionally designed your architecture to allow for individual people to work autonomously and to abstract away the rest of the organization, but then for things like monitoring and security, you do need to have a global picture in order to make sense of what’s happening in production and serverless has actually made that a lot harder…You should begin the transition, not finish the transition, with some kind of monitoring solution that’s able to understand the big picture of how transactions actually behave in your application.”
According to Sigelman, this is why there has been a lot of disruption in the monitoring and security of serverless functions.
According to Sigelman, the use cases for serverless tend to be very diverse, with developers using serverless to achieve a wide range of functionality. For example, serverless can be used to create ETL (Extract, Transform, and Load) applications, which are applications that take in external data for use in their own applications. Another popular use case is building latency sensitive applications, where a user will tap something in their mobile device, which invokes various serverless functions and then returns that information to the user, Sigelman explained.
Since serverless functions are so diverse, their monitoring strategies are also diverse, Sigelman said. “They’re being used for a lot of different things and depending on the application, the monitoring strategies are really quite different.”
Serverless may result in better code
Unlike in traditional monolithic software, where a developer writes code and then hands it off to IT operations,in serverless, developers are responsible for the code after it’s deployed. That means they are the ones doing the monitoring, typically.
That added responsibility on developers tends to result in cleaner code, Sigelman explained. “When developers move to serverless, they have a natural incentive to build things that are reliable,” said Sigelman. This results in higher-quality software, Sigelman explained.
According to Sigelman, this is the mentality that all software developers should have, whether they are writing serverless applications or are working on monolithic software. “I don’t think it’s a good idea, serverless or otherwise, to have complex software written in a way where people who write software aren’t on the hook for its reliability,” Sigelman said.
Differences for monitoring serverless apps
According to Ory Segal, co-founder and CTO of PureSec, there are two different types of monitoring strategies for serverless applications: performance and security. On the performance side, developers need to make sure that their functions are running and healthy so that they’re not wasting money if functions get held up or aren’t running properly. With monolithic software, this isn’t quite so important because the server is running whether the application is working properly or not, so there is no added cost. But with serverless, where you are paying for your usage, it’s important that applications are always running as they should be.
The second piece of monitoring is for security. Developers also have to monitor applications to make sure that they are secure and to be aware of when there are incidents, Segal explained.
Traditionally, security monitoring was handled by the IT security group. But in organizations with a mature DevOps model, the IT security side of monitoring is moving back into the development organization, Segal explained. “In general, when you’re looking at cloud-native, serverless applications, there’s a trend coming from the development organization, where dev organizations want to be able to deploy faster and not rely on the IT teams to host the systems.”
According to Sigelman, the main challenge with handing serverless applications off to IT operations is that it’s hard to make the IT team mirror the development organization in a way where each service has its own dedicated IT Ops team to go with the development team. “In a more monolithic era, it was easier to train up a separate function to operate your monolith from the people developing it,” said Sigelman. “But since these things are getting split into little pieces, and furthermore, those little pieces are getting [reorganized] and moved around and refactored all the time, you’d have to be doing that operation in parallel in two different organizations — one in development and one in operations — because you can’t really expect a single operations team to properly operate all of these distinct services. So I think that’s the organizational driver for the move to a single role that does both development and operations.”
One of the main differences between serverless or the more traditional methods and development and deployment is that with serverless, you are running software on infrastructure that you have no control over, Segal said.
Many monitoring tools are dependent on your ability to deploy things on the infrastructure. “In cloud-native environments, and specifically in serverless, you have no access to the underlying infrastructure,” said Segal. “You’re basically deploying code and configurations and that’s the only thing you control, that’s the only thing under your responsibility. The cloud provider is now responsible for the underlying infrastructure.”
According to Segal, a lot of the traditional monitoring vendors are now beginning to understand that you can’t deploy solutions into the runtime environment, and that you have to inject monitoring into the code itself.
With IT removed, governance is key
Since the developers are the ones responsible for writing and monitoring their software, theoretically, a developer could deploy something that IT operations or the company doesn’t know about. Therefore, it’s necessary to have some sort of governance plan in place.
“If things are not running on infrastructure that you own, it’s harder for you to know what you have, where you have it and how it is behaving,” said Segal. With serverless, things are running outside of your perimeter, and so they can often be outside of your control.
Sigelman recommends not relying on person management alone for governance. He believes that part of the checklist for deploying a serverless application is making sure that it is discoverable by monitoring solutions. “Even at Google where I think things were pretty buttoned up in terms of this type of process, the only way we could get serverless running in production was by using our monitoring system to do that discovery for us,” Sigelman said. “It wasn’t sufficient to expect people to remember to follow some kind of procedure and keep a system diagram up to date.”
The best way to ensure that your organization knows what’s running is to require that whatever is running in production is discoverable by some sort of monitoring solution,” Sigelman explained. “And then you can look at your monitoring tools to look at what’s running in production. And that’s how operations teams would discover that some team five years ago deployed a serverless application. I don’t think it’s reasonable to expect it to happen any other way, in fact.”
Having knowledge of what is deployed is not just important from an IT operations point-of-view, but also from a security standpoint. “If you don’t know what are the assets that you have deployed, you can’t really protect them properly,” said Segal.
Segal predicts that there will be a way for security teams to have insight into functions being deployed. “I believe that in the near future we will see an effort in helping the security team inside companies, the IT security teams, to be able to gain visibility and monitoring, security posture management, to know what it is that they have, where they have it, who’s maintaining it, whether or not it has vulnerabilities, so they will be able to get themselves back into the loop.”
The transition to serverless
When people move to serverless, there are two important transitions and questions they need to address. The first is the move from monolithic software and the second is the decision to choose serverless or microservices.
The transition tends to look similar regardless of what an organization is transitioning from, said Sigelman. This is because serverless is so different from other architectures, he said. “You have to rethink the way you build your software in general,” Sigelman said.
The desire to want to move to serverless is often due to organizations wanting to have their software development team iterating faster and spending less time on things considered as a “tax,” such as human communication and slow release velocity. With monoliths it’s difficult, or even impossible, to do experiments when you’re having to wait a month between releases. As a result of this, a lot of other efforts suffer.
The second thing organizations need to consider is whether to choose microservices or serverless. Sigelman believes that organizations need to first understand what their performance requirements are. “If you can tolerate some of the performance implications of serverless, I think you can’t beat the simplicity or the elegance of the serverless architecture,” Sigelman said.
According to Sigelman, there is actually quite a large tradeoff in terms of performance that comes from all of the encapsulation that happens in serverless computing. Joseph Heller- stein and other researchers from UC Berkeley wrote a paper on serverless titled “Serverless Computing: One Step Forward, Two Steps Back.” In the paper, the researchers discuss the downfalls of serverless, calling it a “bad fit for cloud innovation and particularly bad for data systems innovation.” In the paper they discuss challenges that must be overcome in order to truly unlock the potential of the cloud.
“They basically document that all of this operational simplicity and elegance comes at a price in terms of the sort of latency you can expect to see out of serverless deployment,” Sigelman said of that paper. “I think serverless is an incredibly friendly architecture to develop on, and with the right tools, a friendly architecture to observe and monitor as well… But it’s still important to choose the applications that are right-sized for current data serverless platforms.”
It’s important to understand what types of applications work well with serverless and which don’t. For example, according to Sigelman, it’s not always appropriate to build user-facing applications on top of serverless. He believes that most organizations figure this out before deploying software, but they may figure it out the hard way.
For those that can’t tolerate the performance implications of serverless, microservices is probably the way to go.
Interestingly, Sigelman noted that while he’s seen more marketing hype over serverless than there is around microservices, he sees more microservices than he does serverless deployments. “They both are receiving a lot of attention from a marketing standpoint, but I think serverless has a louder voice proportionally to usage,” he said.
Sometimes developers will want to know what a certain program will look like written as a serverless application, and they end up liking it and realizing that it is better, Sigelman said. But they might see these positives and not consider the fact that they may be introducing all of this potential latency by splitting up functions, he explained. “There’s a need to move to serverless with your eyes open, both as a developer and from a monitoring standpoint, in order to avoid an overcorrection and a move too far in the direction of serverless.”
According to Sigelman, it’s important to make the transition to serverless gradually and with tooling that can help you understand and track performance. He believes that this is something that is a real challenge for large serverless applications.
It’s also important that when making the transition, you don’t completely segregate developers and IT operations teams, Sigelman said. “I think that it’s important to do that transition along with the cultural shift, where your developers are hopefully responsible for what happens to their code after it gets deployed.”
“Now, because of the way serverless works you have new things that you need to be looking at,” said Segal. “When you ran an application on your own server, you didn’t really care about the cost. The server is running anyway. Suddenly you have something where if you screwed up and somebody managed to find a way to exploit it, you find yourself with a denial of service with the purpose of inflicting financial damage, which is something very unique to serverless.”
12 things to watch out for
Earlier this year, PureSec released a list of the 12 most critical risks of serverless applications for 2019.
- Function event-data injection: Injection flaws happen when untrusted input is sent to an interpreter before it has been executed or evaluated. In serverless architectures, this is not limited to direct-user input; many types of event sources, such as cloud storage events, NoSQL database events, or HTTP API calls, can trigger execution of serverless functions.
- Broken authentication: Applying a complex authentication scheme that provides access control and protection to relevant functions, event types, and triggers, can be a huge undertaking that can be catastrophic if done incorrectly. For example, a serverless application that exposes a set of public APIs that do enforce proper authentication may read content from a cloud storage system. If there is not proper authentication on that cloud storage, the system may reveal an unauthenticated entry point.
- Insecure serverless deployment configuration: Settings provided by a serverless vendor might not be sufficient for a particular application’s needs.
- Over-privileged function permissions and roles: Serverless functions should only have the privileges needed to perform its specific task, an idea also known as “least privilege.”
- Inadequate function monitoring and logging: Logs in their out-of-the-box configuration are not typically well-suited to providing a security event audit trail. In order to achieve adequate coverage, developers have to string together logging logic to fit their needs. Attackers can exploit this lack of proper application-layer logging and remain undetected.
- Insecure third-party dependencies: Third-party packages and modules used when developing serverless functions often contain vulnerabilities.
- Insecure application secrets storage: A common mistake that developers make when storing secrets is storing them in a plain text configuration file, which means that any user with “read” privileges on that file can gain access. Developers may also store secrets in plain text as environment variables, which can leak.
- Denial of service and financial resource exhaustion: Denial-of-service (DoS) attacks have skyrocketed in popularity over the past decade. For example, in 2018, a Node NPM package, AWS-Lambda-Multipart-Parser, was vulnerable to regular expression denial-of-service attack vectors and gave attackers the ability to time-out AWS Lambda functions.
- Serverless business logic manipulation: In serverless systems with multiple functions, the order in which functions are invoked may be important for achieving the desired logic. Attackers can exploit bad designs or inject malicious code during execution.
- Improper exception handling and verbose error messages: Developers tend to use verbose error messaging, which helps with debugging of environment variables, but then forget to clean code when moving it to a production environment.
- Legacy / Unused functions and cloud resources: Obsolete functions are targets for abuse and should be looked for and deleted every so often.
- Cross-execution data persistency: Serverless developers often reuse execution environments, and sensitive data may be left behind and exposed.
