Changes in software development that have led to accelerated delivery cadences are stressing other parts of the application life cycle. This is especially true in organizations adopting microservices architecture, where teams are working autonomously to deliver their software, which by definition relies on communication with other services to form a more complete application.

And one area where this has caused major changes to occur is application monitoring.

The decentralization of authority over change means the ‘classic’ way of doing APM – creating a few experts who understand the complexity of an application, understand the underlying data and ultimately dig down to the root cause – no longer holds up.

“We speak about the democratization of the whole APM territory, which allows everyone – be it an occasional user, be it an expert user, or a developer looking at a current problem — to have access to the data immediately, to see what is running there, what is being collected, and to get to the root cause with in a few seconds,” said  Pavlo Baron, chief technology officer at Instana. “There is no central authority anymore that is doing the job of keeping the things together and understanding how they communicate and talk to each other,” and understand how many components can impact each other.

The idea of a dynamic application and changes is actually one of the pain points of monitoring microservices, because things change all the time, and it has to be monitored all the time, according to Baron.. “The only way to do that and not go insane, and to keep up with the changes in the application, is through automation. Every aspect of monitoring — what to monitor, what to look for, what makes it healthy — all those things have to be automated. What you risk if you don’t, or can’t automate it, if you use a tool that requires any kind of manual intervention, you risk being obsolete even before your changes are rolled out. We have customers who are updating their applications or services dozens of times a day.”

There’s an organizational shift within IT application delivery teams, focused on Agile methodologies and achieving CI/CD. In microservices deployment cycles, this shift is automating every other part of the deployment process — monitoring is the last piece.”

That’s why Baron and others in the industry believe in monitoring automation. It is time-consuming and not cost-effective to have people writing monitoring configurations every time a microservice within an application changes, he pointed out. “Whenever you deliver a change, you change something in the whole crowd of your services, you’re effectively monitoring configurations and the rules you use to identify problems, and those must be adjusted. You need to keep people continuously busy, instead of delivering value, just configuring and reconfiguring your monitoring. Our approach is to automate this as much as possible and take that  problem from the teams so they can focus on what really matters.”

The basis for this new APM is intelligence, and building an intelligent system begins with the ability to recognize what is running in the system that is being monitored. From there, the monitoring system needs to know how to attach to a component and consume the data from within — automatically — without any human intervention in terms of configuration.

Baron explained that a big part of making that work is data precision, which is why Instana built one-second resolution for all metrics and metadata, so Instana’s automation engine (the robot) can always be up to date. Whenever a change occurs, the robot will be able to capture the change and re-adjust the underlying graph to the new connections.

In Baron’s mind, automated monitoring should be a commodity. Only a few people in the world have the job of monitoring systems, he said, while most people have the job of keeping systems up and running. With the IT world moving toward combining those roles, Instana is trying to deliver maximum automation, intelligence and visibility to its users.

“We plan to evolve the whole APM territory into something that perfectly supports the modern way teams work,” he said. “While the classic APM tool is functional, from a process perspective, it’s not a good fit.”

The Six Pillars

The only way to effectively manage microservice applications is to automate the entire monitoring life cycle by applying AI to discovery, monitoring and analysis. Instana’s AI strategy is built on six fundamental pillars, the first three of which are focused on automation. Those pillars are:

  • Automatic and Continuous Discovery & Mapping
  • Precise High Fidelity Visibility
  • Cloud, Container & Microservice Native Deployment
  • Full Stack Application Data Model
  • Real-time AI-Driven Incident Monitoring & Prediction
  • AI-Powered Problem Resolution and Troubleshooting Assistance