How APM tools help businesses understand the customer’s journey

Monitoring applications and user experiences is like making coffee. A person can heat up the water, measure the exact amount of coffee grinds, and put the right amount of milk or sugar in it, brewing what he or she thinks is the perfect cup of coffee. The assumption is, the person who drinks this cup of coffee will enjoy it, but unless you ask them whether or not they are having a good experience, how can you possibly know for sure?

The same process applies to user experiences with applications, according to Nik Koutsoukos, an application performance management expert at Riverbed Technology, who said his company likes to use the coffee analogy when explaining the importance of APM tooling.

With APM tooling, companies can monitor everything from network capabilities to performance consumption, but unless they dig deep into the metrics that show what the end user is seeing, it’s just not possible to know if the customer is truly having the best user experience possible.

Today, companies cannot just look at parts of the chain to assess the user experience. Measuring the end user experience needs to involve monitoring of the entire environment, and if the company is delivering an application or software service, it needs an end-to-end view of how the application is performing.

With the right APM tool or solution, companies can slice and dice data based to meet the needs of the business, and the needs of the business’ customers. Without these insights into the application — from code creation to user experience and closing the feedback loop — companies will not be able to keep up with the market’s pace, nor will they be able to go to that next level of agility that DevOps promises.

Fitting APM into the DevOps lifecycle
Application performance management is a key part of the DevOps life cycle, and more and more companies are adopting this methodology to increase the pace of releases while still delivering quality applications with good user experiences. Companies that do not track their application performance throughout the life cycle are going to end up at the finish line with an app that is not performing well, plus it won’t meet the expectations of the user and the business, said Dawn Parzych, an expert in application performance monitoring at Catchpoint Systems.

This means companies should be tracking performance throughout the entire cycle, from the planning stages all the way through coding and testing. Parzych mentioned that testing should be done continuously while the app is being developed because it’s not a one-and-done scenario. Teams want to avoid running into performance issues after these tests are completed, and the earlier these tests are run, the better, said Parzych.

Amy Feldman, an expert in enterprise software and information technology at CA Technologies, said that they see their customers move these APM tools left in the production of the application itself, so developers can use these tools to be able to not only understand the quality of the application, but be able to fix the issues in pre-production prior to the issues getting rolled out to the customers.

This is where APM can provide developers information on their builds. Feldman said APM tools can show developers which build is affecting performance, which build is good, which build isn’t as stable, and which builds might need to be rolled-back for the sake of performance issues.

“By being able to use these tools in the entire life cycle, it gives you that full visibility of not just performance, but being more proactive in how my code changes and what I’m developing truly impacts that overall customer experience,” said Feldman.

Besides having that visibility, APM tools provide empirical data and give companies real-time feedback into how well the applications are performing.

“When you build an application, there are a lot of things you are striving for. One is for it to do whatever it is supposed to do as it’s defined, but how well does it do it, how does it respond, how much load can the app withstand until performance starts to diminish,” said Thomas Hooker, an application performance expert and vice president at CollabNet. “Those types of things are difficult to completely recreate in your development environment,  but as the application makes its way into production, you can get that feedback and [make improvements].”

But the real beauty of an APM tool is that it can integrate into IT and operational management tools, as well as DevOps tools, said HPE director of product management, Balaji Venkatraman. The value of these tools is that if there is a failure, or performance degradation, the APM tool can pinpoint this and then bring that application context and key metrics and experience by the user, back into development to the line of code, he said.

Then it can be fixed, tested and verified, with the code elevated and pushed back into production, said Venkatraman. APM tools are critical to DevOps teams for this reason; by understanding how the user is using the application and what the user workflows are, teams can bring all this information back from the operational context into the development and testing context, and continue to release at an agile speed, according to Venkatraman.

APM ‘core IT discipline’
Software is the primary driver of business revenue, regardless of the industry, said Ashish Kuthiala, a DevOps and enterprise agile expert at HPE. It’s important that companies monitor and manage performance and availability for the software applications they develop and put out, because the idea here is to detect, diagnose and proactively monitor the application to make sure there are no ongoing issues, he added.

If a business has an application that crashes, is buggy, or experiences other performance issues, the organization needs to be able to quickly address the problem and fix it. If not, they are going to lose revenue, said Kuthiala.

Venkatraman said this is why he considers APM to be a core IT discipline, since ensuring that applications are responsive is absolutely critical to the business. With the right APM tool, companies can not only look at the app availability and performance, and monitor the application availability, but they can use these tools to see the utilization of resources. All of these insights can help a company achieve the ultimate goal, which is ensuring that an enterprise can provide the end-user experience its customers want.

Organizations can also take an empathetic approach to monitoring applications, where they put themselves in the user’s shoes to see how they feel about the application, how they are using it, and if they are engaging with the application as it is intended. An APM tool can capture this entire experience, said Kuthiala.

Increasingly, companies that deliver applications and services to external and internal consumers realize and accept that delivering a solid user experience is good for the business, said Riverbed’s Koutsoukos.

Ensuring that your customers have a great user experience in the way that they interact with an application is already important, but going forward, companies need to really consider and establish a set of technologies that allow them to measure and understand these experiences.

“You need to understand end-to-end how the application is performing,” said Koutsoukos. “From the device that the end user is using to consume that information, all the way to the application that is sitting somewhere in the cloud, and everything in between.”

And while measuring the end-to-end experience is important, businesses need to also determine what metrics need to be measured to gauge success.

Measuring different metrics that do not add value or do not tie back to business goals is just meaningless data, said Catchpoint’s Parzych, so teams should make sure the metrics that they are collecting and the data they are looking at makes sense in terms of this overall goal.

“There are thousands and thousands of metrics you can look at, and if you are looking at the wrong metric you may end up making the wrong business decision,” said Parzych.

Metrics that matter
Simply put, a company cannot make informed decisions to deliver good user experiences if it does not have the right information available. Keith Fenech, vice president of software analytics at Revulytics, said that he sees APM and software usage intelligence as complementary areas.

Fenech said performance of the application might change the behavior of the user. For instance, if an application has a feature that takes a long time to run, the user might spend more time in the configuring of that process before it actually launches. The behavior in this case is directly affected by how fast the product or feature is performing. By taking this data, a company can figure out why this particular feature took so long to run and why the user’s behavior was directly impacted.

“The product itself must be sleek and smooth-running, so that’s the area that can be monitored with APM tools,” said Fenech. “However, user experience must be fluent whenever a user is doing something or looking for something, and that is the part that usage intelligence would give you, so applying [APM and usage analytics] together would provide you complimentary intelligence and give you a holistic look at what’s happening on the client side.”

An APM tool can also help software teams tackle unique challenges. For instance, it can show them what is happening and what users are seeing when specific events are going on like during Black Friday sales, according to Rainforest QA’s vice president of product Zack Smocha.

Customers who are deploying their applications to the market at a rapid pace can test their applications in both staging and production, but it’s the staging piece where the APM tool comes in, said Smocha.

Teams can run diagnostics and collect data on what happened in the application during these stages, said Smocha, and essentially, today’s APM tools and QA/testing tools can be utilized to help teams see the big picture and adjust tests and strategies to continue to meet the end-user’s needs.

Every APM tool is different of course, but there are a few key metrics that all APM tools should offer. According to CollabNet’s Hooker, an APM tool should be easy to use. The information provided needs to be simple enough to understand, and the “wealth of detailed data” should benefit the organization and it should be able to be understood in non-technical areas of the business. This way, teams can consume the data and make sense of it, he said.

Delivering a good UX
Today’s customers hold all the power when it comes to using applications and software services. If a customer does not receive the expected experience with the application, it’s all too easy for them to click delete and download a new app from a competitor.

The challenges of today speak to the fact that user experiences are so important, and the customer is so empowered, said Hooker. With this in mind, software providers have much to consider if they want to deliver the best user experience and retain customers. Teams can tackle these challenges by figuring out which parts of the software are being used and in what manner. Knowing this can help a company identify the different user groups, use cases, and companies can identify what users are utilizing a particular features or software parts, said Fenech.

Another challenge is that of information, said Fenech; a company can’t begin to make informed decisions unless they actually have that information available.

CA’s Feldman said that all of the data from managing microservices, managing hybrid apps and distributed systems adds up to a lot of complex information that needs to be sifted through. She added that many APM tools have a hard time working through that complexity to present information in a way that even the novice user can understand. On the other side of that coin are tools that are overly descriptive, giving companies too much information that needs to be understood.

“The approach we take with our tool is we want even the first-level operators to be able to understand that information, by being able to represent the data that makes sense to everyone,” said Feldman.

APM and testing/QA tools help teams keep the pace of DevOps
The benefits of using an application monitoring and management tool is that it allows companies to not only monitor what is happening in their applications and environments, but also to see what is happening across the entire software development life cycle. This gives teams the scalability, the visibility and the ability to really see the big picture, and to make adjustments accordingly.

Some experts say that an APM tool isn’t just for monitoring applications, and it can be beneficial to both testers and QA engineers. Companies can look at their applications in real-time while they are still testing, which lets teams monitor the different environments so they can know what is happening to specific users and events, according to Zack Smocha, vice president of product at Rainforest QA.

Smocha said when they look at the market as a whole, they see developers as the ones with the “latest and greatest” tools at their disposal. When they talk to customers about their testing, they see that both QA and testing applications is still lagging, he said.

One of Rainforest’s newest tools called Exploratory Testing tackles this issue and tries to take on time-consuming activities of QA and testing. The new solution aims to expand testing coverage and discover issues faster, so more bugs can be found and less pressure is put on development teams. With the right testing tools in place and the right APM tools in place, teams can continue their release delivery at a rapid pace, while still catering to their user’s needs.

What should companies consider before selecting an application monitoring/management tool?

Balaji Venkatraman, director of product management, Hewlett Packard Enterprise (HPE)
A robust APM solution should provide visibility across legacy and modern applications, support flexible deployment models (on premise, SaaS, hybrid) and depending on the organizational and operational maturity, accelerate the transformation to a digital enterprise.

APM tools need to provide visibility into the application infrastructure, track application availability and performance using synthetic transactions and real user traffic, trace transaction

workflow for rapid root cause analysis and fault isolation, and measure end-user experience using flow analysis or end-point instrumentation. APM tools must integrate with IT Operations tools for service modeling, unified event correlation and ticketing solutions; furthermore, given the transformation to a digital enterprise and adoption of agile methodologies, APM tools must also integrate with application life cycle management, functional and performance testing to leverage the full power of DevOps. Differentiated capabilities include Big Data analytics and the ability to correlating user experience with sentiment analysis for actionable insights.

I recommend customers consider their entire portfolio of internal and customer-facing applications, critically assess where they are in their journey of digital transformation, their adoption of DevOps and agile methodologies, and existing tool sets, while selecting an APM solution that will help deliver the best user experience, improve customer loyalty and revenue.

Dawn Parzych, director of product and solution marketing, Catchpoint Systems
Defining the business goals and objectives should be the first step when selecting any tool. These generally fall into three main buckets: increase revenue, reduce risk, or reduce cost. The tool selected should help gather the most comprehensive and accurate data and measure it to determine success or failure of the defined objectives.

For example, if the company is launching an API to generate new revenue streams, selecting a tool that can measure performance and availability of the APIs would be a good feature to have and would help reduce risk. If the company is expanding into new geographical areas to increase revenue, having the capabilities to measure performance and availability from specific vantage points within those regions would be needed. Selecting a tool that cannot provide the data or metrics necessary to meet corporate or team objectives will result in yet another tool sitting on a shelf collecting dust.  

Amy Feldman, director of product marketing, CA Technologies
When selecting an application monitoring tool, many companies look for solutions that can be used by both their application support teams as well as developers earlier in the app life cycle. A single tool helps to foster collaboration across DevOps, improves application quality and provides insights into the customer experience.  In addition, many companies are looking for tools that will be able to monitor not only their existing applications but those that are flexible and open to monitor and support their digital transformation initiatives. With these new digital initiatives comes the need to manage microservices architectures that are deployed to cloud environments or containers. Monitoring these new modern apps requires a monitoring tool that easily depicts the dynamic application environment with their dependent components and allows the user to easily pivot on certain aspects such as application, owner, location or business function. I’ve also seen new application or digital owners who are looking for greater insights into the customer experience and digital journey. This means they need to be able to uniquely combine user behavior (usage, retention, drop off rates, etc.) with performance data (crashes, transaction, CPU, Memory, etc.) and monitor the customer’s journey across web and mobile devices.

Nik Koutsoukos, vice president of product marketing, Riverbed
A number of factors should be considered. Applications are complex and there are many causes of poor application performance and end user experiences. So, solutions need to have full visibility of everything that can contribute to performance degradation.

This includes the end user devices, the network, the infrastructure and of course, the application. Secondly, having the ability to capture, blend all that data and present it in an integrated way is critical for fast troubleshooting and proactive performance optimization. This integration also contributes vastly towards improved IT productivity.

Thomas Hooker, vice president of marketing, CollabNet
I believe there are a few key considerations for an organization to evaluate and become comfortable with prior to selecting an application monitoring/management tool. The first is a bit of a combination of two; that being what are your use cases and what are the personas of your users.  

You should spend time up-front to understand, be clear about them and document both of these items. I would also recommend time be given to “future proof” your investment to the extent possible. By that I mean don’t just consider your needs today, but consider to the extent possible future use cases and user persona. I would also recommend that you establish upfront success criteria, a budget and a project timetable.

Keith Fenech, vice president of software analytics, Revulytics

• Effort required: One of the primary concerns is the amount of time it will take your team to implement the solution and start collecting data. Once integration is completed what resources are required to maintain the system, both in terms of data collection and reporting?
• Accessibility: How easy is it for users with different skills within your company to get access and visualize the data collected in real-time? Do you have a fast, flexible visualization dashboard or do you need a technical person to run SQL queries and build reports manually?
• Actionable Reporting: Does the tool offer you actionable intelligence that directly leads to impactful business decisions or are you limited to vanity metrics? The solution should help identify trends and key points of interest within your data that might be otherwise go unnoticed to the naked eye.
• Flexibility: Is the tool simple enough to provide a high-level view, but also powerful enough to answer more detailed questions? The tool should offer out-of-the-box dashboards and the capability to drill down into specific areas to answer more complex questions.
• Scalability & Adapting Needs: Businesses grow and requirements change over time. Can the tool scale and adapt to your changing needs?

Derek Choy, vice president of engineering at Rainforest QA
When selecting an application monitoring or management tool, companies should consider the goals of standing up such a tool and the process that ensures it will be effective. There are many tools available, and they differ greatly in functionality. Some focus on dashboard, diagnostics and analysis, while others focus on integration and making alerts actionable for fast and automated recovery. As applications becomes more complex, and customer expectations increase, faster and automated recovery becomes a greater priority. Tools that can integrate with CI/CD allow fixes to be deployed more rapidly. When developers get alerted to a problem, and subsequently check in a fix, the build can be rapidly tested by an agile and automated testing tool like Rainforest QA. Once the test passes, the fix can deployed to production immediately, minimizing service impact and the engineering time required to make the fix.

An automated, agile testing tool is an integral part of this process. Sending the fix to a traditional QA team for regression testing can sometimes take hours, if not days, missing the window to deploy the fix quickly for minimal impact. A testing tool that allows regression tests to be kicked off automatically by integrating into CI/CD and application monitoring tools allows for testing to be done in minutes, and enables the fix to be deployed quickly without compromising quality.

A guide to application performance monitoring tools
AppDynamics: The App iQ Platform is a unified suite of application and business performance monitoring solutions that ensure every part of even the most complex, modern, multi-cloud environments — from software to infrastructure to business outcomes — is highly visible, optimized, and primed to drive growth. With AppDynamics, you can identify and resolve application issues before they impact users, empowering you to focus on creating better digital experiences with more velocity.

AppNeta: AppNeta provides APM for DevOps and APM for IT. APM for DevOps allows users to monitor and optimize their applications. Features include the ability to identify trends and outliers at a glance, resolve issues faster, track user performance, and measure performance, functionality and availability from the user’s perspective. APM for IT allows users to find and resolve issues. Features include end-to-end visibility and SLA tracking on any network, end-user experience monitoring with synthetics, intelligent application identification with real user data, and flexible and scalable deployment.

BMC Software: TrueSight App Visibility is an APM suite that monitors and manages packaged, web-based and modern application ecosystems, from the infrastructure and code to the customer experience, to correlate the behavior of diverse application architectures with the customer experience. This level of visibility into on-premises, private cloud, and public cloud infrastructures in the context of customer experience makes it easy to identify, diagnose and prioritize issues while proactively tuning performance in alignment with SLA’s and digital business goals.

CA Technologies: CA APM is built to be easy, proactive, intelligent and collaborative, or EPIC. It offers easy deployment of APM agents, including Node.js and PHP. Proactive identification and resolution of issues occur across physical, virtual, cloud, containerized and mobile applications. Intelligent insight comes through 360-degree mobile-to-mainframe visibility, which captures billions of critical metrics per day to verify transactions. CA APM is collaborative, aligned with DevOps methodologies to instill continuous performance improvements at every stage of the software life cycle.

Catchpoint: Catchpoint is a leading digital experience intelligence company that provides unparalleled insight into customer-critical services to help businesses consistently deliver amazing digital experiences. Catchpoint’s digital experience monitoring platform provides integrated synthetic and real user monitoring, comprehensive test types, real-time analytics, and a diverse node network to help you preempt performance issues and optimize service delivery. More than 350 customers in over 30 countries trust Catchpoint to strengthen their brands and grow their businesses.

CollabNet: CollabNet helps enterprises and government organizations develop and deliver high-quality software at speed. CollabNet is a winner of the 2016 Best of Interop Award, a Best in Show winner in the application lifecycle management and development tools category of the SD Times 100 for 13 consecutive years, and a gold winner of the 2016 Golden Bridge Awards. CollabNet offers innovative solutions, consulting, and Agile training services. The company proudly supports more than 10,000 customers with 6 million users in 100 countries.

Crittercism: Crittercism’s mobile application performance-management solution enables enterprises to accelerate their mobile business. Crittercism’s solution monitors every aspect of mobile app performance, and provides a real-time global view of crash, service and mobile transaction metrics across iOS, Android, Windows Phone 8, HTML5 and hybrid apps.

Datadog: Datadog is the world’s leading monitoring service for cloud-scale applications, bringing together data from servers, databases, tools, and services to present a unified view of your entire stack. These capabilities are provided on a SaaS-based data analytics platform that enables Dev and Ops teams to work collaboratively to avoid downtime, resolve performance problems, and ensure that development and deployment cycles finish on time.

Dell: Foglight APM,  a smart APM SaaS-based solution, requires zero configuration. Or, monitor your applications from a single pane of glass with an on-premises solution that can be optionally combined with other Foglight performance-monitoring capabilities for database, virtualization or storage.

Dynatrace: Dynatrace delivers developers and testers the answers they need to ensure superior user experiences, proactively solve problems, and accelerate DevOps/Continuous Delivery strategies. Dynatrace User Experience Management and Dynatrace Synthetic Monitoring help developers to proactively understand and optimize user experience; Dynatrace Application Monitoring is designed to maximize application performance; and Dynatrace Data Center RUM delivers app-aware network insights.

HPE: Hewlett Packard Enterprise provides industry leading solutions that help you continually deliver amazing user experiences across technologies with quality, speed, and confidence helping accelerate the planning, testing, delivery, and support of optimized applications. HPE’s AppPulse Suite of user experience management solutions helps you to automate and accelerate the delivery of user-focused apps by isolating user experience issues from user action down to the line of code.

Neotys: NeoSense is a proactive application performance monitoring solution for all Web and mobile applications. It actively monitors the performance and availability of critical business transactions within recorded user paths. NeoSense leverages the test scenario design capabilities of NeoLoad to quickly create realistic monitoring profiles. Easy-to-read dashboards, alert triggers, and notification rules provide actionable insights for pinpointing performance issues and getting to root causes quickly.

New Relic: New Relic’s comprehensive SaaS-based New Relic Software Analytics Cloud provides a single powerful platform to get answers about application performance, customer experience, and business success for Web, mobile and backend applications. New Relic Insights is embedded in the platform, enabling customers to do detailed, ad hoc queries for real-time analytics across New Relic’s APM, Mobile, Browser and Synthetics products.

Oracle: Oracle provides a complete end-to-end application performance management solution for custom and Oracle applications. Oracle Enterprise Manager is designed for both cloud and on-premises deployments; it isolates and diagnoses problems fast, and reduces downtime, providing end-to-end visibility through real user monitoring; log monitoring; synthetic transaction monitoring; business transaction management and business metrics.

Rainforest QA: Rainforest QA helps agile and continuous delivery engineering teams move faster with the industry’s only AI-powered crowdtesting platform. Its platform leverages 50,000 qualified testers to deliver on-demand, comprehensive and machine learning verified regression test results. Rainforest customers spend less time and money testing so they can ship better applications faster.

Revulytics: Revulytics offers cloud-based software usage analytics and turnkey services that give software producers deep visibility into how their products are being used and misused, providing them with actionable intelligence to generate revenue by converting unlicensed users into paying customers, optimize product development, and make data-driven decisions across their business. Its Compliance Analytics solution and services are used by leading software vendors to increase license revenue and globally reduce software piracy. Its software usage analytics solution provides valuable insight into product adoption, engagement, and environments, enabling product managers and developers to make data-driven decisions.

Riverbed: Riverbed enables organizations to modernize their networks and applications with industry-leading SD-WAN, application acceleration, and visibility solutions. Riverbed’s platform allows enterprises to transform application and cloud performance into a competitive advantage by maximizing employee productivity and leveraging IT to create new forms of operational agility.

SmartBear: AlertSite’s  global network of more than 340 monitoring nodes helps monitor availability and performance of applications and APIs, and find issues before they hit end consumers. The Web transaction recorder DejaClick helps record complex user transactions and turn them into monitors, without requiring any coding.

SOASTA: The SOASTA platform enables digital business owners to gain continuous performance insights into their real-user experience on mobile and Web devices—in real time and at scale.

Stackify: Stackify is a leader in innovative cloud-based application monitoring, performance and troubleshooting for developers and operations. Stackify’s Retrace provides powerful application performance management for developer teams and applications, and it allows developers to quickly find bugs and improve application performance on QA/test and production servers.