Enterprises need workload automation to connect all of the business processes of an organization together, but tool overlap has resulted in a complicated web that is difficult to break free from when trying to migrate.  

While workload automation is one of the older categories within automation, the space is now evolving towards more consolidation as a result of this complexity, according to Cem Dilmegani, founder and chief customer officer at AIMultiple, an AI industry analyst firm. 

He added that large companies have many different enterprise resource planning (ERP) setups on cloud or hybrid models. ITOps, then, has to rely on different vendor solutions and internally modified tools to keep the whole patchwork together. 

“The more tools that you have, the more complexity you have not only in terms of contracting but also in terms of training and getting the team to use the tool effectively,” Dilmegani said. “To be able to capture new crowds, vendors are adding new functionality and using new terminology in their marketing, but most of the time, because one of the primary benefits of this is getting to consolidate your ITOps automation at scale, we don’t see companies say ‘OK, for this specific batch of tasks I’m gonna use this and for this other batch, I’m gonna use this.’ They try to go after one vendor that offers them all sorts of capabilities.”

Previously, organizations used some sort of workflow orchestration or automation capabilities but haven’t used the tools to their full potential, he said.

“To have an enterprise, you pretty much need to be running these sorts of operations and in terms of the industry penetration of typical workload automation, I’m not sure I would expect penetration levels to be on the higher side,” Dilmegani added. “But what I also see is enterprises are transitioning from some complicated setup to a simpler setup and they are working with fewer vendors and fewer internally developed technologies and migrating to a place where they can pretty much offload much of the complex things to a new piece of software. So there is an opportunity for the simplification of versatile automation environments.”

Once workload automation is set up correctly, enterprises can benefit from a wide range of tasks from copying files from one location to another, to more complex tasks like provisioning and configuring new servers. The business tasks can then be viewed through a single application that can be managed by IT departments across physical, virtual, and cloud environments.

Slipping over to SOAPs

Because traditional job scheduling and automation tools failed to keep pace with the complexity of digital businesses, workload automation now has shifted over to Service Orchestration and Automation Platforms (SOAPs), according to BMC Software in a blog post. Gartner predicts that “through 2024, 80% of organizations using workload automation tools will switch to SOAPs to orchestrate cloud-based workloads” in the Market Guide for Service Orchestration and Automation Platforms.

SOAPs offer application workflow orchestration to create and manage workflows across multiple applications, event-driven automation to simplify IT processes, self-service automation, and many more capabilities. 

The field of workload automation can be rather complex to grasp as vendors are using new terminology to describe similar technologies. There are tens of categories of automation tools and their capabilities tend to overlap, Dilmegani said. 

Although it bears many names, IT-based workload automation technologies are different from things like robotic process automation (RPA). While both aim to automate work, RPA is typically used to automate tasks within a single application, while workload automation is used to automate tasks across multiple applications. It’s built to handle much more complex tasks and architectures that have spawned as a result of the move toward microservices and Kubernetes. 

Leveraging workload automation for ETL processes reduces time spent on repetitive data processes, and minimizes human intervention, reducing subsequent data errors. Also, automating data warehouse management through workload automation tools increases the transparency of compliance reports as all processes are recorded and have a detailed audit trail. Lastly, it reduces the number of FTEs hired to complete repetitive tasks, according to Dilmegani.

“The ultimate goal of the workload automation is to eventually have end-to-end control over processes that involve different types of IT or business tasks,” said Alexandra Thurel, the director of product management for automation and solutions at HCL. 

A critical piece of transformation

The workload automation tool is now considered a critical element of the infrastructure that is moving to the cloud or Kubernetes architecture during digital transformation, according to Thurel. 

“[Companies] look at evolving from on-premises complex application and rehosting or rewriting their application in the cloud, and they need to have the layer that helps orchestrate those new applications with the rest of the world because not all applications are going into the cloud at the same time,” Thurel said. “So they will need to ensure that some file transfer or database inventory that still runs on-premises is connected with the processes that are newly created in the cloud. They need to manage entry points and exit points between the processes.”

Companies that are looking towards digital transformation are either still investing in legacy systems that can be distributed, or they’re looking to readjust and re-architect with a lift-and-shift type of approach to the different applications to run on the cloud, or are looking to rebuild and reinvent their applications to become cloud-native. All three of these strategies have one thing in common.

They all have business processes that interconnect with different platforms and heterogeneous systems that bring together challenges and risks. These application workloads are no longer sitting in predefined data centers and are now spread across multiple clouds which is where workload automation becomes essential, according to Thurel. 

Organizations need to embrace a systematic approach, avoiding islands of automation where each context is being managed by a different tool. Organizations also need to manage their data flows as more data becomes available. Here, the file transfer capability that workload automation excels at is becoming more important. 

Some of the workload automation tools out today leverage historical workload execution data with AI to expose observable data and provide an enhanced operational experience. For example, HCL Workload Automation can optimize data transfers and processing by leveraging a single point of control and integration for MFT, RPA, and big data applications.

Expanding automation tasks

When done right, companies often find that workload automation doesn’t just stop being useful at one specific task.

They initially look at specific objectives such as improving a paycheck process or improving inventory management with workload automation, but then they soon realize that they can expand those processes within the ecosystem of applications that flow around that application, Thurel added. 

Their needs are changing too. Previously, companies wanted to have a control point that ran on-premises in their data center and today people want to have that control closer to their new application. Their workload automation now helps them gain observability into their cloud environment, or even against Kubernetes standards to allow for more flexibility, more scalability, and higher speeds.

Vendors now offer new orchestration flexibilities that enable users to define very precise modeling of their processes and users can define where they want to have the point of control.

Workload automation can map automation on specific control points that matter for the business. Then if a job fails, there is an action that can quickly fix that problem and continue to execute business processes. The tool can suggest to users where executions have a high risk of failing because it takes data from the millions of jobs that are executed every day. If there’s an anomaly, the intelligent system can inform the user how to act preventatively. 

Despite the benefits, moving to a new platform along with workload automation is not an easy task.

If a company makes mistakes during reporting, it could be getting taxed higher, or it could be reporting lower revenues along with all sorts of issues. And then there could be outages on the ITOps side which could break the business if the issues are not reported in a timely fashion. 

Workload automation is one of the most difficult things about migrating to different platforms and is therefore one of the biggest obstacles, according to Dilmegani. 

“The workflow automation domain is much more at the core of your business. And, it’s also a bit less known because it’s sort of done in a back office,” Dilmegani said. “But it’s stuff that shouldn’t break, and that brings some risk aversion with that, which is why you are ending up with a complex landscape. Today, there are plenty of opportunities for most companies to simplify.”