The latest contest facing modern business is the quest to become an “algorithmic enterprise,” the next step in the evolution of analytics. Rather than just finding insights within data lakes, an algorithmic enterprise uses insights to automate action. The shift is fundamental — we’re moving from an era of human-centric business to automation-centric business.

The IT infrastructure of an algorithmic business is like the human nervous system, sensing and responding automatically to input based on experience, learning, and training. Algorithmic decision-making is fueled by analytics, and the white-hot nexus of artificial intelligence, machine learning, and Big Data. A digital business is an algorithmic business.

But most companies are not ready to compete in this contest. Like everything in life, fitness requires preparation and hard work. Consider the industry that has made the biggest shift to algorithmic excellence thus far — financial services. Ten years ago, less than 5% of trading was algorithmic; today, it’s over 80%. But it took a decade to get there.

Over that same decade, I was involved in running another type of race. I’ve completed 10 marathons. At first, I ran 3:37 (or 8:17 per mile). That’s respectable — 25% of marathoners can break 4-hours. Eventually, I ran 2:51 (6:30 per mile). Only about 2% of marathoners run that fast. But again, it took 10 years to reach personal best.

When it comes to being data-driven, companies should leverage the algorithmic lessons of Wall Street and look at the process like training for a marathon, beginning with basic conditioning, then completing your first race, and finally achieving peak performance.

Basic Conditioning
The first step to becoming algorithmic is conditioning. In technology terms, this means the creation of an interconnected data pipeline and data lake strategy. Time and again we see businesses that want to embark on analytics, but have only partial access to data and/or data that’s riddled with errors and inconsistency; or they want to become real-time, but don’t have access to basic streaming data. It won’t work.

High-speed messaging and integration technologies were literally born on Wall Street. Twenty years ago, human traders would screen and signal as an intrinsic part of the trading workflow. Now such stock exchanges are museums.

Even though most of the processes that govern business are human-driven, even on main street, the shift to become purely digital is under way. IT systems are increasingly interconnected, you must have a mobile strategy, and you must now also anticipate IoT and sensor-driven business. Modern integration technologies, microservices, hosted PaaS, and Hadoop data architectures create the basic enterprise muscle needed to run in this race.

If the goal is to perfect an enterprise nervous system, then an interconnected data pipeline forms its structure. A healthy system has a strong data fabric wrapped with application programming interfaces that automate communication, with a microservices architecture breaking business logic down into small, discrete, typically cloud-native services automatically triggered by events, deployed and decommissioned independently, and scaled to meet changing demands. With a dynamic, automated pipeline for data, you’re properly equipped.

Run a Good Race
With an enterprise data pipeline in place, you’re ready for your first race. Technologically, this means applying data analytics, and most industries are still amateurs.

On Wall Street, leadership is stacked with executives from business and math backgrounds who are algorithmically inclined. Many financial services CIOs today were algorithmic innovators yesterday. Risk management is algorithmic. Compliance is algorithmic. Every bank is an algorithmic enterprise.

Other industries are building good data pipelines and sound analytics initiatives. Some have data science centers of excellence. Some process more than static data. But in running terms, this amounts to a leisurely five-mile jog.

From an analytics perspective, advancing to the next level means stretching into sensor technology and non-traditional connected devices, developing machine learning and deep learning expertise. It means deploying the insights you find in data lakes directly into your real-time stream of data. Rather than having a small center of analytics excellence, you have analytics pervasiveness. Masters have a culture that reaches for automation first, not human process improvement.

Breaking the Speed Barrier
If you establish a data-centric culture, you can tackle the final element: Speed. This is where you accelerate your regimen, change your diet, take on hill work. My favorite workout used to be running 15 miles at training pace, then pushing through another 6 miles at better-than-race pace. It was the hard 6 miles at the end that provided the most benefit. That’s what separates the top 25% from the top 2%.

In analytics, speed barriers are broken by depth and breadth of algorithmic expertise. I recently met with one Wall Street firm that had just added another petabyte of data to its data lake, dropped its algorithm evaluation time from a week to mere seconds, and had discovered a dozen new algorithms in the past year. This level of sophistication is rarely found in other industries, but the tools and techniques to achieve it do exist.

Further, streaming algorithms mean we can no longer think only in terms of human speed. The algorithmic enterprise works at machine speed. The CIO of an innovative insurance company told me they use streaming analytics to respond to pricing quote requests in under five milliseconds via enhanced pricing and fraud-aware algorithms. Just last year, such work took two-to-three days to process.

Human decision making is still essential to an algorithmic enterprise. But the work is less manual and more scientific — human staff create, monitor, and optimize algorithms rather than performing math themselves.

It takes many years of dedicated focus, but achieving algorithmic excellence is an incredibly powerful position. And unlike my marathon running, it’s not just a hobby — all enterprises are in this race. Dealing with digitally connected customers, assets, and supply chains; sensing and responding to changing digital conditions; mastering real time; automating to compete on cost and disrupt traditional revenue-generating modes of operation. But you must put in the work to become a true algorithmic enterprise. See you at the starting line.