Artificial intelligence (AI) is a broad field that spans academic research with ambitions to create an artificial human brain (general AI) through to practical applications of deep learning (DL), a branch of machine learning (ML, itself the part of AI concerned with learning systems built on data rather than prepared rules).

DL has many real-world applications but before we delve into that world, let me level-set by saying I do not believe we have reached narrow AI, and are an exceedingly long way from general AI. There is no standard definition of narrow AI but at a minimum it is an AI system that can learn from a few examples (just like humans can) and not from iterating hundreds of thousands or millions of times data through the model.

To finish my level-setting, this pre-narrow AI era of ours I label machine intelligence, and I continue to refer to the whole space as AI.

DL has successes and disappointments, the latter mostly driven by hype but there is also a better understanding of DL’s limitations. For example, there is some gloom right now around the prospect for autonomous driving vehicles (AV), despite existence in limited domains of robot taxis and buses. On the success side DL is used in many practical scenarios today such as online recommender systems, wake word technology, voice recognition, security systems, production line fault detection, and image recognition from assisting radiologists in diagnostic imaging to remote medical services, as well as a host of prospective technologies for smart cities that will flow with 5G rollout.

In a recent study on AI chips for Kisaco Research, where we closely examined 16 chip providers, we also mapped the AI chip landscape and found 80 startups globally with over $10.5 billion of investor funding going into the space, as well as some 34 established players.

Among them are the ‘heavy’ hardware players such as the racks of Nvidia GPUs, Xilinx FPGAs, and Google TPUs available on the cloud, as well as where high performance computing (HPC) overlaps with AI. Training DL systems tends to be done here, but not exclusively; there are use cases of training at the edge.

Then there are systems where AI inferencing is the main activity, and there are many AI chips exclusively designed for inferencing. In practice this means these chips run in integer precision, which has been shown to provide good enough accuracy for the application but with a reduction in latency and power consumption, critical for small edge devices and AV.

The need for AI hardware accelerators has grown with the adoption of DL applications in real-time systems where there is need to accelerate DL computation to achieve low latency (less than 20ms) and ultra-low latency (1-10ms). DL applications in the small edge especially must meet a number of constraints: low latency and low power consumption, within the cost constraint of the small device. From a commercial viewpoint, the small edge is about selling millions of products and the cost of the AI chip component may be as low as $1, whereas a high-end GPU AI accelerator ‘box’ for the data center may have a price tag of $200k.

The opportunity for software developers
The availability of AI chips to support ML and DL applications has opened up an opportunity for software developers. Whereas some decades ago the programming of GPUs and FPGAs was the preserve of embedded engineers, the growth of DL has led to the chip designers realizing they need a full software stack to enable traditional software developers, data scientists and ML engineers to build applications with the programming languages they are familiar with, such as Python.

Many of the AI chips on the market now support popular ML frameworks/libraries such as Keras, PyTorch, and TensorFlow. It is possible for developers to test ideas out on low-cost AI chips, to name some of the familiar brands and their offerings: Intel Neural Compute Stick 2, Google Coral board, Nvidia Jetson Nano and Nvidia CUDA GPU cards.

We are inviting AI chip users, supply chain players and AI-related enterprise decision-makers to join our Kisaco Analysis Network; please contact the author