Apps are getting smarter, which is affecting what developers do and how they do it.  While programmers don’t have to be AI experts to include intelligent elements in their app, they should understand something about what they’re building into their app and why.  

For example, if you’re trying to improve a shopping experience or the stickiness of a content site, you’ll likely use a recommendation engine.  If you’re building a social app, an agriculture app, or a pet monitoring app, image recognition may make sense. If real-time context is important, such as for search or marketing purposes, location recognition may be worth considering.  And regardless of what you’re building, you’ll probably add a conversational interface sooner or later.

The use cases and the opportunities for including AI in mobile apps are practically limitless, but it’s wise to understand the limitations of what you’re using, including how it will affect app performance and user experience.

AI is not one thing
AI often is used interchangeably with other terms including machine learning, deep learning, and cognitive computing, which can be confusing for those who haven’t yet taken the time to understand their differences.  Others, such as technology analyst, advisor, and architect Janakiram MSV view the terms more narrowly. In a workshop at the recent Interop conference, he explained the various types of AI and their associations as follows:

  • AI = recommendation engines
  • Machine learning = pattern recognition
  • Cognitive computing = sensing
  • Deep learning = multi-level (“deep”) neural networks that model the human brain to enable more human-like decisions.  

For the purpose of this article, AI is used as an umbrella term.

While it’s not necessary for developers to become data scientists to take advantage of AI, they should familiarize themselves with the basics so they can use AI and remediate issues more effectively.

“AI that is integral to new mobile experiences, such as voice-based assistants and location-based services, increasingly require mobile developers to have a rudimentary understanding of AI to be effective,” said Vinodh Swaminathan, principal, Intelligent Automation, Cognitive & AI at professional services firm KPMG.  “AI platform providers are increasingly packing a lot of developer-friendly features and architectures [into their products] that take the burden of knowing AI off developers.”

What developers should know
Given the shortage of data science talent, it’s no surprise that there is a growing body of easier-to-use frameworks and platforms, as well as Alexa Skills, APIs and reusable models.  Simplicity does not alleviate the need for thought, however.

Rather than just grabbing a machine learning model and plugging it into an application, for example, developers should understand how the model applies to the particular application or use case.  According to Swaminathan, that may require getting a better sense of what data was used to train the model and what levers are available to further refine the pre-trained model to improve its effectiveness and performance.

Most developers haven’t been trained to think in terms of models and algorithms, however.  They’ve been taught how to code.

“Mobile applications have been about user experience and not so much about how you make the application more intelligent.  It’s only recently that chatbots and intelligent components have started to get exposure,” said Dmitri Tcherevik, CTO of cognitive-first business application development platform provider Progress Software. “If you want to do simple object or image recognition or speech processing, or if you want to build a simple chatbot, there are many tools from which to choose.”

Like anything else, though, what’s available off the shelf may not be exactly what your application requires.  Specialized use cases tend to require specialized models and algorithms to yield the desired results. While specialized models and algorithms may also be available off-the-shelf, they may also need some fine-tuning or modification to deliver the value the app is intended to provide.

“If you’re building an end-to-end application, you need to know how to collect data, how to store data, how to move data stored in the cloud, how to clean data, and extract the features to make that suitable for algorithm training and model training,” said Tcherevik.

Data scientists know how to do all of that, but most developers do not. Given that just about everything is data-driven these days, including AI, it’s wise for developers to learn something about working with data, how machines learn, commonly used statistical techniques and the associated ethical issues, all of which are typically included in an introductory data science course or book.

“Depending on the application, there may be liability issues if [the] machine learning is not properly trained and makes a wrong decision,” said Tom Coughlin, IEEE senior member and president of data storage consulting firm Coughlin Associates. “A developer should test the application under all edge conditions to try and find such issues before release, or create some type of fail-safe that can avoid dangerous situations, if the application will be used for mission-critical applications.”

An important thing to understand when working with AI is that things are not static.  For example, if a dataset changes, then the model using that dataset will need to be modified or retrained.

“Developers need to understand that the AI is only as good as its model and training.  Without constant feedback and input, an AI model can become something else entirely,” said Pam Wickerham, director of Solutions Development at contract management platform SpringCM.  “A trained model is never done and will always evolve and change. As an app gets smarter, it’s important to remember that it can evolve to [include] bias. It’s important to have a large sample set and review the feedback and training loop constantly to be sure the focus doesn’t become narrow or go in [an unintended] direction.”

Nir Bar-Lev, co-founder and CEO of deep learning computer vision platform Allegro.ai, thinks developers should understand how the fundamental nature of coding and AI differ.  Namely, that with standard code, answers are deterministic and with AI they are statistical.

“AI delivers a prediction answer on a given question with a corresponding statistical score,” said Bar-Lev. “Each score is also a product of the specific API, the specific question, the actual ‘noise’ in the environment and the version of the API.”

Why APIs alone aren’t enough
Adding intelligence to a mobile app isn’t necessarily as easy as calling an API because some refinement may be necessary to suit the app and use case.  Like anything else, new capabilities should not be added to an app simply because it seems fashionable to do so.

“The more your application requires specific domain knowledge, the less you can rely on AI APIs available today as solutions for your needs,” said Bar-Lev.  “AI is a learning paradigm. [The] conventional best practice rule of thumb is that the more data there is, the more accurate the results. However, it’s not only the quantity of data that’s important but also its specificity to the problem or use case the AI model is being asked to address.  This means that the more specific or esoteric your domain area is, the less performance and quality should be expected from existing AI APIs.”

He also warns that since AI APIs are mostly cloud-based, connectivity and communications overhead need to be considered.

“AI, and specifically deep learning, are computationally heavy.  As such, most of them are provided from the cloud, where there are abundant compute resources,” said Bar-Lev. “Obviously, developers need to consider cloud-based API calls for AI as they would any cloud-base actually making a local on-device call.”

There’s also the question of integrating AI into the SDLC to constantly monitor what’s working and what’s not.

“As with any new feature or new technology that supports the application, it’s important to pay attention to key lifecycle metrics.  Why have we added AI? What are we trying to improve? How do we assess the effectiveness of other techniques?” said Pavel Veller, CTO of the Digital Engagement Practice on AI Competency Center and Capabilities at software development service provider EPAM.  “It’s easier [to answer those questions] for business processes that have a clear funnel to optimize and evaluate, such as eCommerce, and more difficult for others that focus on engagement. The key, though, remains in identifying KPIs and putting continuous improvement in place.”

As mobile developers know, many factors can impact the effectiveness of an app.  AI adds another layer of complexity that needs to be understood, alone and in relation to the mobile app.  Getting things “right” is a continuous process of experimentation, measurement, monitoring, and improvement to ensure that the new capabilities not only work, but continue to work.

“Having dedicated AI/data science/machine learning resources focused on this loop is important to ensure you are successful,” said Antonio Alegria, head of Artificial Intelligence at enterprise software provider Outsystems.  ‘If you don’t have the resources for this, set up the automated measurement loop and try to make your processes very lean to retrain, test and deploy models. Then, manage your effort in a way that you can keep an eye on monitoring and improving this periodically.”

Developers also need to be aware of IP leaks because with every API call, they risk sharing their core IP with the API provider.

“Developers should carefully consider if and when to use third-party APIs and to check the terms of service and privacy policy very carefully” said Allegro.ai’s Bar-Lev.

Be mindful of resources
As always, mobile apps need to be developed with resource utilization in mind.  What one can do with GPU clusters in the cloud is not the same thing one can accomplish on a mobile phone. Therefore, developers need to consider how the models and algorithms they intend to use would affect resources including battery power and memory usage.

“Often, there’s not a clear path to getting something working quickly on mobile device with the prepackaged libraries such as Apple’s Core ML and Google TensorFlow Lite,” said independent developer Kevin Bjorke. “If I have all of the resources and time I need because the client will pay for it, it’s [a different scenario] than when it has to run inside the Samsung phone.  You want to do stuff and execute stuff that doesn’t use all the memory on the phone.”

According to Progress’ Tcherevik, resource utilization is just another parameter that developers should be monitoring.

“It goes back to the developer’s continuous deployment workflow,” said Tcherevik. “Establish a feedback loop, and have a process and culture of continuous evaluation and improvement.  Things don’t change solely because of technology. A change in consumer behavior or a market trend can make as negative an impact as wrongly implemented AI.”

Sooner or later something will break or discontinue working as intended, which is another reason why developers need to understand the basics of what it is they’re adding to an app.

“You need to have some knowledge of how the thing that you just put together is working or how it could be made to run faster, because if it has to run on mobile, it needs to be thermally responsible,” said Bjorke. “In production, you really care about how much it costs to generate just this piece of information.”

Know where the data came from
Typically, there’s no shortage of data; however, the existence of data does not mean that the data is available, clean, reliable or can be used as intended.

Data availability, even within a company, can be an issue if the data owner refuses to share the data.  In other contexts, the availability of the data may hinge on its intended use, the party using it, the purpose for which its being used and whether payment is required.

Also, data cleanliness is a huge issue.  For example, a typical company will have multiple customer records for a single individual, all of which differ slightly.  The problem is caused by the field discrepancies in various systems, manual entry errors, corrupt databases, etc. Meanwhile, companies are amassing huge volumes of data that are growing even faster with the addition of streaming IoT data.  In many cases, data is stored without regard for its cleanliness, which is one reason data scientists spend so much time cleaning data. While there are tools that help speed and simplify the process, data cleanliness tends to evade the minds of those who have not been taught to think critically about the data they intend to use.

If the data isn’t clean, it isn’t reliable.

Even if the data is available, clean and reliable, it may be that its use is restricted in certain ways or in certain contexts. Personally identifiable information (PII) is a good example of this.  It may be fine to use certain information in the aggregate as applied to a population but not an individual. Regulations, terms of service and corporate governance policies cover such information use.  However, just because certain types of information can’t be used doesn’t mean that the same information cannot be inferred using data points that have been deemed acceptable to use for other purposes. For example, income level can be inferred from zip code, financial transactions and even the words used in social media posts. Typically, several data points are triangulated to infer a missing piece of data, which can raise ethical issues if not liability issues.

Sadly, the degree to which companies understand their own data can be questionable, let alone third-party data.  Ergo, it’s bad practice to just grab data and use it without regard for its origin.

“There’s very little or zero discussion of where data came from, said Bjorke. “Developers are trained to program, so they understand less about data collection.”

Getting started
Today’s developers are feeling the pressure to embed some sort of intelligence into their mobile apps, whether it’s for competitive reasons or because their companies want to provide new types of value to customers.

“Instead of hand-rolling code, developers should try to leverage the major platform providers such as IBM Watson, Google, etc. [because] they provide easy on-ramps, good economics and great features,” said KPMG’s Swaminathan. “Investing in skills that can ‘explain’ AI will also be important as there will likely be some regulatory scrutiny.  Mobile app developers should be able to demonstrate that they not only understand how their AI works, but that they have ‘control’ over it.”

Meanwhile, development managers should accommodate AI development with specific steps in the development lifecycle such as training data curation, corpus management, improvement and refinement of the training models, he said.

There are a lot of resources available now that provide solid overviews of AI, the different types of AI, data science and working with data. While most working professionals are unable attend on-campus courses, there are many online offerings from vendors and highly reputable universities including MIT, many of which are available free of charge.

“Although better tools are lowering the barrier of entry, especially in mobile, developers should have knowledge about how the techniques used work at a high level, how to evaluate the results, how to tune and iterate, how to structure data, etc.” said Outsystems’ Alegria.  “Developers should also have the knowledge and experience in data manipulation [and] know how to best transform [the data] to leverage the AI algorithms. Additionally, developers should understand the end-to-end solution from data intake, transformation, algorithm setup, training and evaluation and then how it all maps to a production scenario.”

One misconception is that if machine learning is simply applied to data, magic happens. Also, some underestimate the amount of training data required for machine learning.

“Deploying AI models is still hard not from [the standpoint of] putting the model into production and using it in your app, but deploying all of the code and logic that ensure the data is transformed in the same way that the model saw during training time,” said Alegria.  “This can be a bigger challenge in mobile because the stacks used in training and in production could be significantly different, and you might need to handle more than one implementation, such as IoS and Android, which is why doing things in the cloud is still a better option in a lot of cases.”

Of course, the best way to learn is through hands-on experience and observation.  Like Agile development or technology pilots, it’s wise to start small and create something simple that tests the entire cycle, in this case, from data gathering to prediction in a production setting.  Alegria recommends developers focus on iteration by picking a single metric to improve.

“Ensure you know what metric represents [AI] effectiveness and that you have a way to measure it.  Also, ensure you are gathering the data on the AI element’s performance and that you set up your app to optimize getting at that data, such as through user feedback,” said Alegria. “Continuously measure the model performance, across important segments and set up alerts when the performance degrades.  Be sure to keep an eye on the failed cases by looking at the data manually to learn if there are some common patterns you’re missing.”

There’s more to AI than may be apparent to the uninitiated, in other words.  While the fundamentals are not difficult to grasp, and there are plenty of resources available for review, there’s also no substitute for hands-on experience.

Three machine learning tools to consider
Mobile developers ready to experiment with machine learning or deep learning should consider the following frameworks:

Apple Core ML is a machine learning foundation upon which domain-specific frameworks operate.  The domain-specific frameworks include Vision (image analysis), Foundation (natural language processing) and GamePlayKit (which evaluates decision trees). Developers can use Core ML to add pre-trained machine learning models to their apps.

Caffe2 is a lightweight, modular deep-learning open-source framework contributed by Facebook.  It allows users to experiment with deep learning and take advantage of the models and algorithms community members have contributed.  Caffe2 comes with Python & C++ APIs so developers can prototype immediately and optimize later.

Google Tensorflow is available as two solutions for deploying machine learning applications on mobile and embedded devices. TensorFlow Lite is a modified version of TensorFlow Mobile that allows developers to build apps that have a smaller binary size, fewer dependencies and better performance. However, TensorFlow Lite is available as a developer preview, which does not cover all the use cases TensorFlow Mobile covers.  TensorFlow Light also offers less functionality than TensorFlow Mobile. For production cases, developers should use TensorFlow Mobile.