Machine learning is a hot topic in the software industry right now. More and more companies are taking advantage of this artificial intelligence technique to train machines on their data and make predictions. Researchers from MIT’s Computer Science and Artificial Laboratory (CSAIL) want to take it a step further by revealing how a machine makes those insights.

If you are using a machine learning system in the medical field to help diagnose the illness of a patient, you want to make sure that the system is correct. Similarly, if you are a business that uses machine learning to make investment decisions, you want to make sure you can justify the cost.

(Related: Machine learning tackles the BIG in Big Data)

“One big thing with most artificial systems is, essentially, trust,” said Tommi Jaakkola, an MIT professor of electrical engineering and computer science. “If you are communicating with an AI system, you want to trust that the business decisions it is making [are] actually reasonable. One way to do that is to force the method to communicate the basis by which it made a particular decision. Any context where you have to take an action that has consequences, you would want to justify the predictions.”

To accomplish that, the researchers looked at a branch of machine learning: deep learning. Deep learning uses neural networks to learn from data and make intelligent decisions. Neural networks are a computational approach to mimicking the human brain.

To obtain the rationale behind the neural networks, the researchers divided them into in two modules: generator and encoder. The role of the generator is to extract, while the role of the encoder is to predict. The researchers then trained the two components to operate together.

“The encoder is essentially just a deep learning approach,” said Jaakkola. “It takes text and makes a prediction. You can run it in terms of the original text and learn the mapping, but it wouldn’t reveal anything about the justification for the prediction. Now we couple that with the generator that makes the selection. It selects the rationale and tests it through the encoder to see if it alone would suffice to make that prediction.”

The researchers tested this technique on beer reviews. According to Jaakkola, the researchers chose beer reviews because they provide a predictable aspect (the score people gave based on the beer). “The goal is not to do justification for beer reviews, but because they have the data already annotated, it served as a great test bed for seeing whether the method would work,” he said.

The experiment included a five-star rating system based on aroma, palate and appearance. The results showed 95% to 96% agreement with human annotation based on appearance and aroma, and 80% based on palate.

In addition, the researchers applied the technique to pathology reports to examine breast biopsies and diagnoses. According to Tao Lei, an MIT graduate student in electrical engineering and computer science, they were able to predict the results with 97% to 98% accuracy.

Jaakkola explained that this technique could already be broadly applied to many systems by reformatting them. “Everything happens totally unsupervised, meaning that we don’t need any annotation of what the rationales are. We are just decoupling the process of prediction in order to force the method to come up with the rationale,” he said.

Going forward, the researchers will look into how to make the technique more sophisticated. For instance, a broader direction of the research is looking into how to communicate with complex learning systems. “You might have an idea about the rationale that is suitable for this particular prediction. How would you communicate that with a system? It opens up the possibility of exploring bidirectional communication on human terms with complex machine learning predictors,” Jaakkola said.

Specifically, the researchers will look into expressing types of rationales from one context to another. “What is a reasonable rationale, say, for a medical context might not be the same as a reasonable rationale in investment decisions,” said Jaakkola. “There are structure and constraints in each domain. We can explore more sophisticated ways of articulating those rationales and incorporating them in the overall learning algorithm.”

More information is available in the paper “Rationalizing Neural Predictions” by Regina Barzilay, Jaakkola and Lei.