Artificial intelligence (AI) that explains its decisions like a human? It’s not as far-fetched as you might think.
In a paper published by MIT’s Lincoln Laboratory Intelligence and Decision Technologies Group (“Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning“) presented this summer at the Conference on Computer Vision and Pattern Recognition (CVPR), researchers describe a system that performs “human-like” reasoning to answer questions about the world — namely, the contents of images.
The neural network — dubbed the Transparency by Design Network (TbD-net) — depicts its visual analysis with heat maps. As it deciphers a photo, it highlights items of interest with bright, easily identifiable colors (“attention masks,” in the researchers’ parlance) representing the output of modules, or small neural networks specialized to perform specific tasks.
“Like workers down an assembly line, each module builds off what the module before it has figured out to eventually produce the final, correct answer,” according to MIT News. “As a whole, TbD-net utilizes one AI technique that interprets human language questions and breaks those sentences into subtasks, followed by multiple computer vision AI techniques that interpret the imagery.”
So how might it answer a question like “What color is the large metal cube”? First, a module would isolate large objects. Then a second module would identify which of those objects were metal. A third would look for cubes that fit the previous criteria, and finally, a fourth would determine the color of the objects.
In a test involving 85,000 training images and 850,000 questions, TbD-net was 98.7 percent accurate, outperforming other neural module network-based approaches. More importantly, it achieved state-of-the-art performance — 99.1 percent accuracy — thanks in part to the heat map illustrations it produced, which enabled the researchers to tune it extremely precisely for a subsequent test.
Such insight into a neural network’s reasoning might go a long way to establish user trust, the authors of the paper posit.
In recent months, a number of corporations, government agencies, and independent researchers have made attempts at tackling the so-called “black box” problem in AI — the opaqueness of some AI systems — with varying degrees of success.
In 2017, the U.S. Defense Advanced Research Projects Agency launched DARPA XAI, a program that aims to produce “glass box” models that can be easily understood without sacrificing performance. In August, scientists from IBM proposed a “factsheet” for AI that would provide information about a model’s vulnerabilities, bias, susceptibility to adversarial attacks, and other characteristics. And Microsoft, IBM, Accenture, and Facebook have developed automated tools to detect and mitigate bias in AI algorithms.
“AI-powered systems hold enormous potential to transform the way we live and work … [but] these [transparency] issues must be addressed in order for AI services to be trusted,” Aleksandra Mojsilovic, head of AI foundations at IBM Research and codirector of the AI Science for Social Good program, told VentureBeat in an interview in August.