Opanować sztuczną inteligencję

© Fraunhofer IPA

© Fraunhofer IPA

Artificial intelligence most often has a black box character. But only transparency can build trust. To explain the respective solution process, there is special software. A study by the Fraunhofer IPA has now compared and evaluated different methods that make machine learning procedures explainable.

Artificial intelligence, which just a few decades ago was science fiction, has now become part of everyday life. In manufacturing, it detects anomalies in the production process; in banking, it decides on loans; and on Netflix, it finds the right film for each customer. Behind this are highly complex algorithms operating in the background. The more demanding the problem, the more complex the AI model—and thus also less transparent.

However, users especially want to understand how a decision is made in critical applications: Why was a workpiece sorted out as defective? What causes the wear of my machine? Only in this way can improvements be made, which increasingly also concern safety. Additionally, the European General Data Protection Regulation (GDPR) requires decisions to be understandable.

Software comparison for xAI

To solve this problem, an entire research field has emerged: "Explainable Artificial Intelligence," abbreviated as xAI. There are now numerous digital tools on the market that make complex AI solution paths explainable. For example, they highlight in an image the pixels that led to the sorting out of defective parts. Experts from the Fraunhofer Institute for Manufacturing Engineering and Automation IPA in Stuttgart have now compared nine common explanation methods—such as LIME, SHAP, or Layer-Wise Relevance Propagation—and evaluated them using sample applications. Three criteria were particularly important:

- Stability: For the same task, the program should always provide the same explanation. It should not happen that, for an anomaly in the production machine, Sensor A is responsible once and Sensor B another time. This would destroy trust in the algorithm and complicate deriving action options.
- Consistency: At the same time, only slightly different input data should produce similar explanations.
- Fidelity: It is also particularly important that explanations accurately reflect the behavior of the AI model. It must not happen that the explanation for denying a bank loan cites an age that is too high, although the actual reason was the insufficient income.

The application case is decisive

Conclusion of the study: All examined explanation methods proved to be useful. "But there is no one perfect method," says Nina Schaaf, who is responsible for the study at Fraunhofer IPA. There are significant differences, for example, in the runtime required by a method. The choice of the best software also depends heavily on the specific task. For example, Layer-Wise Relevance Propagation and Integrated Gradients are particularly suitable for image data. "And finally, the target audience of an explanation is always important: an AI developer wants and should receive an explanation presented differently than the production manager, because both draw different conclusions from the explanations," summarizes Schaaf.

The study "Explainable AI in Practice" is available free of charge at the following link: https://www.ki-fortschrittszentrum.de/de/studien/erklaerbare-ki-in-der-praxis.html