ABC-GAN: Spatially Constrained Counterfactual Generation for Image Classification Explanations

Mindlin, Dimitry; Schilling, Malte; Cimiano, Philipp

Research article in edited proceedings (conference) | Peer reviewed

Abstract

There is a growing interest in methods that explain predictions of image classification models to increase algorithmic transparency. Counterfactual Explanations (CFEs) provide a causal explanation as they introduce changes in the original image that change the classifier’s prediction. Current counterfactual generation approaches suffer from the fact that they potentially modify a too large region in the image that is not entirely causally related to a classifier’s decision, thus not always providing targeted explanations. We propose a new method, Attention Based Counterfactuals via CycleGAN (ABC-GAN), that combines attention-guided object translation with counterfactual image generation via Generative Adversarial Networks. To generate an explanation, ABC-GAN incorporates both a counterfactual loss and the classifier’s attention mechanism. By leveraging the attention map generated by GradCAM++, ABC-GAN alters regions in the image that are important for the classifier’s prediction. This approach ensures that the generated explanation focuses on the specific areas that contribute to the change in prediction while preserving the background and non-salient regions of the original image. We apply our approach to medical X-ray datasets (MURA Bone X-Ray, RSNA Chest X-ray) and compare it to state-of-the-art methods. We demonstrate the feasibility and, in the case of the MURA dataset, the superiority of ABC-GAN in all the measured metrics with the highest percentage of counterfactuals (99% Validity) and image similarity. On the other dataset, our method outperforms the competitive methods in small changes and image similarity. We argue that ABC-GAN is thus beneficial for classification problems requiring precise and minimal CFEs.

Details about the publication

PublisherLongo, Luca
Book titleExplainable Artificial Intelligence
Page range260-282
Publishing companySpringer
Place of publicationLissabon
StatusPublished
Release year2023
Language in which the publication is writtenEnglish
ConferenceFirst World Conference xAI 2023, Lissabon, Portugal
ISBN978-3-031-44063-2
DOI: 10.1007/978-3-031-44064-9_15
KeywordsCounterfactuals; XAI; Generative Explanations

Authors from the University of Münster

Schilling, Malte
Professorship of Practical Comupter Science (Prof. Schilling)