Explaining misclassification and attacks in deep learning via random forests

Artificial intelligence, and machine learning (ML) in particular, is being used for different purposes that are critical for human life. To avoid an algorithm-based authoritarian society, AI-based decisions should generate trust by being explainable. Explainability is not only a moral requirement but also a legal requirement stated in the European General Data Protection Regulation (GDPR). Additionally, it is also beneficial for researchers and practitioners relying on AI methods, who need to know whether the decisions made by the algorithms they use are rational, lack bias and have not been subjected to learning attacks. To achieve AI explainability, it must be possible to derive explanations in a systematic and automatic way. A common approach is to use a simpler, more understandable decision algorithm to build a surrogate model of the unexplainable, a.k.a. black-box model (typically a deep learning algorithm). To this end, we should avoid surrogate models that are too large to be understood by humans. In this work, we focus on explaining the behavior of black-box models by using random forests containing a fixed number of decision trees of limited depth as surrogates. In particular, our aim is to determine the causes underlying misclassification by the black-box model. Our approach is to leverage partial decision trees in the forest to calculate the importance of the features involved in the wrong decisions. We achieve great accuracy in detecting and explaining misclassification by deep learning models constructed via federated learning that have suffered attacks.

Tags
Data and Resources
To access the resources you must log in
  • Link to PublicationPDF

    The resource: 'Link to Publication' is not accessible as guest user. You must login to access it!
Additional Info
Field Value
Author Haffar, Rami
Author Domingo-Ferrer, Josep josep.domingo@urv.cat
Author Sánchez, David david.sanchez@urv.cat
DOI https://doi.org/10.1007/978-3-030-57524-3_23
Group Select Group
Publisher Lecture Notes in Computer Science vol. 12256 (Modeling Decisions for Artificial Intelligence-MDAI 2020 )
Source International Conference on Modeling Decisions for Artificial Intelligence MDAI 2020: Modeling Decisions for Artificial Intelligence pp 273-285
Thematic Cluster Privacy Enhancing Technology [PET]
system:type JournalArticle
Management Info
Field Value
Author Wright Joanna
Maintainer Jesus Manjon
Version 1
Last Updated 5 March 2021, 12:48 (CET)
Created 15 February 2021, 14:41 (CET)