Abstract: Despite achieving much success, the deep learning approach to AI has been criticised for being “black box”: the decisions made by such large and complex learning systems are difficult to explain or analyse. If the system makes a mistake in a critical situation then the consequences can be serious. The use of black box systems has obvious implications to transparency but also fairness and ultimately trust in current AI. System developers might also like to learn from system errors so that errors can be fixed. The area of explainable AI (XAI) has sought to open the black box by providing explanations for large AI systems mostly through the use of visualization techniques and user studies that seek to associate the decisions made by the system with known features of the deep learning model. In this talk, I will argue that XAI needs knowledge extraction and an objective measure of fidelity as a pre-requisite for visualization and user studies. As part of a neurosymbolic approach, knowledge extraction creates a bridge between sub-symbolic deep learning and logic-based symbolic AI with a precise semantics. I will exemplify how knowledge extraction can be used in the analysis of chest x-ray images as part of a collaborative project with Fujitsu Research to find and fix mistakes in image classification. I will conclude by arguing that knowledge extraction is an important tool, but is only one of many elements that are needed to address fairness and accountability in AI.
Speaker: Artur Garcez is Professor of Computer Science and Director of the Data Science Institute at City, University of London. He holds a PhD in Computing (2000) from Imperial College London. He is a Fellow of the British Computer Society (FBCS) and president of the steering committee of the Neural-Symbolic Learning and Reasoning Association. He has co-authored two books: Neural-Symbolic Cognitive Reasoning, 2009, and Neural-Symbolic Learning Systems, 2002. His research has led to publications in the journals Behavioral & Brain Sciences, Theoretical Computer Science, Neural Computation, Machine Learning, Journal of Logic and Computation, IEEE Transactions on Neural Networks, Journal of Applied Logic, Artificial Intelligence, and Studia Logica, and the flagship AI and Neural Computation conferences AAAI, NeurIPS, IJCAI, IJCNN, AAMAS and ECAI. Professor Garcez holds editorial positions with several scientific journals in the fields of Computational Logic and Artificial Intelligence, and has been Programme Committee member for several conferences, including IJCAI, IJCNN, NeurIPS and AAAI.