Preprint
Machine Learning

A Survey on the Explainability of Supervised Machine Learning

Nadia Burkart(Fraunhofer Institute of Optronics, System Technologies and Image Exploitation), Marco F. Huber(University of Stuttgart)
January 19, 2021Journal of Artificial Intelligence Research966 citations

966

Citations

36

Influential Citations

Journal of Artificial Intelligence Research

Venue

2021

Year

Abstract

Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or finance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Learning (SML). We conduct a state-of-the-art survey that reviews past and recent explainable SML approaches and classifies them according to the introduced definitions. Finally, we illustrate principles by means of an explanatory case study and discuss important future directions.

Analysis

Why This Paper Matters

As machine learning models, particularly deep neural networks, achieve high accuracy in critical domains such as healthcare and finance, their lack of transparency becomes a major barrier to trust and adoption. This survey addresses the pressing need for explainability by providing a structured overview of the field. It is especially timely given the increasing regulatory and ethical demands for accountable AI. By clarifying definitions and categorizing methods, the paper helps practitioners choose appropriate explainability techniques and guides researchers toward open problems.

The paper's comprehensive classification of explainable SML approaches into intrinsic vs. post-hoc, model-specific vs. model-agnostic, and global vs. local explanations offers a clear framework. This taxonomy is valuable for both newcomers and experts, as it organizes a rapidly growing body of work. The inclusion of a case study further bridges theory and practice, demonstrating how different methods can be applied to a concrete problem.

Technical Contributions

  • Definitions and Taxonomy: Establishes clear definitions for interpretability, explainability, and related terms, and proposes a classification scheme based on the stage of explanation (ante-hoc vs. post-hoc), scope (global vs. local), and model dependency.
  • Comprehensive Survey: Reviews a wide range of methods including LIME, SHAP, Grad-CAM, feature importance, rule extraction, and surrogate models, categorizing them within the proposed taxonomy.
  • Case Study: Applies multiple explainability methods to a supervised learning task, illustrating their outputs and trade-offs in a practical setting.
  • Future Directions: Identifies key challenges such as evaluating explanation quality, handling complex models, and integrating human-centric evaluation.

Results

The paper does not present novel experimental results but synthesizes existing work. It reports that the field has produced numerous methods, yet lacks standardized evaluation metrics. The case study demonstrates that different methods can yield complementary insights, but also highlights inconsistencies and the need for careful interpretation. No quantitative metrics are provided in the abstract.

Significance

This survey has become a highly cited reference (966 citations) in the explainable AI community. It provides a common vocabulary and framework that facilitates communication and comparison across research groups. By outlining future directions, it helps steer the field toward more rigorous evaluation and practical deployment. The work is particularly relevant for AI practitioners in regulated industries who need to justify model decisions.