Preprint
Machine Learning

Explainable Generative AI (GenXAI): a survey, conceptualization, and research agenda

Johannes Schneider(University of Liechtenstein)
September 15, 2024Artificial Intelligence Review147 citations

147

Citations

11

Influential Citations

Artificial Intelligence Review

Venue

2024

Year

Abstract

Abstract Generative AI (GenAI) represents a shift from AI’s ability to “recognize” to its ability to “generate” solutions for a wide range of tasks. As generated solutions and applications grow more complex and multi-faceted, new needs, objectives, and possibilities for explainability (XAI) have emerged. This work elaborates on why XAI has gained importance with the rise of GenAI and the challenges it poses for explainability research. We also highlight new and emerging criteria that explanations should meet, such as verifiability, interactivity, security, and cost considerations. To achieve this, we focus on surveying existing literature. Additionally, we provide a taxonomy of relevant dimensions to better characterize existing XAI mechanisms and methods for GenAI. We explore various approaches to ensure XAI, ranging from training data to prompting. Our paper provides a concise technical background of GenAI for non-technical readers, focusing on text and images to help them understand new or adapted XAI techniques for GenAI. However, due to the extensive body of work on GenAI, we chose not to delve into detailed aspects of XAI related to the evaluation and usage of explanations. Consequently, the manuscript appeals to both technical experts and professionals from other fields, such as social scientists and information systems researchers. Our research roadmap outlines over ten directions for future investigation.

Analysis

Why This Paper Matters

Generative AI (GenAI) has rapidly shifted the AI paradigm from recognition to generation, introducing novel challenges for explainability. Traditional XAI methods designed for discriminative models often fall short when applied to generative outputs that are open-ended, high-dimensional, and lack ground truth. This survey is timely because it systematically maps the landscape of explainability for GenAI, a domain where trust, safety, and regulatory compliance are increasingly critical. By highlighting new criteria like verifiability and interactivity, the paper addresses practical needs that arise when users must validate AI-generated content or interact with models in real time.

The paper also serves as a bridge between technical and non-technical audiences, making it valuable for interdisciplinary collaboration. Its taxonomy provides a common language for researchers from machine learning, information systems, and social sciences to discuss and compare XAI methods. The explicit research agenda with over ten directions helps prioritize future work, especially in areas like security and cost-efficient explanations.

Technical Contributions

The paper's main technical contribution is a multi-dimensional taxonomy for classifying XAI methods in GenAI. Key dimensions include:

  • Explanation target: whether the explanation covers the model, a specific output, or the training data.
  • Explanation format: textual, visual, or interactive explanations.
  • Method type: post-hoc vs. intrinsic, model-agnostic vs. model-specific.
  • Stage of application: during training, inference, or after generation.

The survey also categorizes approaches by where explainability is injected:

  • Data-level: curating training data to improve interpretability.
  • Model-level: designing architectures with built-in explainability (e.g., attention mechanisms).
  • Prompt-level: using prompts to elicit explanations from large language models.

Additionally, the paper introduces novel criteria for explanations in GenAI:

  • Verifiability: ability to check correctness of the explanation.
  • Interactivity: allowing users to query or refine explanations.
  • Security: robustness against adversarial manipulation of explanations.
  • Cost: computational and human effort required to generate explanations.

Results

As a survey, the paper does not present new experimental results. Instead, it synthesizes existing literature to identify trends and gaps. For example, it notes that most current XAI methods for GenAI focus on text and image domains, with limited work on video, audio, or multimodal generation. The paper also observes that interactive and verifiable explanations are still rare, pointing to a clear gap for future research. No quantitative metrics (e.g., accuracy, fidelity) are reported because the paper's contribution is conceptual rather than empirical.

Significance

This paper has broad significance for the AI field. It provides a structured foundation for researchers to design and evaluate explainability methods tailored to generative models, which are increasingly deployed in high-stakes applications like content creation, code generation, and scientific discovery. By emphasizing criteria like security and cost, it aligns XAI research with real-world deployment constraints. The interdisciplinary appeal—targeting both technical experts and social scientists—can foster more holistic approaches to trustworthy AI. The research roadmap will likely influence funding priorities and collaboration directions in the coming years.