ASQ-IT: Interactive explanations for reinforcement-learning agents

Yotam Amitai, Ofra Amir, Guy Avni

Research output: Contribution to journalArticlepeer-review

Abstract

As reinforcement learning methods increasingly amass accomplishments, the need for comprehending their solutions becomes more crucial. Most explainable reinforcement learning (XRL) methods generate a static explanation depicting their developers' intuition of what should be explained and how. In contrast, literature from the social sciences proposes that meaningful explanations are structured as a dialog between the explainer and the explainee, suggesting a more active role for the user and her communication with the agent. In this paper, we present ASQ-IT – an interactive explanation system that presents video clips of the agent acting in its environment based on queries given by the user that describe temporal properties of behaviors of interest. Our approach is based on formal methods: queries in ASQ-IT's user interface map to a fragment of Linear Temporal Logic over finite traces (LTLf), which we developed, and our algorithm for query processing is based on automata theory. User studies show that end-users can understand and formulate queries in ASQ-IT and that using ASQ-IT assists users in identifying faulty agent behaviors.

Original languageEnglish
Article number104182
JournalArtificial Intelligence
Volume335
DOIs
StatePublished - Oct 2024

Bibliographical note

Publisher Copyright:
© 2024 Elsevier B.V.

Keywords

  • Explainable reinforcement learning
  • Formal verification
  • Interactive explanations

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'ASQ-IT: Interactive explanations for reinforcement-learning agents'. Together they form a unique fingerprint.

Cite this