Navigating the Human-oversight Dilemma in AI-based Systems

As artificial intelligence (AI) systems increasingly assume responsibility for critical decision-making, organizations face the challenge of balancing full automation with appropriate human oversight. While automation offers notable gains in efficiency, scalability, and performance, it also raises concerns regarding safety and ethical integrity. Overdependence on AI may result in biased outcomes, diminished transparency, and erosion of user trust, whereas excessive human intervention can compromise system efficiency and usability. In this research in progress, we investigate the inherent trade-offs between full automation and human oversight by analyzing the tensions that arise within AI decision-making processes. Utilizing an ontology-based framework, we systematically address these trade-offs and explore conditions under which human intervention is most beneficial. Drawing on a multi-criteria decision analysis (MCDA) approach, we propose methods for optimizing human oversight without undermining the strengths of AI. By doing so, this study offers advances for the development of trustworthy AI systems that uphold regulatory compliance and ethical responsibility, while maintaining operational effectiveness. Future research is needed for evaluating the effectiveness of the proposed methods when implemented in the field.