How much of a pluralist is ChatGPT? A comparative study of value pluralism in generative AI chatbots

This study introduces pluralism as a benchmark for evaluating generative AI chatbots’ ability to navigate and potentially endorse diverse values and perspectives. Pluralism—the recognition and valuing of multiple, distinct perspectives—is essential for contemporary democratic societies, making systematic assessment of how AI systems handle value pluralism crucial. Using the Magic Wand Survey (MWS), which presents diverse value conflicts, we empirically measured 2 dimensions of pluralism—“Both/and Reasoning” (cognitive pluralism) and “Willingness to Preserve Difference” (behavioral pluralism)—across 120 iterations of 4 prominent AI chatbots, comparing them to a human sample (N = 335). AI chatbots exhibited higher cognitive than behavioral pluralism. Furthermore, while chatbots’ cognitive pluralism significantly exceeded human levels, their behavioral pluralism was comparable overall yet notably lower than humans on some morally charged issues (e.g., capital punishment). Clear and systematic variations also emerged between models: while general patterns held, the degree of pluralism varied substantially, with Gemini 1.5 Pro being the most pluralistic and ChatGPT 4o the least among those tested. These findings underscore the importance of pluralism as a key benchmark in AI evaluation.