Brains vs. Algorithms? How Experts and Students See AI-Generated Distractors
Abstract
Abstract Multiple-choice questions (MCQs) are central to instruction and assessment, with distractors revealing student understanding and misconceptions. However, creating high-quality distractors is time-consuming, especially for emerging domains like K–12 AI education. This study explores using generative AI to support distractor creation in a self-paced online module integrating AI and Algebra 1. Five MCQs were selected to compare distractors written by human developers and ChatGPT, using expert reviews and log data from 80 students. Experts rated human distractors higher overall, though AI ones consistently ranked second. Log analysis showed human distractors drew more initial selections, while students who chose AI distractors spent more time engaging without differences in hint use or revisits. Transition patterns across attempts suggest AI-generated distractors can effectively guide students toward correct answers, highlighting their potential for scalable MCQ design.