An NYU professor recently conducted oral exams using a voice AI agent, revolutionizing evaluation in his “AI/ML Product Management” course. The experiment cost only $15 for 36 students, revealing significant knowledge gaps and flaws in traditional grading methods. Written assignments appeared overly polished, but many students struggled to explain their decisions during oral questioning. Ipeirotis argues that take-home papers can no longer reliably measure understanding due to AI’s capabilities. The AI oral exams were quick, averaging just 25 minutes per student, and far less expensive compared to human grading costs. However, the initial version faced criticism for the agent’s tone and questioning style. Despite these challenges, feedback showed the AI exam was seen as stressful but fairly assessed actual understanding. This new method could revitalize oral exams, allowing for random, relevant questions and eliminating issues of leaked exam content. The full prompts and grading methods used are available to the public.
Source link
