Run oral final exams with a voice agent
This is brilliant. Task: A professor implemented an oral final exam system using a voice agent to conduct interviews and a council of LLMs to grade responses. The workflow involved cold-calling students to verify understanding, then transitioning to AI-driven oral exams to assess real-time reasoning. Human instructors defined the evaluation criteria and reviewed the generated transcripts and feedback.; tools: custom-built or unnamed agent. Human role: The original post does not say how a person stayed involved.; limitations: The source is a social media post, so specific technical configurations and prompt engineering details are not provided. The low cost and high efficiency may not scale linearly to larger classes without infrastructure adjustments. Student preference for the AI format was low, indicating potential resistance to adoption..
Tools
custom-built or unnamed agent
Published on Agent Work Atlas:
Overview
This is brilliant.
Workflow
A professor implemented an oral final exam system using a voice agent to conduct interviews and a council of LLMs to grade responses. The workflow involved cold-calling students to verify understanding, then transitioning to AI-driven oral exams to assess real-time reasoning. Human instructors defined the evaluation criteria and reviewed the generated transcripts and feedback.
Reported result
The system examined 36 students over 9 days with an average duration of 25 minutes per exam, costing approximately $15 total ($0.42 per student). It produced full transcripts, audit trails, and actionable feedback, revealing a class-wide weakness in A/B testing. Only 13% of students preferred the AI oral format.
Human role
The original post does not say how a person stayed involved.
Limitations
The source is a social media post, so specific technical configurations and prompt engineering details are not provided. The low cost and high efficiency may not scale linearly to larger classes without infrastructure adjustments. Student preference for the AI format was low, indicating potential resistance to adoption.
Try it
I want to pilot an AI-assisted oral exam for my [Course Name] class. First, create a voice agent prompt that asks [Student Name] to explain their reasoning behind [Specific Project Decision] for 25 minutes. Then, set up a council of three LLMs to grade the transcript based on [Rubric Criteria]. Generate a full transcript and a summary of key insights. Before running this for real students, perform a dry run with a test transcript I provide. Show me the grading output and cost estimate. Do not send any actual calls or emails until I approve the test results. If the LLMs disagree on the grade, flag it for my manual review.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.