Grade student submissions from Canvas against a rubric
An instructor's pipeline pulls student files from Canvas, has Claude score each per criterion with comments, and posts grades back after human review. Task: Vaibhav Bhandari built a pipeline using Canvas API and Claude to grade Hacker Techniques & Exploits labs. Scripts download submissions, extract text or render scanned PDFs as images, and send them with a YAML rubric to Claude for scoring. Claude returns per-criterion scores, first-person comments, and flags for issues like AI-generated content. The instructor reviews a CSV of results, adjusts comments, and confirms uploads to Canvas.; tools: Claude. Human role: Source evidence is unavailable for specific human roles; the provided text indicates the instructor reviews a CSV and confirms grades, but details are truncated.; limitations: Scores were not calibrated to the instructor's grading; adding examples of high scores in rubric instructions is recommended. Responses are not cached, so re-runs incur additional API costs. The workflow requires a personal Canvas API token and manual review of flagged items before finalizing grades..
Tools
Published on Agent Work Atlas:
Overview
An instructor's pipeline pulls student files from Canvas, has Claude score each per criterion with comments, and posts grades back after human review.
Workflow
Vaibhav Bhandari built a pipeline using Canvas API and Claude to grade Hacker Techniques & Exploits labs. Scripts download submissions, extract text or render scanned PDFs as images, and send them with a YAML rubric to Claude for scoring. Claude returns per-criterion scores, first-person comments, and flags for issues like AI-generated content. The instructor reviews a CSV of results, adjusts comments, and confirms uploads to Canvas.
Reported result
On Lab 5, 13 of 22 submissions were flagged, mostly for suspected AI-generated content or thin evidence. Scoring was internally consistent but not calibrated to the instructor's own grading standards. The instructor still read every submission and heard every student present in class, using AI feedback to prepare specific questions.
Human role
Source evidence is unavailable for specific human roles; the provided text indicates the instructor reviews a CSV and confirms grades, but details are truncated.
Limitations
Scores were not calibrated to the instructor's grading; adding examples of high scores in rubric instructions is recommended. Responses are not cached, so re-runs incur additional API costs. The workflow requires a personal Canvas API token and manual review of flagged items before finalizing grades.
Try it
I need to set up an AI-assisted grading pipeline for my course. First, generate a personal Canvas API token from Account Settings. Create a YAML rubric file defining criteria, points, and extra instructions with examples of high scores. Write a script to fetch student submissions from Canvas, extracting text from PDFs/Word docs or rendering scanned PDFs as images. Send these contents plus the rubric to Claude with a system prompt requiring first-person feedback, per-criterion explanations, and a JSON response including scores, comments, and flags for AI-generated content or incomplete work. Save results to a CSV. Review the CSV, especially flagged items, and adjust scores/comments as needed. Finally, use the script to upload confirmed grades and comments back to Canvas SpeedGrader. Ensure human review is mandatory before posting. Start with a small sample of 3 submissions to test the prompt and rubric effectiveness before running the full batch.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.