Screen interview code submissions with a Codex review agent
Hey all. Task: Hey all. I was talking to someone at work recently about the entry level position they're trying to fill, and they said they've been completely inundated with applicants, far more than we've gotten in the past. This makes sense given the state of the industry, but they're bumping into a new issue: a ton of people are straight up lying about their qualifications, which bumps them to the top of the list, but then the screening comes and they're very obviously just plugging questions into an LLM and waiting to spit the answer back out. When pressed for details about their decision making, they come up blank. The biggest issue is that these people, who are presumably taking the job posting and running it through some AI to create the perfect application, are probably pushing down the applicants who actually have the experience we're looking for. We don't hire super often, so I'm wondering if places that have dealt with this more often have solutions?; tools: Codex, ChatGPT. Human role: The original post does not say how a person stayed involved.; limitations: The source is a Reddit post with no verified data on effectiveness. Prerequisites include access to Codex and ChatGPT APIs. Integration uncertainty exists regarding how to distinguish between legitimate use of AI for drafting and dishonest use for answering technical questions. Human approval is mandatory for all hiring decisions..
Tools
Published on Agent Work Atlas:
Overview
Hey all.
Workflow
Hey all. I was talking to someone at work recently about the entry level position they're trying to fill, and they said they've been completely inundated with applicants, far more than we've gotten in the past. This makes sense given the state of the industry, but they're bumping into a new issue: a ton of people are straight up lying about their qualifications, which bumps them to the top of the list, but then the screening comes and they're very obviously just plugging questions into an LLM and waiting to spit the answer back out. When pressed for details about their decision making, they come up blank. The biggest issue is that these people, who are presumably taking the job posting and running it through some AI to create the perfect application, are probably pushing down the applicants who actually have the experience we're looking for. We don't hire super often, so I'm wondering if places that have dealt with this more often have solutions?
Reported result
No measured results are available from the source. The original post describes a problem with AI-generated applications but does not report outcomes of any specific solution. A proposed acceptance check is to verify that the AI tool correctly identifies at least 80% of known LLM-generated responses in a controlled test set without false positives on genuine human answers.
Human role
The original post does not say how a person stayed involved.
Limitations
The source is a Reddit post with no verified data on effectiveness. Prerequisites include access to Codex and ChatGPT APIs. Integration uncertainty exists regarding how to distinguish between legitimate use of AI for drafting and dishonest use for answering technical questions. Human approval is mandatory for all hiring decisions.
Try it
Act as a technical recruiter assistant. I have a set of [replaceable: 5 sample screening responses] from entry-level applicants. Use Codex to analyze these responses for signs of LLM generation, such as generic phrasing or lack of specific decision-making details. Compare these against [replaceable: 2 known genuine responses]. Output a JSON report for each sample with fields: 'candidate_id', 'llm_suspicion_score' (0-100), 'key_indicators', and 'recommendation' (review/reject/pass). Do not make final hiring decisions. First, process one sample and show me the output format for confirmation before proceeding with the rest. If the analysis is inconclusive, flag it for human review. Ensure no personal data is stored beyond this session.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.