Run an AI software factory that opens, reviews, and merges PRs
TL;DR. Task: TL;DR An AI software factory is five stages with a gate at each one, and the coding agent is the cheapest piece of it. Every company that made this work built the gates before the fleet. Spotify's Fleetshift shipped in 2023, two years before it had an agent to put in it. Token spend buys more generated code, and review capacity has to be built some other way, which is most of the design work. Where Firecrawl fits. Agents need live web context the repo does not carry. Search + scrape for the open web and a curated developer index for code, both behind one MCP block, with prompt injection detection on every fetch. On January 6, 2026, Stephen Toub opened nine pull requests from his phone at 35,000 feet. Seven of them merged. He works on dotnet/runtime, and he wrote up what the experience told him: AI changes the economics of code production. One person with good judgment and a phone can generate PRs faster than a team can review them. Ten months of data from the same repository backs him up. Agents opened 878 pull requests there, and two reviewers wrote 36% of all the feedback on them. The bottleneck had moved from writing code to checking it, and the checking still ran through a handful of people. Stripe, Spotify, Shopify, Uber, and Ramp each hit the same wall, and each built the same kind of system around its agents. That system is what this article means by an AI software factory: a queue that decides what is worth starting, isolated workspaces, a shared tool layer, automatic verification, and a human at the merge gate. The five stages below cover the published architecture behind each one and the configuration to build it yourself. Two things this guide assumes. You have already picked an agent, which we compared in our roundup of the best AI coding agents, and you know roughly how AI agents reason, call tools, and pull in web context. The factory is everything around the agent. What is an AI software factory? An AI software factory is the system around a coding agent rather than the agent itself. Work arrives from a queue, agents run in isolated workspaces, verification happens automatically, and a human sits at an explicit merge gate. Some writers call it an agentic software factory. When you run a coding agent on your laptop, you choose the task, you watch it work, you read the diff, and you merge. Everything except the typing is still you, and your attention is the limit. A software factory moves those steps into infrastructure. Intake rules decide which issue an agent picks up. Workspaces are provisioned rather than set up by hand. Verification runs before a human is involved at all. The person shows up at the end, for the decisions that carry accountability. Addy Osmani puts it more compactly: A software factory is harnessing loops at scale. The loops he means are the ones we covered in our guide to loop engineering: an agent that runs, checks its own work, and runs again until a verifier says stop. The software factory is the machinery that runs many of them at once without anyone watching. Three properties separate a software factory from a pile of scripts: Work arrives from a queue. It enters from issues, alerts, or a Slack channel, and the system decides what is worth starting rather than a person typing prompts. Environments are disposable. Every agent gets a clean workspace it can destroy, so a bad run costs nothing and parallel runs cannot corrupt each other. Verification runs before review. By the time a diff reaches a person, it has already compiled, passed tests, and been checked for scope. Without the third you have not built a software factory. You have built a machine that generates review work faster than you can absorb it, and most of this article is about avoiding that. Guillermo Rauch, Vercel's CEO, made the strategic case when Vercel open sourced its own reference platform for cloud coding agents, and he named the same systems this article draws on: You've heard that companies like Stripe (Minions), Ramp (Inspect), Spotify (Honk), Block (Goose), and others are building their own "AI software factories". Why? [...] On a business level, the moat of software companies will shift from 'the code they wrote', to the 'means of production' of that code. The alpha is in your factory. @rauchg, April 14, 2026 The five stages every published software factory shares; tools: custom-built or unnamed agent. Human role: The original post does not say how a person stayed involved.; limitations: Prerequisites include a pre-selected coding agent and infrastructure for isolated workspaces (e.g., containers or cloud sandboxes). The primary constraint is human review capacity, which becomes the bottleneck if automated verification is insufficient. Operating costs involve token spend for generation and infrastructure for isolation. Integration uncertainty exists regarding how well automated gates catch subtle logic errors, as seen in the lower success rate for complex performance work. Human approval is mandatory at the merge gate for accountability..
Tools
custom-built or unnamed agent
Published on Agent Work Atlas:
Overview
TL;DR
Workflow
TL;DR An AI software factory is five stages with a gate at each one, and the coding agent is the cheapest piece of it. Every company that made this work built the gates before the fleet. Spotify's Fleetshift shipped in 2023, two years before it had an agent to put in it. Token spend buys more generated code, and review capacity has to be built some other way, which is most of the design work. Where Firecrawl fits. Agents need live web context the repo does not carry. Search + scrape for the open web and a curated developer index for code, both behind one MCP block, with prompt injection detection on every fetch. On January 6, 2026, Stephen Toub opened nine pull requests from his phone at 35,000 feet. Seven of them merged. He works on dotnet/runtime, and he wrote up what the experience told him: AI changes the economics of code production. One person with good judgment and a phone can generate PRs faster than a team can review them. Ten months of data from the same repository backs him up. Agents opened 878 pull requests there, and two reviewers wrote 36% of all the feedback on them. The bottleneck had moved from writing code to checking it, and the checking still ran through a handful of people. Stripe, Spotify, Shopify, Uber, and Ramp each hit the same wall, and each built the same kind of system around its agents. That system is what this article means by an AI software factory: a queue that decides what is worth starting, isolated workspaces, a shared tool layer, automatic verification, and a human at the merge gate. The five stages below cover the published architecture behind each one and the configuration to build it yourself. Two things this guide assumes. You have already picked an agent, which we compared in our roundup of the best AI coding agents, and you know roughly how AI agents reason, call tools, and pull in web context. The factory is everything around the agent. What is an AI software factory? An AI software factory is the system around a coding agent rather than the agent itself. Work arrives from a queue, agents run in isolated workspaces, verification happens automatically, and a human sits at an explicit merge gate. Some writers call it an agentic software factory. When you run a coding agent on your laptop, you choose the task, you watch it work, you read the diff, and you merge. Everything except the typing is still you, and your attention is the limit. A software factory moves those steps into infrastructure. Intake rules decide which issue an agent picks up. Workspaces are provisioned rather than set up by hand. Verification runs before a human is involved at all. The person shows up at the end, for the decisions that carry accountability. Addy Osmani puts it more compactly: A software factory is harnessing loops at scale. The loops he means are the ones we covered in our guide to loop engineering: an agent that runs, checks its own work, and runs again until a verifier says stop. The software factory is the machinery that runs many of them at once without anyone watching. Three properties separate a software factory from a pile of scripts: Work arrives from a queue. It enters from issues, alerts, or a Slack channel, and the system decides what is worth starting rather than a person typing prompts. Environments are disposable. Every agent gets a clean workspace it can destroy, so a bad run costs nothing and parallel runs cannot corrupt each other. Verification runs before review. By the time a diff reaches a person, it has already compiled, passed tests, and been checked for scope. Without the third you have not built a software factory. You have built a machine that generates review work faster than you can absorb it, and most of this article is about avoiding that. Guillermo Rauch, Vercel's CEO, made the strategic case when Vercel open sourced its own reference platform for cloud coding agents, and he named the same systems this article draws on: You've heard that companies like Stripe (Minions), Ramp (Inspect), Spotify (Honk), Block (Goose), and others are building their own "AI software factories". Why? [...] On a business level, the moat of software companies will shift from 'the code they wrote', to the 'means of production' of that code. The alpha is in your factory. @rauchg, April 14, 2026 The five stages every published software factory shares
Reported result
On dotnet/runtime, agents opened 878 pull requests, with two reviewers providing 36% of all feedback, highlighting a severe review bottleneck. Small PRs (1-50 lines) succeeded 76-80% of the time, while performance work landed at only 54.5%. Adding specific instruction files and firewall configs increased agent success rates from 41.7% to 71%. However, the source does not provide overall business impact metrics beyond these repository-specific success rates and reviewer load statistics.
Human role
The original post does not say how a person stayed involved.
Limitations
Prerequisites include a pre-selected coding agent and infrastructure for isolated workspaces (e.g., containers or cloud sandboxes). The primary constraint is human review capacity, which becomes the bottleneck if automated verification is insufficient. Operating costs involve token spend for generation and infrastructure for isolation. Integration uncertainty exists regarding how well automated gates catch subtle logic errors, as seen in the lower success rate for complex performance work. Human approval is mandatory at the merge gate for accountability.
Try it
Act as an engineering lead implementing an AI software factory pilot. I need to set up a controlled workflow for [replaceable: repository name] using [replaceable: coding agent tool]. First, define an intake filter to select only small, well-specified issues (1-50 lines) labeled [replaceable: issue label]. Provision isolated workspaces using [replaceable: isolation method, e.g., git worktrees or containers]. Configure a verification stage that runs linting and tests before any human review. Set up a merge gate requiring human approval for all agent-generated PRs. Execute a dry run on [replaceable: number] sample issues. Measure success by PR merge rate and reviewer feedback volume. If the dry run shows >50% failure or excessive reviewer load, halt and adjust the intake criteria. Do not deploy to production without confirming the verification gates catch at least 80% of syntax errors. Output a report detailing the workflow configuration, dry run results, and recommended next steps for scaling.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.