Refactor a million-line repo with 1,393 subagents
Or: How to get $1.8M of value from $19K of tokens. Task: Or: How to get $1.8M of value from $19K of tokens TLDR: Hermes Agent autonomously plowed through about a million lines of unglamorous cleanup, freeing up Teknium and team to continue pushing features to users. We had long been putting off a thorough cleanup of Hermes, our open-source agent, because it meant taking engineers away from features and bug fixes. By September, the repository had more than a million lines of non-test Python. gateway/run.py alone was 34,847 lines long. I wanted smaller files, shared helpers, and fewer enormous functions to work through when something broke. On September 2nd, I asked my regular Hermes agent to do the cleanup. The main run lasted about nineteen active hours and dispatched 1,393 subagents, reaching 218 running at once. After a restart, a continuation session, and two rounds of community review and fixes, I merged the PR on September 4th. It reduced non-test Python source by 34.4%. The estimated model cost was about $19,300 for the main run, or roughly $25k including follow-up sessions. That excludes human review time. Our rough staffing estimate for doing the work manually was $150k-$1.8M for a small team working for two months to two years. We couldn't justify scheduling it alongside everything else we needed to ship. I use Hermes Agent everyday to develop Hermes Agent. As we fix bugs and review changes together, Hermes records what worked and updates its skills when I correct its approach or it finds the right pathway to solve new problems. By the time I asked for this refactor, it had learned my preferred procedures and standards and could apply them to a much larger job: I want a massive simplification set of PRs. or a single monolithic PR. I want LOC to drop dramatically. Minimum 30% overall. I want god files broken up. I want simplification across the board. I want unification of helpers and methods that can be reused. I want less if-if-if-if-if-if-else routing. I want code legibility up. I want interpretability of the codebase and how things connect to each other up. I want elegance. I want superfluous excess bloat code cleaned up and removed. I want it all done fully. No excuses. No waiting for my decisions. Get it all done, and present me a PR or set of PRs when done. I used /goal, which gives Hermes a standing objective and prompts it to continue when it would otherwise stop. Self-improvement (for real) My hermes-agent-dev skill grew out of my everyday work on the repository. When we worked out a procedure or I corrected a mistake, Hermes (automatically) noticed this and saved the reusable lesson. Over time, it accumulated instructions about how to prepare a PR, which shortcuts to avoid, and how to verify a change. Skills are readable Markdown documents, with reference files and scripts where needed, that the agent can load for later tasks. Hermes writes and revises them as it works. The current version of hermes-agent-dev includes this instruction for a failing check: repro on origin/main HEAD in clean env to check whether it's pre-existing In other words, run the failing test on unchanged code to help determine whether your change caused it. Hermes used the same kind of comparison during the refactor: it established a frozen baseline and checked failures against it as it integrated the workers' changes. I send that skill around to all our engineers. They can install it in their own Hermes setups, so their agents can use procedures and corrections developed in my sessions. They get the benefit of that work without having to repeat the sessions themselves, and their agents can adapt the skill as they use it. Running the refactor The orchestrator measured the codebase and divided it into 36 non-overlapping groups. It used my objective and the accumulated guidance to prepare written assignments, without my having to brief each worker. Workers used git worktrees, separate checkouts where they could make changes without overwriting one another's files. Their briefs identified the code to simplify, the interfaces to preserve, and the checks required before committing. Some workers delegated parts of their assignments again. The tree reached three levels below the original agent, which handled coordination rather than editing source files: it wrote assignments and scripts, read worker reports, integrated branches, and ran checks.; tools: Hermes Agent. Human role: The original post does not say how a person stayed involved.; limitations: The workflow requires a robust agent system capable of managing hundreds of concurrent subagents and git worktrees. Authentication token expiration caused a mid-run failure, necessitating a restart and handoff. Human review is critical to catch regressions that automated tests miss, such as breaking external plugin interfaces..
Tools
Published on Agent Work Atlas:
Overview
Or: How to get $1.8M of value from $19K of tokens
Workflow
Or: How to get $1.8M of value from $19K of tokens TLDR: Hermes Agent autonomously plowed through about a million lines of unglamorous cleanup, freeing up Teknium and team to continue pushing features to users. We had long been putting off a thorough cleanup of Hermes, our open-source agent, because it meant taking engineers away from features and bug fixes. By September, the repository had more than a million lines of non-test Python. gateway/run.py alone was 34,847 lines long. I wanted smaller files, shared helpers, and fewer enormous functions to work through when something broke. On September 2nd, I asked my regular Hermes agent to do the cleanup. The main run lasted about nineteen active hours and dispatched 1,393 subagents, reaching 218 running at once. After a restart, a continuation session, and two rounds of community review and fixes, I merged the PR on September 4th. It reduced non-test Python source by 34.4%. The estimated model cost was about $19,300 for the main run, or roughly $25k including follow-up sessions. That excludes human review time. Our rough staffing estimate for doing the work manually was $150k-$1.8M for a small team working for two months to two years. We couldn't justify scheduling it alongside everything else we needed to ship. I use Hermes Agent everyday to develop Hermes Agent. As we fix bugs and review changes together, Hermes records what worked and updates its skills when I correct its approach or it finds the right pathway to solve new problems. By the time I asked for this refactor, it had learned my preferred procedures and standards and could apply them to a much larger job: I want a massive simplification set of PRs. or a single monolithic PR. I want LOC to drop dramatically. Minimum 30% overall. I want god files broken up. I want simplification across the board. I want unification of helpers and methods that can be reused. I want less if-if-if-if-if-if-else routing. I want code legibility up. I want interpretability of the codebase and how things connect to each other up. I want elegance. I want superfluous excess bloat code cleaned up and removed. I want it all done fully. No excuses. No waiting for my decisions. Get it all done, and present me a PR or set of PRs when done. I used /goal, which gives Hermes a standing objective and prompts it to continue when it would otherwise stop. Self-improvement (for real) My hermes-agent-dev skill grew out of my everyday work on the repository. When we worked out a procedure or I corrected a mistake, Hermes (automatically) noticed this and saved the reusable lesson. Over time, it accumulated instructions about how to prepare a PR, which shortcuts to avoid, and how to verify a change. Skills are readable Markdown documents, with reference files and scripts where needed, that the agent can load for later tasks. Hermes writes and revises them as it works. The current version of hermes-agent-dev includes this instruction for a failing check: repro on origin/main HEAD in clean env to check whether it's pre-existing In other words, run the failing test on unchanged code to help determine whether your change caused it. Hermes used the same kind of comparison during the refactor: it established a frozen baseline and checked failures against it as it integrated the workers' changes. I send that skill around to all our engineers. They can install it in their own Hermes setups, so their agents can use procedures and corrections developed in my sessions. They get the benefit of that work without having to repeat the sessions themselves, and their agents can adapt the skill as they use it. Running the refactor The orchestrator measured the codebase and divided it into 36 non-overlapping groups. It used my objective and the accumulated guidance to prepare written assignments, without my having to brief each worker. Workers used git worktrees, separate checkouts where they could make changes without overwriting one another's files. Their briefs identified the code to simplify, the interfaces to preserve, and the checks required before committing. Some workers delegated parts of their assignments again. The tree reached three levels below the original agent, which handled coordination rather than editing source files: it wrote assignments and scripts, read worker reports, integrated branches, and ran checks.
Reported result
The refactor reduced non-test Python source by 34.4%, with gateway/run.py shrinking from 34,847 to 5,512 lines. The model cost was approximately $25,000, significantly lower than the estimated $150k-$1.8M manual effort. However, the process revealed regressions in exception handling and public name removals that required manual fixes before merging.
Human role
The original post does not say how a person stayed involved.
Limitations
The workflow requires a robust agent system capable of managing hundreds of concurrent subagents and git worktrees. Authentication token expiration caused a mid-run failure, necessitating a restart and handoff. Human review is critical to catch regressions that automated tests miss, such as breaking external plugin interfaces.
Try it
I want to refactor [replaceable: large codebase] using Hermes Agent. Set a standing objective to reduce LOC by [replaceable: percentage] and split files over [replaceable: line count]. Dispatch subagents to handle [replaceable: number] non-overlapping code groups using git worktrees. Ensure workers preserve public interfaces and run checks against a frozen baseline. After the main run, I will review the PR for regressions in exception handling and removed public names before merging. First, perform a dry run on [replaceable: small sample module] to verify the agent's ability to maintain functionality and report token usage. Do not merge any changes without my explicit confirmation.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.