Speed up Rust code by having agents iteratively optimize it
In January 2025, I had a fun hypothesis for a blog post: can LLMs write better code if you keep asking them to “write better code”? Task: In January 2025, I had a fun hypothesis for a blog post: can LLMs write better code if you keep asking them to “write better code”? That was prior to the advent of robust agentic coding, but Claude Sonnet 3.5 was still able to iteratively improve on algorithmic Python code. The “better” instruction turned out to be underspecified: Sonnet abused that ambiguity to instead add a ton of useless features but the code was indeed faster. Even with the rise of agentic LLMs specifically RLHFed to handle solving common pass/fail coding problems, optimization is generally not a part of that suite. At the end of that blog post, I opined on a hypothetical future where LLMs could be able to write superfast Python code by instead writing Rust code and using PyO3 to bridge the languages to get Python’s ergonomics with Rust’s speed. An earlier draft of that post asserted that the same “write code better” instruction could instead be applied to the base Rust code and drastically improve its speed which would then propagate down to the Python code: however, back then I did not know enough about Rust and making such a claim would be too spicy without evidence. After months of testing and experimenting since the release of Opus 4.5 made agentic coding more viable, I can confidently confirm that modern agentic LLMs can indeed write Rust code that is significantly faster than current state-of-the-art approaches if given appropriate guardrails and constraints. Additionally, as LLMs have made drastic improvements in coding in each successive frontier model release since Opus 4.5, the optimizations have become even better, cumulatively resulting in anywhere from 2x-20x speedup depending on the domain. More importantly, this blog post is not a vaguepost and I am including both the prompts I used and the benchmark results. You’ve been warned. Iterative “Benchmaxxing”# At first, making software faster was a good quantitative way for me to test and compare these new agentic models. I used Rust as the target language primarily due to the Python integration and speed, but there are other aspects of the Rust language that make it particularly useful if a fast implementation is indeed discovered, such as memory safety and the ability to compile it to WebAssembly/WASM so it can run in a web browser without much effort. However, one important constraint I will follow that technically may not result in the fastest code is to forbid unsafe code whenever possible. My first test case was reimplementing machine learning algorithms in Rust, which would lead to a meaningful productivity increase for me as a data scientist if I had faster and scalable tooling. At the time, it was arrogant to assume that I could beat battle-tested algorithms that have been iterated on for over a decade and are already written in C so Rust’s low-level benefits are not as pronounced. The algorithm I wanted to optimize the most was UMAP, which is a valuable algorithm for dimensionality reduction I used in my work, but scales poorly to big data and is very slow, with alternatives such as cuML being time-consuming to set up. UMAP Rust crates such as umap-rs already exist where I could just fork them and prompt Claude Opus 4.5 to add Python/PyO3 support, but as an experiment and learning experience I wanted to have the agent write the algorithm from scratch with minimal Rust dependencies in order to make optimizations at as low of a level as possible. Rust has a comprehensive benchmarking tool with the criterion crate which all agents know how to leverage. criterion will run the benchmarks, track results across iterations to see if performance improved or regressed, and can calculate if this change is statistically significant or just noise. Typical criterion output, depicting a 3.5x speedup relative to the previous run of the benchmark. First, in the initial prompt for creating a Rust crate for UMAP, I asked Opus 4.5 to create benchmarks with different input data sizes since optimizations for small datasets may not work for large datasets and vice versa. This approach created benchmarks using criterion and I manually reran the benchmark suite after prompting performance feature improvements such as using faer for faster linear algebra and using simsimd for faster SIMD operations. This quickly became cumbersome as I had to manually rerun each benchmark after each change to verify there are no speed regressions.; tools: custom-built or unnamed agent. Human role: The original post does not say how a person stayed involved.; limitations: The workflow requires strict guardrails to prevent benchmark gaming, such as forbidding parallel benchmark runs and `target-cpu=native` flags. Human oversight is critical to detect subtle cheating methods like removing comments to fake code reduction or manipulating benchmark inputs. The approach is resource-intensive, requiring manual verification of statistical significance and code quality after each agent iteration..
Tools
custom-built or unnamed agent
Published on Agent Work Atlas:
Overview
In January 2025, I had a fun hypothesis for a blog post: can LLMs write better code if you keep asking them to “write better code”?
Workflow
In January 2025, I had a fun hypothesis for a blog post: can LLMs write better code if you keep asking them to “write better code”? That was prior to the advent of robust agentic coding, but Claude Sonnet 3.5 was still able to iteratively improve on algorithmic Python code. The “better” instruction turned out to be underspecified: Sonnet abused that ambiguity to instead add a ton of useless features but the code was indeed faster. Even with the rise of agentic LLMs specifically RLHFed to handle solving common pass/fail coding problems, optimization is generally not a part of that suite. At the end of that blog post, I opined on a hypothetical future where LLMs could be able to write superfast Python code by instead writing Rust code and using PyO3 to bridge the languages to get Python’s ergonomics with Rust’s speed. An earlier draft of that post asserted that the same “write code better” instruction could instead be applied to the base Rust code and drastically improve its speed which would then propagate down to the Python code: however, back then I did not know enough about Rust and making such a claim would be too spicy without evidence. After months of testing and experimenting since the release of Opus 4.5 made agentic coding more viable, I can confidently confirm that modern agentic LLMs can indeed write Rust code that is significantly faster than current state-of-the-art approaches if given appropriate guardrails and constraints. Additionally, as LLMs have made drastic improvements in coding in each successive frontier model release since Opus 4.5, the optimizations have become even better, cumulatively resulting in anywhere from 2x-20x speedup depending on the domain. More importantly, this blog post is not a vaguepost and I am including both the prompts I used and the benchmark results. You’ve been warned. Iterative “Benchmaxxing”# At first, making software faster was a good quantitative way for me to test and compare these new agentic models. I used Rust as the target language primarily due to the Python integration and speed, but there are other aspects of the Rust language that make it particularly useful if a fast implementation is indeed discovered, such as memory safety and the ability to compile it to WebAssembly/WASM so it can run in a web browser without much effort. However, one important constraint I will follow that technically may not result in the fastest code is to forbid unsafe code whenever possible. My first test case was reimplementing machine learning algorithms in Rust, which would lead to a meaningful productivity increase for me as a data scientist if I had faster and scalable tooling. At the time, it was arrogant to assume that I could beat battle-tested algorithms that have been iterated on for over a decade and are already written in C so Rust’s low-level benefits are not as pronounced. The algorithm I wanted to optimize the most was UMAP, which is a valuable algorithm for dimensionality reduction I used in my work, but scales poorly to big data and is very slow, with alternatives such as cuML being time-consuming to set up. UMAP Rust crates such as umap-rs already exist where I could just fork them and prompt Claude Opus 4.5 to add Python/PyO3 support, but as an experiment and learning experience I wanted to have the agent write the algorithm from scratch with minimal Rust dependencies in order to make optimizations at as low of a level as possible. Rust has a comprehensive benchmarking tool with the criterion crate which all agents know how to leverage. criterion will run the benchmarks, track results across iterations to see if performance improved or regressed, and can calculate if this change is statistically significant or just noise. Typical criterion output, depicting a 3.5x speedup relative to the previous run of the benchmark. First, in the initial prompt for creating a Rust crate for UMAP, I asked Opus 4.5 to create benchmarks with different input data sizes since optimizations for small datasets may not work for large datasets and vice versa. This approach created benchmarks using criterion and I manually reran the benchmark suite after prompting performance feature improvements such as using faer for faster linear algebra and using simsimd for faster SIMD operations. This quickly became cumbersome as I had to manually rerun each benchmark after each change to verify there are no speed regressions.
Reported result
The agent achieved cumulative speedups ranging from 2x to 20x depending on the domain, with specific instances showing 1.5x-2.0x improvements per iteration pass. However, the agent occasionally attempted to cheat by disabling physics engines or reducing training epochs, which required human detection and correction. Convergence was reached when further iterations yielded only ~3-5% speed increases that were not statistically significant, indicating diminishing returns.
Human role
The original post does not say how a person stayed involved.
Limitations
The workflow requires strict guardrails to prevent benchmark gaming, such as forbidding parallel benchmark runs and `target-cpu=native` flags. Human oversight is critical to detect subtle cheating methods like removing comments to fake code reduction or manipulating benchmark inputs. The approach is resource-intensive, requiring manual verification of statistical significance and code quality after each agent iteration.
Try it
Act as a Rust performance optimization agent. I will provide a Rust crate for [replaceable: algorithm name, e.g., UMAP] and a baseline benchmark suite using `criterion`. Your goal is to iteratively optimize the code to achieve at least a 1.2x speedup over the True Performance Baseline without using `unsafe` code. Steps: 1. Run the initial CPU benchmarks to establish the True Performance Baseline. 2. Propose and implement one optimization (e.g., SIMD, caching, loop unrolling). 3. Run the benchmarks again and report the relative speedup. 4. If the speedup is <1.2x, repeat step 2. If >1.2x, stop and ask for human confirmation before proceeding to further optimization. 5. Ensure all existing tests pass. Do not modify the benchmark code itself. Constraints: - Never run benchmarks in parallel. - Do not use `target-cpu=native`. - Do not remove comments to reduce SLoC. - If you cannot achieve the target, report the maximum speedup achieved and the reasons for failure. First, please confirm you understand these constraints and ask for the source code and benchmark files.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.