On October 27, Katelyn Lesse and Angela Jiang from Anthropic talked on the Latent Space podcast about how the Claude Platform is built and scaled. One thread is especially useful for companies running an older system and thinking about modernising it. Anthropic rewrote the Messages API, the main API for Claude models, from Python to Rust. The team did not have a single Rust developer, and Claude wrote most of the code. Source: Latent Space, How Anthropic ships the #1 LLM in the World safely (segments “CPU constraints and scaling” and “Rebuilding the Messages API in Rust”).

What happened

According to the conversation:

  • Starting point. The Messages API was a Python monolith. As traffic grew fast, CPU became the bottleneck, and the Python monolith used it inefficiently.
  • The decision. The team chose to move the whole API to Rust, even though, as they put it, there was not “a single Rust developer in the room”. Along the way they hired a couple of people with Rust experience as a sanity check.
  • Division of work. Humans designed the system, which is at heart a distributed systems architecture. Claude wrote the code and helped the engineers learn what they needed to design it.
  • Verification. The team built a complete fork of the API in Rust, ran production shadow traffic through it and verified that the Python and Rust versions returned the same outputs. The new version was turned on gradually, behind feature flags.
  • Scope. It was a pure port. The team deliberately resisted adding features or changing how the system behaved. A small group ran the migration while the rest of the team kept building features into the API, and static analysis flagged changes to the old code that had not been carried over to Rust.
  • Result. At the time of recording, the Rust version was fully rolled out. Asked for a ballpark, the Anthropic guest estimated CPU savings of roughly 40–50% and a 10–20% improvement in time to first token, with bigger gains on longer-running workloads. These are estimates from the conversation, not a published benchmark.

One line from the conversation is worth remembering: the two things that mattered most were the plan and the verification. The implementation itself was Claude writing code. Previously, as the speakers noted, a project like this would have been a multi-year effort carried out piece by piece. This time the whole system was ported in one go and then turned on slowly.

What this means for companies

The points below are our interpretation, not Anthropic’s position.

The cost of writing code is falling; the cost of proving it behaves the same is not. The bottleneck moves from implementation to specifying how the old system behaves and how to check that the new one behaves identically. Without that, faster code generation just means faster risk generation.

Trust comes from comparison on real traffic, not from code review. With thousands of AI-generated files, line-by-line review does not scale. Comparing the responses of both systems to the same requests is evidence you can measure.

Anthropic’s case comes with conditions not everyone has. Huge production traffic to compare against, an experienced distributed systems team and an API with clearly defined inputs and outputs. If your system does not meet those conditions, adapt the approach rather than copy it.

Checklist before an AI-assisted migration

  1. A measurable goal. Why migrate: infrastructure cost, latency, no people left for the old stack, security? Measure the baseline before you start.
  2. A frozen scope. A port, not a rebuild. New features and architecture changes come after cutover. This is the rule most often broken in rewrites.
  3. A definition of equivalence. Write down what “behaves the same” means: response bodies, error codes, headers, ordering, side effects (database writes, queues, emails), and which differences are acceptable.
  4. Tests on the old system. Before the first line of new code, capture current behaviour in contract and characterisation tests, including the quirks customers rely on.
  5. Shadow traffic. A copy of traffic goes to the new version while users still get the response from the old one. Comparison is automated and every mismatch is investigated. Watch out for writes: the shadow version must not send an email twice or book a payment twice. Also check that copying traffic containing personal data complies with GDPR and your contracts.
  6. Gradual cutover. Feature flags, a percentage of traffic, selected endpoints or customers. Written exit criteria for every stage.
  7. A tested rollback. Going back to the old version should be one proven step. Keep the old system on standby until the new one has gone through a full cycle, such as month-end billing.
  8. Keeping up with ongoing development. If the old system keeps changing, you need a mechanism that catches changes missing from the new version, such as static analysis or a CI rule.
  9. People and maintenance. Humans design the architecture and the verification. Have at least one person with experience in the target stack, and decide up front who will maintain the new code a year from now.
  10. Observability. Comparable metrics, logs and alerts on both versions, so differences in performance and errors are visible from day one.

What this example does not tell us

  • The figures are spoken estimates from a podcast. No methodology, project duration, team size or number of mismatches found during comparison has been published.
  • Porting an API with a clear contract is a favourable case. Systems whose logic is hidden in the database, reports or partner integrations are harder to compare.
  • The speakers themselves stressed that operating and evolving a system at this scale still leaves a lot of work for humans. AI does not take over responsibility for the rollout.

FAQ

Can AI rewrite our system on its own? It can write a large part of the code. The architecture, the definition of equivalence, the cutover plan and the decision to go live still have to be designed and made by people.

What if we don’t have much production traffic? You can replay recorded traffic, build reference request sets with expected responses, and keep the shadow phase running longer to cover rarer scenarios such as monthly reports.

Where should we start? With an inventory of how the old system behaves and tests that pin that behaviour down. Generating code is the last step, not the first.

Let’s talk

Considering modernising an older system with AI and want to plan it safely? Book a free consultation or write to kontakt@weban.pl. You can read more about our approach on the AI implementation page and in our DevOps and CI/CD work.