Martin Spier has spent more than 16 years in performance engineering, roughly nine of them at Netflix, and he has worked at Snowflake, Expedia, Parasail, and PicPay. Now he leads ChatGPT Performance at OpenAI, where the problem is not just keeping a product fast, but keeping it fast while the entire company ships code at a pace no human team could match. Speaking at QCon AI, a practitioner-led conference on scaling AI workloads, Spier laid out the twin accelerations reshaping his work: user growth that went from zero to hundreds of millions in months, and an internal development velocity boosted by AI agents that review every pull request.
His talk, titled "Keeping ChatGPT Fast as AI Development Accelerates," ran 50 minutes and 14 seconds, and it is now available on InfoQ with video and transcript. The core message is simple. Performance engineering at OpenAI cannot rely on the old playbook of slow, careful, human-driven optimization loops. The company needs its own agents to keep up with the agents writing the code.
From Research Preview to 900 Million Users
ChatGPT launched as a research preview in late 2022. It reached 1 million users in 5 days from launch, a pace that stunned even the team building it. By February 2025, the product had reached 900 million weekly active users, which is roughly 11% of the human population. Spier repeated that figure slowly, as if to let the scale sink in. Nine hundred million people, every week, opening a chat interface and expecting a response in seconds.
The growth did not stop with text. When image generation spiked, the system handled 700 million images generated in the first 7 days by 130 million users. That is not a slow ramp. That is a flash flood of inference requests, storage writes, and content delivery, all hitting the same infrastructure that serves chat traffic. Spier noted that ChatGPT is used globally, which adds complexity to infrastructure, and that the product has evolved with voice, agents, image generation, and other features that each place different demands on the stack.
He argued that companies are reaching their first million, first 10 million, and first 100 million users faster than before. The old assumption, that a startup has months to scale its architecture, no longer holds. ChatGPT compressed that timeline into days. The performance team had to build for a world where the user base could double before a single optimization shipped.
The scale of the operation is hard to overstate. A product that serves hundreds of millions of people weekly cannot afford to go down for maintenance, cannot afford to have a slow database query during peak hours, and cannot afford to have a serialization bug that adds 100 milliseconds to every response. The performance team at OpenAI is not optimizing for a single endpoint or a single feature. They are optimizing for a platform that handles text, voice, images, and agentic tasks, all at once, all over the world.
Spier walked through the implications of global usage. A user in Singapore has a different network path than a user in São Paulo. A user on a mobile device has a different client experience than a user on a desktop browser. The infrastructure has to account for all of these variations, and the performance team has to measure them all. You cannot just optimize for the median user in the United States. You have to optimize for the tail, the slowest network, the oldest device, the most complex conversation.
He also touched on the product evolution. ChatGPT started as a text chat interface. Now it has voice, agents, image generation, and other features. Each of these features places different demands on the stack. Voice requires low-latency streaming. Agents require long-running task execution. Image generation requires massive GPU allocation. The performance team has to understand all of these workloads and build tooling that can handle them all.
The user growth is not just a number. It is a continuous pressure on every part of the system. Every new user means more conversations, more stored history, more files, more context to fetch. The system has to scale horizontally, but it also has to scale in complexity. A user with a long conversation history is a different load than a user with a single question. The performance team has to model all of these scenarios.
Spier's point about the compressed timeline is worth emphasizing. In the past, a startup might have had months to scale its architecture before hitting a million users. ChatGPT hit that mark in 5 days. The team did not have time to redesign the system. They had to make it work, and then make it fast, and then make it faster, all while the user base was exploding. That is the reality of performance engineering at OpenAI.
Codex Accelerates Shipping, and the Side Effects
The second acceleration is internal. Codex, OpenAI's AI coding agent, launched late last year, in 2024, and it changed how the company writes software. Almost every engineer at OpenAI uses Codex weekly or daily. Every PR at OpenAI is automatically reviewed by Codex. The result, as of October 2025, is that PR volume per engineer per week increased by 70%. That is a massive jump in code change velocity, and it comes with hidden costs.
Spier explained that developers now work on 7 to 10 things simultaneously with agents. A human might juggle two or three branches. An engineer with Codex can spin up a dozen parallel efforts, each producing pull requests, each touching shared code paths. The parallelism is powerful, but it also means that small code changes compound to cause performance issues over time. A single PR might add a few milliseconds of latency. Ten PRs, merged in a day, might add fifty. Nobody notices the individual changes. The aggregate is where the trouble lives.
He referenced DX, a research and benchmark organization, which reports that smaller tech companies see roughly 5 PRs per engineer per week at the p90 level. OpenAI is now far beyond that, and the gap is growing. The performance team cannot review every change manually. They cannot even profile every new feature before it ships. The volume is simply too high.
The implications of this velocity are profound. When a human writes code, they tend to think about the performance implications of their changes. They know that adding a database query to a hot path is a bad idea. They know that serializing a large object on every request is wasteful. An AI agent, on the other hand, is optimized for correctness and speed of delivery. It will happily add a query, a serialization step, or a network call if it makes the feature work. The performance team has to catch these issues after the fact, or better yet, before they merge.
Spier described the hidden costs of this shipping speed. The 70% increase in PR volume is a headline number, but the real story is what happens after those PRs merge. Each change is small. Each change is reviewed by Codex. Each change passes tests. But the cumulative effect on performance is real, and it is easy to miss. A new feature adds a database query. Another feature adds a serialization step. A third adds a network call. Individually, none of these is a problem. Together, they can turn a fast endpoint into a slow one, and the regression might not be caught until users complain.
The comparison to smaller tech companies is instructive. DX reports that smaller companies see about 5 PRs per engineer per week at the p90 level. OpenAI is far beyond that. The gap is not just a matter of scale. It is a matter of approach. Smaller companies can still rely on manual code review and manual performance testing. OpenAI cannot. The volume is too high, and the speed is too fast. The performance team has to automate everything.
Spier also noted that the developers themselves are not the bottleneck. They are using Codex to be more productive, and that is a good thing. The problem is that the performance engineering practices have not kept up. The tools that make developers faster also make the codebase more complex. The performance team has to build tools that are just as fast and just as autonomous as the development tools.
The 7 to 10 simultaneous workstreams per developer is a staggering number. A human developer might be able to keep track of two or three branches in their head. With Codex, they can have a dozen parallel efforts, each with its own pull requests, each touching shared code paths. The coordination problem is immense. The performance team has to understand all of these changes, even the ones that are not yet merged, because they will all land eventually.
The aggregate effect is what matters. A single PR might add a few milliseconds of latency. Ten PRs, merged in a day, might add fifty. Over a week, that could be hundreds of milliseconds. Over a month, seconds. The performance team cannot wait for the aggregate to become a problem. They have to catch the individual changes that are likely to compound.
Performance Engineering's Two Loops
Spier described performance engineering as a set of loops. The reactive loop is the classic one: detect a problem, profile it, root cause it, fix it. This loop handles slowness, errors, and downtime, and it is driven by user impact. The active loop is newer: agents look for optimizations before problems occur, scanning code and infrastructure for inefficiencies that have not yet caused visible damage.
Both loops matter, but the reactive loop is where the pain shows up first. Performance issues manifest as slowness, errors, and downtime, and they affect user retention and acquisition. Spier said that this relationship was modeled at both Netflix and OpenAI. A slow response is not just a technical annoyance. It is a lost user, a bad review, a competitor's advantage.
The active loop is where AI agents become essential. Instead of waiting for a latency spike, agents can continuously profile code paths, detect regressions, and suggest optimizations. Spier's team is building exactly this. They are not trying to slow down engineers with roadblocks. Instead, they speed up performance engineering loops using AI agents. The goal is to make the performance team as fast as the development team, which means automating the detection, profiling, and even the fixing of performance issues.
The reactive loop is well understood. Detect a problem, profile it, root cause it, fix it. This is the classic performance engineering workflow, and it has been used for decades. But at OpenAI's scale, it has to be automated. Detection cannot rely on users complaining. Profiling cannot be a manual process. Root cause analysis cannot take days. The entire loop has to be accelerated, and AI agents are the tool.
Spier explained that the reactive loop is driven by user impact. When a user experiences slowness, errors, or downtime, that is a signal. The performance team has to respond quickly, find the root cause, and fix it. But at OpenAI's scale, the signals are overwhelming. There are millions of requests per second, and any one of them could be slow. The team cannot manually inspect every request. They need automated detection, automated profiling, and automated root cause analysis.
The active loop is the newer concept. Instead of waiting for problems to occur, agents look for optimizations before problems happen. They scan the codebase for hot paths, for inefficient queries, for unnecessary serialization. They profile the infrastructure for bottlenecks, for underutilized resources, for potential failures. They suggest changes before the latency spikes, before the errors occur, before the users complain.
Spier's team is building both loops. The reactive loop is being automated with AI agents that can detect, profile, and root cause issues faster than any human. The active loop is being built with agents that continuously scan the codebase and infrastructure for optimization opportunities. The goal is to make the performance team as fast as the development team, which means automating everything.
The relationship between performance and user retention is not theoretical. Spier said that this relationship was modeled at both Netflix and OpenAI. A slow response leads to a lost user. A bad review leads to fewer acquisitions. A competitor's advantage grows with every millisecond of latency. The performance team is not just a cost center. They are a revenue protection team.
The two loops are complementary. The reactive loop catches problems that have already occurred. The active loop prevents problems from occurring in the first place. Both are essential, and both need to be automated. Spier's talk made it clear that the old way of doing performance engineering, with humans manually reviewing code and manually profiling systems, is no longer viable at OpenAI's scale.
Inference Is Only Part of the Request Path
A common misconception, Spier said, is that performance work on an AI product is all about inference. The GPU is the bottleneck, the thinking goes, so optimize the model and everything else will follow. He pushed back on that. Inference is only part of the request path. Other components include client work, networking, data fetching, serialization, tokenization, and streaming. Each of these can add latency, and each can fail.
He walked through what happens when a user sends a chat request. The system must fetch conversation history, files, and project context. It must perform tokenization, truncation, and compaction before sending anything to the inference engine. Conversations can be megabytes to hundreds of megabytes long. A user with a long history, multiple attached files, and a complex project context is not just sending a prompt. They are sending a payload that must be assembled, processed, and prepared, all before the model even sees it.
This is where the latency breakdown chart comes in. Spier's team uses what he called a layer cake chart, a visual breakdown of where time is spent across the request path. The inference layer is one slice. The data fetching layer is another. The serialization layer is another. Each slice can be optimized, and each slice can become a bottleneck.
Non-GPU resources are equally important. CPU, memory, I/O, database, and blob storage all play critical roles. A database query that takes 200 milliseconds can dominate a request that would otherwise take 50. A blob storage read that is slow can stall the entire pipeline. Spier's point is that performance engineering at OpenAI is not just GPU optimization. It is full-stack performance work, and the stack is enormous.
The request path is more complex than most people realize. When a user sends a chat request, the system has to fetch the conversation history, which could be megabytes long. It has to fetch any files that are attached, which could be even larger. It has to fetch the project context, which includes the user's settings, preferences, and any relevant data. All of this has to happen before the inference engine even sees the prompt.
Then there is the processing. The system has to tokenize the input, which means breaking it down into tokens that the model can understand. It has to truncate the input if it is too long, and it has to compact the input to fit within the model's context window. All of this takes time, and all of it can be optimized.
Stay ahead of the AI curve
The most important updates, news, and content — delivered weekly.
No spam. Unsubscribe anytime.
The layer cake chart is a visual tool that helps the team understand where time is spent. Each layer of the cake represents a component of the request path. The inference layer is one slice, but so is the data fetching layer, the serialization layer, the tokenization layer, and the streaming layer. By breaking down the latency into these slices, the team can identify which layer is the bottleneck and focus their optimization efforts there.
Non-GPU resources are often overlooked. Everyone focuses on the GPU, but the CPU, memory, I/O, database, and blob storage are all critical. A database query that takes 200 milliseconds can dominate a request that would otherwise take 50. A blob storage read that is slow can stall the entire pipeline. The performance team has to optimize all of these resources, not just the GPU.
Spier's point is that performance engineering at OpenAI is full-stack work. The stack is enormous, and every layer can be a bottleneck. The team has to understand all of it, measure all of it, and optimize all of it. That is a massive undertaking, and it is only possible with automation.
Different Users, Different Expectations
Not all requests are equal, and Spier said that users have different expectations based on intent. A simple question, like "what is the capital of France," should get a fast, short answer. An agentic task, like "plan my week and book meetings," is expected to take longer. The performance team has to account for these different expectations, and they cannot apply a single latency budget to every request.
This complicates the work. A user who is waiting for an agent to complete a multi-step task will tolerate a few seconds. A user who just asked a factual question will not. The system needs to route requests appropriately, set expectations, and allocate resources based on the type of work being done. Spier did not claim this is solved. He presented it as an ongoing challenge, one that grows as the product adds more agentic features.
He also mentioned that "AI slop" is a topic of discussion, though he did not dwell on it. The phrase refers to low-quality, mass-produced AI content, and it is relevant to performance because generating more content, even low-quality content, consumes resources. The image generation spike is a perfect example. Seven hundred million images in a week is a massive load, and not all of those images are masterpieces.
The different expectations are a real challenge. A user who asks a simple factual question expects a response in under a second. A user who asks an agent to plan their week and book meetings expects the task to take several seconds, or even minutes. The system has to handle both types of requests, and it has to allocate resources accordingly.
Routing is a key part of this. The system needs to identify the type of request and route it to the appropriate infrastructure. A simple question can be handled by a fast, lightweight model. An agentic task needs a more powerful model and more time. The routing logic has to be fast and accurate, and it has to be continuously updated as the product evolves.
Setting expectations is also important. If a user knows that an agentic task will take a few seconds, they are less likely to be frustrated by the wait. The system can show a progress indicator, or it can provide intermediate updates. This is a product design challenge as much as a performance challenge, and the performance team has to work with the product team to get it right.
The "AI slop" discussion is relevant because it highlights the resource consumption of content generation. Generating 700 million images in a week is a massive load, and not all of those images are high quality. The performance team has to handle the load regardless of quality, and they have to do it without degrading the experience for other users.
Spier did not claim that the different expectations problem is solved. He presented it as an ongoing challenge, one that grows as the product adds more agentic features. The performance team is working on it, but it is not a solved problem. It is a continuous process of measurement, optimization, and adjustment.
Speeding Up the Performance Loops
OpenAI does not want to slow down engineers with roadblocks. That is a key design principle, and Spier stated it directly. The company wants to ship fast, and it wants to ship a lot. The performance team's job is not to gate releases or demand manual reviews. It is to keep the product fast while the codebase evolves at machine speed.
The solution is to speed up performance engineering loops using AI agents. This means agents that detect regressions, agents that profile code, agents that suggest fixes, and agents that even implement fixes. The performance team becomes a team of humans supervising a fleet of automated performance engineers.
Spier's analysis of the impact of agentic coding on performance engineering includes a clear warning about hidden costs. The 70% increase in PR volume is a headline number, but the real story is what happens after those PRs merge. Each change is small. Each change is reviewed by Codex. Each change passes tests. But the cumulative effect on performance is real, and it is easy to miss.
He argued that small code changes compound to cause performance issues over time. A new feature adds a database query. Another feature adds a serialization step. A third adds a network call. Individually, none of these is a problem. Together, they can turn a fast endpoint into a slow one, and the regression might not be caught until users complain.
This is why the active loop matters. Agents can look for optimizations before problems occur. They can profile the codebase continuously, identify hot paths, and suggest changes before the latency spikes. Spier's team is building this capability, and it is a direct response to the velocity problem. You cannot manually review every PR, but you can have an agent review every PR for performance impact.
The talk also covered the reactive loop in detail. Detect, profile, root cause, fix. This loop is well understood, but at OpenAI's scale, it has to be automated. Detection cannot rely on users complaining. Profiling cannot be a manual process. Root cause analysis cannot take days. The entire loop has to be accelerated, and AI agents are the tool.
The hidden costs are not just about latency. They are also about complexity. Every new feature adds code, and every line of code is a potential source of bugs, security vulnerabilities, and performance issues. The codebase is growing at an unprecedented rate, and the performance team has to keep up.
The compounding effect is the most dangerous part. A single PR might add a few milliseconds of latency, and that is not a problem. But ten PRs, merged in a day, might add fifty milliseconds. Over a week, that could be hundreds of milliseconds. Over a month, seconds. The performance team cannot wait for the aggregate to become a problem. They have to catch the individual changes that are likely to compound.
This is where the active loop comes in. Agents can scan the codebase for patterns that are likely to cause performance issues. They can identify hot paths, inefficient queries, and unnecessary serialization. They can suggest changes before the latency spikes. The performance team is building this capability, and it is a direct response to the velocity problem.
The reactive loop is also being automated. Detection can be done by agents that monitor latency, error rates, and other metrics. Profiling can be done by agents that automatically capture stack traces and performance data. Root cause analysis can be done by agents that correlate changes with performance regressions. The entire loop is being accelerated, and AI agents are the tool.
Spier's team is not trying to slow down engineers with roadblocks. That is a key design principle, and he stated it directly. The company wants to ship fast, and it wants to ship a lot. The performance team's job is not to gate releases or demand manual reviews. It is to keep the product fast while the codebase evolves at machine speed.
Spier's background makes him well suited to this challenge. He spent roughly nine years at Netflix, where performance engineering is a core discipline. He worked at Snowflake, Expedia, Parasail, and PicPay, and he led engineering at Parasail, an AI inference-as-a-service company. At PicPay, a Brazilian fintech, he led the infrastructure platform and developer experience. This mix of streaming, cloud data, travel, AI inference, and fintech gives him a broad view of where performance problems live.
At QCon AI, he brought that experience to bear on the specific question of how to keep ChatGPT fast as AI development accelerates. The answer, in his telling, is not to slow down. It is to build performance tooling that is as fast and as autonomous as the development tooling that is causing the problem.
The design principle is clear. The performance team is not a gatekeeper. They are not trying to slow down engineers or demand manual reviews. They are trying to keep the product fast while the codebase evolves at machine speed. That means they have to be as fast as the development team, and the only way to do that is with AI agents.
The agents are not just for detection. They are for profiling, for root cause analysis, and for fixing. An agent can detect a regression, profile the code, identify the root cause, and suggest a fix. It can even implement the fix and submit a PR. The human performance engineer reviews the work and approves it. The loop is much faster than a human-only process.
Spier's background is relevant here. At Netflix, performance engineering is a core discipline, and he learned how to build systems that are fast and reliable at scale. At Snowflake, he worked on cloud data infrastructure. At Expedia, he worked on travel platforms. At Parasail, he led an AI inference company. At PicPay, he led infrastructure and developer experience. This mix of experiences gives him a broad view of where performance problems live and how to solve them.
The talk at QCon AI was a practitioner-led presentation, and the audience was full of engineers who are building similar systems. The talk's availability on InfoQ, with video and transcript, means it will reach a wide audience. The message is clear: performance engineering must evolve, or it will become the bottleneck that slows down the very acceleration it is meant to support.
Spier did not offer a simple formula. He did not claim that OpenAI has solved the problem. He presented the challenges, the approaches, and the ongoing work. The reactive loop and the active loop are both essential. Inference is only part of the request path. Non-GPU resources matter. Users have different expectations. Small changes compound. And the only way to keep up is to use AI agents to speed up the performance engineering loops themselves.
The talk is a snapshot of a moment in time, a moment when a product with 900 million weekly active users is being built by engineers who ship 70% more code per week than they did a year ago, with AI agents reviewing every pull request. Keeping that product fast is not a side project. It is the core challenge of the next phase of AI development.
The industry is watching. QCon AI is a practitioner-led conference, and the audience is full of engineers who are building similar systems. The talk's availability on InfoQ, with video and transcript, means it will reach a wide audience. The message is clear: performance engineering must evolve, or it will become the bottleneck that slows down the very acceleration it is meant to support.
Spier's talk is not just about OpenAI. It is about what happens when any company adopts agentic development at scale. The tools that make developers faster also make the codebase more complex, and the performance engineering practices that worked for human-paced development do not work for AI-paced development. The industry is watching, and the lessons from OpenAI will be applied elsewhere.
The figures tell the story. ChatGPT reached 1 million users in 5 days. It reached 900 million weekly active users by February 2025, which is 11% of the human population. The image generation spike produced 700 million images in the first 7 days, from 130 million users. Codex launched in 2024, and by October 2025, PR volume per engineer per week was up 70%. Smaller tech companies, per DX, see about 5 PRs per engineer per week at the p90 level. OpenAI is far beyond that.
Spier has more than 16 years in performance engineering, with roughly 9 years at Netflix. He has worked at Snowflake, Expedia, Parasail, and PicPay. He leads ChatGPT Performance at OpenAI. The talk was given at QCon AI, and it is available on InfoQ with video and transcript.
The timeline is compressed. Late 2022, research preview. Five days later, one million users. February 2025, 900 million weekly active users. 2024, image generation spike and Codex launch. October 2025, the 70% PR volume increase measured. Each milestone compounds the performance challenge.