If AI Accelerates Your Software Factory, Where’s the ROI?

We’re hearing a lot lately about the “Software Factory,” an iterative process for turning ideas into shipped changes: products, features, bug fixes, and optimizations. It’s a reductive but useful framing for how software gets built. And now the factory is getting robotic arms. Coding agents let teams ship more software, faster, without necessarily sacrificing quality.

I’ve felt the benefits firsthand while engineering at Firetiger, in the weeds helping companies apply AI to level up their production software and operations. Coding agents automate tedious work and allow us to ship more code than we ever imagined was possible. They’re probably the largest and most valuable application of “AI” discovered so far (besides ads…).

But talk to executives at software companies paying for coding agents rather than selling into the AI boom, and the story is murkier. Product and engineering productivity is accelerating, token spend doubly so, while most other business indicators show only marginal improvement, if any! Are our glorious Software Factories, despite producing incredible amounts of code, actually producing more “business value”? Where is the ROI?

Here’s an uncomfortable theory: perhaps software production was never the binding constraint on most companies’ success, even before product and engineering teams gained these new capabilities.

“No shit,” some of you muttered. But believe me, from what I’m seeing on the factory floor, this is not as obvious as it sounds! Those of us working inside the Software Factory typically forget to step outside, smell the flowers, chat with the truck drivers, and consider how the rest of the business works. From the production line, more software can look a lot like more business value. But if we’re going to use a manufacturing analogy, we might as well consider some of the fundamental ideas from industrial engineering and operations management. In the physical world, a factory is just the means of production. Its output still has to be distributed, marketed, sold, and used by real customers with limited attention. The Software Factory is not exempt.

Companies pay for coding agents to implement changes faster. They are also trying new AI-enabled software development tools to discover, validate, and safely ship changes faster. But none of this necessarily increase a company’s capacity to metabolize change as fast as they increase software production. The company must still decide which changes are worthwhile, get employees and customers to adopt them, understand whether they helped, and turn the added capacity into company progress.

And damn it, change is hard. One big reason is that every change requires the company to make tradeoffs, often without a clear objective function. A code change may improve performance while increasing cost. It may increase engagement without improving customer satisfaction. It may generate revenue while creating complexity that makes the next change harder. A/B experiments help only when we can isolate a change and measure its outcome. And sometimes the objective function isn’t even financial! A company may decide to build a rickety homegrown Slack agent not because it will produce obvious returns, but because doing so yields capability, promotions, ownership, and camaraderie among the team.

Being good at making these tradeoffs is a big part of what people mean by having “taste.” As product strategy gets cheaper to execute, deciding what is worth doing matters more. But taste is not enough, because you can’t make every great change at once.

Jeff Wilke made this point to Jeff Bezos at Amazon. Bezos alone, Wilke told him, had enough good ideas to destroy Amazon if he released them faster than the organization could absorb them. Every new idea created work-in-process, backlog, and distraction. The insight was to rate-limit the work while developing Amazon into an organization that could metabolize more.

Every organization has a saturation point: a limit on how much change it can process at once. This is the logic behind the Theory of Constraints, and it’s interesting to apply it to our messy companies. A business is obviously more complicated than a serial production line. It has many interconnected processes, and what counts as the bottleneck depends on what you’re trying to accomplish. But it’s still a system, and making one part move faster will not make the whole process move faster if the bottleneck is elsewhere. You might just cause a pile up of inventory somewhere down the line.

And to be clear, we’re not talking about the heaping inventory of unreviewed agent pull requests. We’re talking about how an engineer can create a bug fix before anyone has decided whether the bug even matters. A product team can ship a feature before sales can sell the last one, support can explain it, customers can adopt it, or the market can respond to it. When the Software Factory is not the bottleneck, its output can become backlog, complexity, cost, distraction, and even counterproductive work.

This dynamic explains how billions of dollars of token spend can create obvious value for individual users without producing equally obvious ROI for their companies.

Of course AI can still drive enormous software ROI, and to say “code was never the hard part” is to dismiss the skill and effort required to write good software. That work can be difficult and valuable and still not set the pace of the whole system. Where production is the constraint, coding agents raise throughput. Where it isn’t, they lower the cost of producing “enough”: a smaller team does the same work, the systems get optimized, and the AWS bill shrinks. Cheap code makes entirely new categories and companies economical, and some workflows really can be automated end to end. If any of this eliminates a true bottleneck, great! The system is then bounded by the next one.

I wonder if many of today’s software company builders are destined to relearn some principles of lean manufacturing. Not that product teams should build only what customers explicitly request, or that software development should run at the tick of a Toyota assembly line. Just the basic idea that developing software and products is but one part of building a successful business, and for most, it’s not the bottleneck.