Writing code used to be the slow part of building software. Now an AI agent can turn out a pull request in minutes, and the slow part has quietly moved somewhere else: deciding whether that code should ship.
CodeRabbit’s recent announcement shows this well. The company says it now ships about 20 times more software than it did six months ago, with the same engineering team. It credits OpenAI models for governing and trusting AI-generated code, and Codex for building and shipping it. The headline is about output, but the more interesting story is what had to change around it.
CodeRabbit ships 20x more
CodeRabbit builds AI code review and governance tools. It says it has multiplied its output twentyfold in half a year without adding engineers, thanks to an agent-driven workflow built on OpenAI models and Codex.
The detail that stuck with me is the volume. CEO Harjot Gill estimates he personally pushed somewhere between 2,500 and 3,000 pull requests in the 30 days before the announcement. That isn’t a modest productivity bump. It’s a different kind of work.
At that rate, a PR is probably not one person carefully crafting a change. It’s one output from a fleet of agents handling features, bug fixes, tests, docs, security patches and infrastructure all at once. Someone still has to decide which of those changes actually reach production, and fast.
More code, less certainty
For decades, software teams ran on a simple rhythm. A developer wrote a change, opened a PR, waited for a colleague to review it, ran the tests, talked it through, and merged.
That rhythm breaks when one engineer can generate thousands of PRs. Human reviewers can’t keep up. Gill calls code review “the first bottleneck” teams hit after adopting coding agents, and it makes sense: the agents speed up one end of the pipeline while the other end stays the same size.
| Traditional development | Agent-driven development |
|---|---|
| Developers write most of the code | Agents generate a large share of changes |
| Review capacity grows with headcount | Review demand can outgrow the team |
| A PR reflects a person’s deliberate intent | A PR may reflect an agent’s reading of the intent |
| Quality checks are manual and sequential | Quality checks must be automated, parallel and auditable |
| The constraint is implementation | The constraint is verification and judgment |
Here’s the paradox. AI makes code cheap, but it makes trust, context, validation and accountability expensive.
Writing code to deciding what ships
Engineers aren’t just writing less code. The most valuable part of their job is shifting from producing diffs to making decisions about them.
That shift happens at three points:
- Before the work: defining the outcome, constraints, acceptance criteria, security boundaries and how much risk is acceptable.
- During the work: setting up automated checks like tests, static analysis, security scans, dependency audits, policy enforcement and agent-based review.
- After the work: making the call. Merge it, ask for changes, split it up, roll it back, or hand it to a human owner.
CodeRabbit says its platform helps teams validate, prioritize, explain, secure and verify agent-written changes at scale. You can think of that as a change-management layer, something that turns a flood of machine-written changes into decisions a team can act on with confidence.
Why PR count is a misleading metric
“3,000 pull requests a month” sounds impressive, but it measures activity, not results. It tells you the agents are busy. It doesn’t tell you customers are better off.
A better scorecard separates generating code from delivering value:
| Metric | What it tells you |
|---|---|
| PRs generated | How much the agents are producing. Easy to inflate. |
| PRs merged | How much was accepted. Shows throughput, not quality. |
| Change-failure rate | How often releases cause incidents or rollbacks. |
| Review latency | How long a PR waits for a decision. Shows if governance is the new bottleneck. |
| Human intervention rate | How often changes need rework or escalation. |
| Security findings per change | Whether review depth is keeping pace with volume. |
| Time to customer value | How long it takes to go from idea to real impact. The one that matters most. |
One caveat is worth stating plainly. The 20x figure is a company-reported case study. OpenAI’s write-up doesn’t define exactly what “software shipped” means, what the baseline was, what the defect rate looked like, or whether anyone independent checked the numbers. That doesn’t make the story useless. It just means it’s a signal about workflow design, not a guaranteed result you can expect to copy.
Verification layer
If agents are the factory floor, the verification layer is quality control, safety inspection and release sign-off rolled into one.
Before a human ever opens a diff, a good governance system should already have answered a few questions:
- What changed? A plain-language summary of the intent, the affected parts and the user-visible behavior.
- Does it work? Tests run, failures reproduced, behavior checked against the requirement.
- Is it safe? Checks for vulnerabilities, risky dependency changes, exposed credentials and dangerous infrastructure edits.
- Can someone understand it? The reasoning, the alternatives considered and the likely side effects.
- How risky is it? A rating based on blast radius, production exposure, data access and how easily it can be undone.
- Who needs to sign off? Low-risk changes get approved automatically, and human attention goes to the consequential ones.
CodeRabbit uses several specialized subagents for review and explanation, and saves its more capable models for planning and orchestration. That points to where things are heading: no single model or agent can safely own the whole life of an autonomous change.
What CodeRabbit’s workflow reveals
CodeRabbit didn’t just plug in Codex and call it done. The company says the jump in volume exposed strain everywhere: local compute, testing, build and packaging tools, dependencies and approval flows. It had to rethink the whole delivery process.
Three practices stand out:
- Approval based on risk. Low-risk changes ship automatically without human involvement, while higher-risk ones still get a person’s sign-off.
- Agents running in the cloud. Moving development work off local machines removes hardware limits and gives teams tighter control over access and auditing.
- Agents outside the editor. The company built a Codex-powered Slack bot that triages incidents, messages and support tickets, then hands back a PR for humans to review and merge.
This is what agentic development looks like in practice. The agents aren’t autocomplete inside an IDE. They take part in incident response, fixes, testing, documentation and release decisions.
The new engineering operating model
The teams that come out ahead won’t be the ones generating the most code. They’ll be the ones who can turn agent output into production value safely.
Think of it as three things multiplied together: how much the agents can generate, how good your verification is, and how fast you can decide. If any one of them is weak, the total suffers. A team with huge generation capacity still ships slowly if every change needs a senior engineer to inspect it by hand. A team with strong automated verification can hand off routine work and keep its scarce human judgment for architecture, trade-offs, security and customer impact.
The question has changed. It used to be “Can we build this?” Now it’s “Do we have enough confidence to ship this?”
What leaders should do next
- Measure decisions, not diffs. Track review latency, change-failure rate, rollbacks, escaped defects and time to value, not just PR volume.
- Grant autonomy in stages. Let agents handle reversible, low-risk changes on their own. Require human approval for anything touching production, security, sensitive data or a wide blast radius.
- Make evidence machine-readable. Attach test results, security scans, impact analysis, provenance and an explanation to every proposed change.
- Treat review as a product. Put real investment into review agents, policies, dashboards and audit trails, the same way you do for coding agents.
- Rebuild the release pipeline. If agents can create 20x more changes, your CI/CD, test environments, dependency management and incident response have to scale with them.
- Keep humans accountable. Automation can filter and recommend, but someone must clearly own customer impact, security posture and production risk.
Also Read: Why Niche ERP Platforms Are Baking Native AI Agents Directly Into the Engine
The code generation paradox is that AI made the easy part of software, producing plausible code, nearly free. That made the hard part, knowing what’s safe, correct and worth shipping, matter more than ever.
CodeRabbit’s story isn’t proof that engineering work is disappearing. It’s evidence that the work is changing shape. The advantage will go to teams that build a trustworthy decision layer between endless agent output and the production systems their customers rely on.