A diagram of OpenAI’s internal engineering pipeline went semi-viral on X this week. It showed ten boxes, from “software builder defines outcome” all the way to an agent that watches production graphs and files its own incident reports. The source is Gergely Orosz’s newsletter The Pragmatic Engineer, in a piece called “OpenAI’s agentic software factory”, and the diagram itself spread fast on X.
Most of the reaction skipped a detail that matters: several of those ten boxes describe OpenAI’s internal engineering setup, not the Codex product you can install today. Conflating the two is the single most common mistake in the discourse around this diagram. This article goes box by box and stops at the one stage outside teams can actually copy: CI.
The ten stages, in order
The diagram reads left to right as a loop: a human sets a goal, an agent writes code, automated gates check it, and the agent keeps iterating until those gates pass. Here is the full sequence with the internal-versus-shipped line marked for each stage.
| # | Stage | What happens | Internal only or shipped in Codex |
|---|---|---|---|
| 1 | Software builder | An engineer or PM defines the outcome | Human step, not software |
| 2 | Codex writes/edits code | Pulls context from source, docs, GitHub, Slack, Notion, internal skills, and data systems like Databricks and Datadog | Shipped Codex writes code; the internal context graph (Slack, Notion, internal data) is internal only |
| 3 | CI: build + test | A pipeline being rebuilt for agent-scale load, plus a “Perf Harness” | Shipped (your own CI); OpenAI’s specific scaling work is internal |
| 4 | Agentic code review | Parallel review from data, infra, cloud, and security specialist agents, plus risk classification | Internal only |
| 5 | Low-risk decision | Low-risk changes proceed; higher-risk changes get an additional human engineer review | Internal only |
| 6 | Agentic deploy | An agent “babysits” the change into production, including feature-flag rollout, and builds its own dashboards | Internal only |
| 7 | Production monitoring | The agent watches graphs, signals, and alerts on OpenAI’s internal observability stack | Internal only |
| 8 | Outage detected -> Sevbot | Investigates the incident, proposes mitigations, answers questions | Internal only |
| 9 | Perf Factory | Filters duplicate alerts, finds latency regressions, proposes fixes | Internal only |
| 10 | Loop back | The agent fixes issues until CI and reviews pass, then the builder gets proposed fixes | Describes the internal loop |
Exactly one stage, the CI step, plus a slice of stage 2, is something an outside team can point at and say “we have that too.” Everything from agentic review through Sevbot is OpenAI’s internal build.
That internal-only column is also a sourcing problem. The pieces OpenAI keeps behind its firewall, the orchestration, the review gate, the human sign-off, are exactly the layer most teams have nothing for. Sharkly is one provider-neutral place to build it: a builder defines the outcome as a Task, assigns it to an Agent, and the Agent runs on a Computer using whatever Runtime you already have, Claude Code or Codex. Ready for Release is the status a human has to move before anything reaches Done, so the sign-off stays a person’s job, not the pipeline’s. Sharkly does not replace Codex or write code itself; it runs the Runtime you already pay for and gives the surrounding loop somewhere to live.
Stage 1-2: a human still sets the goal, Codex still writes the code
A software builder, meaning an engineer or a product manager, defines what outcome they want. Codex then makes a series of code changes until it reaches that goal and verifies the result works. That verification loop is the part of Codex that ships today: the desktop app (Mac in February 2026, Windows in March), ChatGPT Work integration from July 2026, role-based plugins and skills, and the /goal command for tasks that run over a long stretch. If you want the mechanics of that command, we covered /goal for autonomous agent runs separately.
What is not shipped is the context graph feeding Codex internally: pretty much every OpenAI system, from Slack threads to Databricks dashboards. Orosz’s article puts it directly: OpenAI’s internal Codex “is a lot more advanced than its external counterpart because it’s plugged into pretty much every OpenAI system.” That gap between internal and external Codex is the article’s real thesis.
Stage 3: CI is where the loop is actually enforced
This is the stage worth slowing down on, because it’s the one box in the whole pipeline that any team, not just OpenAI, already owns. The article notes CI at OpenAI is being rebuilt for roughly a 10x increase in load over about six months, because agents now push far more changes through the pipeline than humans alone ever did. A “Perf Harness” runs alongside it to catch performance regressions before they reach review.
Here’s the part that gets missed: an agent that “fixes issues until CI and reviews pass” is only as trustworthy as what CI actually checks. If your test suite covers unit logic but not the API contract, an agent can loop to a green build that still ships a breaking change. Status codes, schema shape, auth behavior, and response-time budgets under load are exactly the class of check that most teams under-invest in relative to unit tests. This is the box where Apidog fits: running Apidog’s test scenarios through the Apidog CLI inside your CI step gives an agent a harder gate to satisfy than “the code compiled.” We’ve written about wiring that up in Apidog CLI inside Codex. That’s the only role Apidog plays in this pipeline. It is not the agent, and it does not touch deploy, review, or incident response.
Stage 4-5: specialist review agents and the risk decision
After CI passes, OpenAI’s internal setup routes the change through parallel review from data, infrastructure, cloud, and security specialist agents. Orosz describes it as “the equivalent of having a human domain expert from each relevant infrastructure team review every change,” which is a heavier review bar than most human teams can staff for every pull request. Codex’s shipped code review feature is a different, lighter thing than these internal specialist agents; if you’re deciding what a general-purpose agentic reviewer can do for your own stack, our roundup of AI code review tools is a fair starting point, and our companion piece on OpenAI’s agentic review and risk gate design goes deeper on this one box (sibling, confirm live before linking).
Risk classification then decides what happens next. Low-risk areas of the codebase can opt an agent into auto-approving its own PRs, cutting human sign-off out of the loop entirely for that class of change. Higher-risk changes get more AI review passes, a mandatory human review, or both. OpenAI hasn’t published the exact rule for what counts as low risk, and we’re not going to guess at it here. One idea that generalizes past OpenAI’s specific setup: a breaking change to a public API contract should never be classified as low risk, no matter how small the diff looks. Spec-first tooling that keeps your OpenAPI definition and your tests in the same place makes that distinction easier to enforce automatically, since a schema diff is a much cleaner risk signal than a line-count diff.
Stage 6-8: deploy, watch, and respond, without a human paged first
If a change clears review, an internal agent “babysits” it into production, including feature-flag rollout, and builds its own monitoring dashboards for that specific change. Once live, the same agent (or a related one) watches graphs and alerts on OpenAI’s internal observability stack. When something breaks, Sevbot takes over: it investigates the incident, proposes mitigations, and answers developer questions in Slack. It’s worth being precise about what Sevbot does not do. It proposes; it does not execute. A human still authorizes the mitigation and still holds oncall. As the article states plainly, “oncall duty is not a thing of the past.” None of stages 6 through 8 exist in the Codex product you can buy.
Stage 9-10: performance regressions and the loop back
Perf Factory sits alongside the incident path. It sifts through alerts and dashboards, filters out duplicate signals, roots out real latency regressions, and proposes fixes, which then flow back to the original builder. Together with Sevbot, this is OpenAI’s answer to alert fatigue: instead of an on-call engineer triaging every ping, an agent pre-filters and pre-diagnoses first. The loop closes with stage 10: the agent keeps revising until CI and every review layer pass.
Why the internal/external line matters for your team
If you’re evaluating whether “OpenAI-style” agentic engineering is something your team can adopt this quarter, the honest answer is: you can adopt stages 1 through 3 today, and stages 4 through 9 describe a direction, not a purchasable feature. That’s not a knock on OpenAI; internal tooling at that scale takes years. One open-source project, orchflows, is a public attempt to approximate this loop with a /software-factory command for Claude Code and Codex; its README is blunt about the goal, arguing you only need two skills instead of a library of them. It’s an early, unaffiliated project, not an OpenAI release, so treat it as a reference implementation rather than a drop-in factory.
Adoption inside OpenAI itself has moved fast in the parts that don’t need custom internal plumbing: Codex usage in non-engineering teams went from roughly 0% to 90% adoption in four months, February to May 2026. That’s a sharper signal than the pipeline diagram alone, because it says the easy part (an agent that writes code toward a stated goal) is already normal at OpenAI, while the hard part (agentic deploy, review, and incident response wired into every internal system) is still bespoke.
What stays human
The article is careful about what doesn’t change. Builders still define outcomes. Humans still approve high-risk changes and authorize incident mitigations. Someone still reviews what Sevbot did after the fact, and oncall rotations still exist. The line the article closes on captures the shift better than any stat: “Judgment, prioritization, and taste are becoming more important.” Two caveats keep this grounded, too: mobile app store review is still a manual bottleneck no agent can route around, and infra scaling is a monthly fight, not a solved problem.
If you’re building your own version of stages 1 through 3 rather than waiting for a vendor to ship stages 4 through 9, start with the gate that already exists in your pipeline: CI. Our companion article on building a lighter-weight software factory around Codex walks through that build (sibling, confirm live before linking), and the Sevbot/Perf Factory design gets its own treatment in our piece on OpenAI’s perf factory and Sevbot (sibling, confirm live before linking). An agent that loops until tests pass is a good idea only if the tests it’s looping against actually assert something. Apidog keeps API contract tests next to the spec so that gate stays honest as agents, not just humans, start pushing changes through it.