Sapilon is in private development and preparing for early access. Join the waitlist →

Blog

AI agents now write software. Who's flying the plane?

· Sapilon

TL;DR

AI can now plan, write, test, and propose changes end to end. But autonomy without boundaries is just risk at machine speed. Aviation solved this long ago: autopilot flies within a defined envelope while a pilot stays in command. Software needs the same model. Declare what agents may change alone, what needs review, and what they must never touch.

An airplane wing above the clouds: autopilot does the flying, but a pilot is still in command

From autocomplete to autonomy

The last three years of AI coding tools can be told in three steps. First autocomplete: the model finished your line. Then chat: you asked for a function and pasted the answer. Now agents: you describe an outcome and the AI plans the work, edits files across the repo, runs the tests, fixes what failed, and opens a pull request, often while you’re doing something else.

For once the noise around a technology shift is justified. Agents don’t just make developers faster. They change who and what can produce working software. Teams wake up to branches written overnight. Non-engineers ship internal tools that used to sit in a backlog for quarters. The bottleneck has visibly moved, from writing code to deciding which code should be trusted.

That second problem is much older than AI. Aviation hit it in the 1970s.

What autopilot actually taught us

Modern autopilot can fly a jet for nearly the entire route and land it in fog no human could land in. No airline responded by removing pilots, and not for sentimental reasons. What aviation worked out instead was a discipline that made extreme automation safe enough to bet lives on.

Autopilot is not an on/off switch. It operates inside defined limits (altitudes, speeds, phases of flight) and disengages loudly the moment conditions leave that envelope. A named human remains responsible at every second of every flight; automation executes, but someone is in command. And trust is earned per system rather than granted globally. The autothrottle is certified for things the flight computer isn’t, and each subsystem has an explicit scope of authority.

Compare that to how most teams adopt coding agents. There’s a single permission dialog, “allow the agent to edit files,” and from then on the AI has the same authority over the payment integration as over a README typo. That’s not an envelope. That’s handing over the aircraft because the autopilot sounded confident.

Autonomy without boundaries is risk at machine speed

The failure modes of ungoverned agents are already the industry’s favorite war stories. The agent that “fixed” a failing test by deleting it. The refactor that weakened an auth check without anyone noticing. The migration that ran against the wrong database. None of it is surprising once you accept two things about agents. They are non-deterministic, so the same request can produce different diffs on different days, which means review can’t be an afterthought. And they are fast. A junior developer makes one risky change a day; an agent can make forty an hour. Speed multiplies whatever governance you have, including none.

Meanwhile the outside world is moving toward accountability, not away from it. Regulators, auditors, and enterprise buyers keep asking the same three questions about AI in the development process: what is it allowed to do, who reviewed what it did, and can you prove both? “We prompt and hope” does not survive a compliance review.

The industry’s current fixes attack the symptom. More code review, better evals, CI gates. All useful, but they check the output. The missing piece is upstream: nothing in a typical codebase declares what the agent was allowed to touch in the first place.

Give the codebase an envelope

This is the principle Sapilon is built on. If AI is going to work inside your codebase permanently, as a colleague rather than a demo, then the codebase itself should carry the rules of engagement.

//!AUTO is the agent’s cruise phase. CRUD screens, list pages, standard forms: AI changes these autonomously, because the blast radius is small and the patterns are proven. //!SAFE means automation proposes and a human disposes. For business logic and integrations, the agent writes the change and a person approves it before it lands. //!DEV means hands off the controls. Payments, permissions, compliance-critical code: AI must not modify it, the same way autopilot never gets authority over the decision to take off.

The zones live in the code, in plain Git, so they survive tool changes, staff changes, and model upgrades. The envelope also extends beyond the diff. Agents do their work against a build environment and a rehearsal environment with realistic data, never the live system, because “the autopilot performed well in the simulator” is the only acceptable way to find out.

None of this slows the agent down where speed is safe. That’s the part the governance-versus-velocity debate gets wrong. Aviation’s automation rules didn’t make flying slower; they made extreme automation acceptable. Boundaries are what let you say yes to more autonomy, not less.

Who’s flying yours?

Agentic coding is not hype. It’s the biggest change to software production since open source, and it’s compounding monthly. But every mature autonomous system, from planes to factory robots, earned its autonomy the same way: explicit scope, a human in command, and a record of who did what.

So when you evaluate any agentic tool, ours included, skip “how much can it do alone?” and ask the aviation question instead. When the agent is flying, what exactly is it cleared to do, and how does it know?

If the answer is a permission dialog and good intentions, you have an impressive demo with your production system attached to it. If the answer is an envelope written into code you own, you have automation you can actually trust with the controls.

Ready to take your prototype to production?

Request early access

← All articles

Sapilon is an AI-first software system that governs AI to modernize legacy systems and graduate AI prototypes into production-ready, AWS-native software.