Software Has Become an Actor
The governance built for it has not caught up
Abstract. Software has become an actor. The governance that worked when software was a tool cannot govern software that acts. This manifesto names the categorical shift, walks through how existing controls fall short of governing autonomous AI action, articulates what governance must become, and traces the historical pattern that makes an independent authorisation layer structurally inevitable.
For most of computing history, software produced outputs and humans decided what to do with them. The boundary between what software produced and what happened in the world ran through the human in the chair. Identity systems, role-based access control, audit logs, content filters — the entire apparatus of software governance was designed for that era and continues to solve real problems within it.
Autonomous AI systems remove the human from the chair. They observe a situation, reason over it, and execute an action — directly, without anyone reviewing the action between the decision and its consequence. The actions are not specified in advance by a human; they emerge from inference. The result is a categorical change in what software is. Software was a producer of outputs. It is becoming a producer of consequences. The two are not the same thing. An output is a value a human interprets. A consequence is a state change in the world — a fact, not an artefact. Outputs can be reviewed before they take effect. Consequences are facts the moment they happen. They cannot be uncomputed.
None of the controls built for the previous era governs the question that the new era requires: was the specific action this system is about to commit authorised — by whom, under what constraint, and can you prove it? That question lives at the action boundary.
Action boundary. The point in a system where intent becomes consequence. Everything before it is intent; everything after, fact. Governance of autonomous AI action can exist only exactly there.
The governance that fits this era has to live there.
The Three Phases of Software
The change in what software is happens in three identifiable phases. The boundary between what software produces and what happens in the world moves each time.
Software computed. For the first several decades, software did one thing: accepted inputs, ran a calculation, produced an output. A spreadsheet computed totals; a person looked at the total and decided whether to send the invoice. A database returned query results; a person decided who to email. A compiler produced an executable; an engineer decided when to deploy it. The software was a tool. The person was the actor. The boundary between what the software produced and what happened in the world ran straight through the person sitting at the keyboard. Software-side governance was modest — file permissions, who could log into which system, basic audit logs. The human in the chair was the load-bearing governance mechanism.
Software automated. Then software started doing things humans had been doing manually. A cron job ran a nightly batch import. A pipeline moved data between systems on schedule. A deployment script provisioned servers. The actions were no longer triggered by a person hitting return; they were triggered by time, by events, by predefined conditions. But the actions themselves were still designed by humans in advance. The cron job did exactly what the engineer who wrote it intended. If an action produced an unexpected consequence, you could trace it to a human design choice, find the line of code, and fix it. Governance moved to design time — change control, code review, deployment gates, compliance reviews. The human moved from the moment of consequence to the moment of design, but the model still held: every consequence the system would produce was downstream of a human choice that could be reviewed before the system was ever turned on.
Software acts. Autonomous AI systems break that model. They observe a situation, reason over it, and execute an action — and the action they take is not a thing a human designed in advance. The system was deployed with a goal, a set of tools, and access to a context, but the specific tool invocations emerge from inference at runtime.
The engineer who built the system cannot point to the line of code that issued the refund, sent the email, or modified the database record, because there is no such line. The action is a product of the model’s reasoning over a context the model was given, and that reasoning produces sequences no one designed and no one reviewed.
The pattern is visible across domains: coding agents committing to production repositories, customer-service agents posting account changes, research agents executing cloud queries against datasets they were given access to, scheduling agents sending messages that bind the organisation. In each case, the action lands in the world before any human checkpoint exists.
The boundary between intent and consequence has nowhere to land. In the compute phase, it lived in the human at the keyboard. In the automate phase, it lived in the engineer at design time. In the act phase, neither point exists. The system observes, decides, and executes within a single continuous loop, and by the time anyone outside that loop could look at the action, the consequence has already happened.
The distinction this exposes is categorical, not incremental. An output is a value a human interprets. A consequence is a state change in the world. Outputs can be reviewed before they take effect; the entire model of software governance assumed they would be. Consequences are facts the moment they happen — a refund posted, a message transmitted, a record modified, a deployment triggered. They cannot be reviewed before taking effect because they have already taken effect.
This is why the controls built for the previous two phases do not transfer cleanly to the third. They were not wrong; they solved real problems for software that produced outputs. They are insufficient for software that produces consequences. To see why, it helps to look at exactly what each one does, and exactly where it stops.
What Existing Controls Actually Do (and Don’t)
Every organisation deploying autonomous AI systems already has governance tooling. Identity systems, role-based access control, audit logs, guardrails, observability platforms, network controls — collectively the apparatus that came out of decades of enterprise software security. Each one solves a real problem within its own domain. None of them governs the action boundary, because none of them was designed to.
Identity systems answer who the actor is. Okta, Auth0, Azure AD — mature, federated, standards-based, and load-bearing for human access. They tell you that the entity making a request is who it claims to be. They do not tell you that the specific action this entity is about to commit is authorised. Identity is a necessary input to an authorisation decision; it is not the decision itself. For autonomous systems this matters more, not less, because the entity making the request is not the principal whose intent the action represents — it is a delegate acting on behalf of someone else, and the question that needs answering is whether the delegate’s authority covers this specific action under this specific delegation. Identity systems are silent on that question.
Role-based access control maps identities to static permission sets. The user has the database-write role, so the user can write to the database. This worked when users were humans whose intent matched the operations they triggered. Autonomous systems break the model: they hold all the roles their job ever requires, all the time, with no equivalent of a user choosing not to invoke a permission they hold. And the question that matters has nothing to do with role membership. A role check answers “could this entity ever do this kind of thing?” Governance has to answer “should this entity do this exact thing, right now?”
Guardrails and content filters evaluate what the model says: input and output classifiers that catch prohibited content, prompt injections, sensitive-data leakage in generated text. They are useful, and any serious deployment should include them. They are also categorically misplaced for action governance, because text production and action execution are different operations at different boundaries. A model can produce text the guardrail accepts: “I’ll help you process that refund.” Meanwhile the tool invocation it generates issues a refund that exceeds the delegation scope. The guardrail evaluated the language. Nobody evaluated the action.
Audit logs and observability platforms record what happened. They are essential for incident response, debugging, and operational improvement. They are not governance. A log reconstructs history. Governance writes history. The two work in opposite temporal directions, and no enhancement to logging closes the gap. “We have detailed audit trails of everything the system did” is a different claim from “we have evidence that every action was authorised before it executed.”
Network controls like service meshes, perimeter firewalls, and mutual TLS govern transport. The action flowing across the connection is invisible to them. Transport governance and action governance are different problems.
None of the controls is wrong. Each was designed for a real problem and solves it. The issue is that no combination of them produces a pre-execution record binding the proposed action, the delegation chain, the policy in force, and the resulting decision into a single tamper-evident artefact that proves the action was authorised before it executed. That artefact is what governance of autonomous AI action requires, and it does not fall out of stacking existing controls on top of each other.
There is also a deeper failure mode that even the right per-action check would miss.
The Composition Argument
Each of the controls above evaluates one entity at one boundary — identity at the session, RBAC at the call site, guardrails at the model output, audit at the log line, network at the connection. Even controls that did evaluate each action one by one would still miss the class of failure that emerges from how components compose.
Autonomous AI systems are built out of multiple components that share state. A retrieval component reads customer records into a shared workspace. A messaging component reads the shared workspace and emits an outbound email. Each component holds an individually well-bounded permission. Neither permission, by itself, authorises customer data leaving the system. The exfiltration emerges from how the components compose, and no per-component check sees it, because no single component crossed a boundary it was not allowed to cross.
Locally authorised does not imply globally authorised.
This is the strongest reason no governance plane assembled from existing controls can do the work. The action that breaches authority is the composition of authorised steps, not any single step. Governance has to evaluate the composed action — what the system is collectively about to do — not just each step it takes to get there.
The Distinction That Changes Everything
The distinction is categorical, and naming it clarifies almost everything that follows.
Safety is a property of the model. Governance is a property of the system the model is wired into.
Safety asks: is this model well-behaved? Does it refuse harmful requests? Does it produce accurate answers? Does it generalise to inputs it was not trained on? These are questions about the model’s internal disposition, addressed by training, evaluation, red-teaming, alignment work, interpretability research. AI safety is a real and important discipline, and the work being done there matters.
Governance asks a different question. Was this specific action — this tool invocation, with these parameters, against this target, under this delegation, at this moment — authorised? The question is about the action, not the disposition. It is answered at the boundary where the model’s reasoning crosses into the world’s state. And it has to be answered by something outside the model, because a system that decides whether its own actions are authorised has no external accountability.
Here is what makes the distinction load-bearing rather than rhetorical. A perfectly safe model — one that refuses every harmful prompt, never hallucinates, generalises flawlessly — wired into a delegation it should not be exercising will still take an unauthorised action, because the model has no way to know it should not. A less well-behaved model wired through a governance plane that refuses unauthorised actions cannot, because the governance plane does not need the model’s cooperation to deny. The two layers solve different problems and neither subsumes the other.
This is also why “identity for AI agents” is not the right framing for what is needed. For human users, identity is presented once at login and authority is inferred from it for the session — the identity-once, authority-many model that decades of enterprise infrastructure are built on. For software actors, that model fails. Actors do not authenticate; they compose. Their identity is what they are composed of — model version, system prompts, tools, policy bindings, retrieval configuration — and that composition can shift between the moment authority is granted and the moment an action is forwarded. Their authority has to be verified at every action they propose, not once at the start of a session. The primitive that fits autonomous AI action is not actor identity, taken alone. It is actor identity bound to action-level authority, evaluated at every consequence.
Once the safety/governance distinction is named and the actor-identity formulation follows, the architectural question simplifies. If governance has to be its own layer — outside the model, at the action boundary, evaluating delegation per-action — what does that layer have to look like?
What Governance Has to Be
Four properties. Each one is forced by the structure of the problem; none is a design preference.
Pre-execution. Deterministic. Evidenced. Independent.
Governance has to precede consequence. This is the asymmetry that defines the action boundary. Consequences cannot be uncomputed; the database row is modified, the message is sent, the transfer is posted. A control that evaluates the action after it executes is doing investigation, not governance. Investigation is valuable (it diagnoses, it improves the system, it informs the next decision), but it cannot prevent the consequence it observes. Governance has to live before commit, or it is not governance.
The decision has to be deterministic. Given the same canonical action, the same delegation, and the same policy, the governance plane returns the same decision — always, everywhere, regardless of load, model version, or operational conditions. The reason is verifiability. A decision that is non-deterministic cannot be replayed by a third party, checked by an auditor, or reproduced after the fact. Non-deterministic governance is governance taken on faith — which is to say, not governance at all.
Every decision has to produce evidence. Authorisation that occurred without a verifiable record of having occurred is indistinguishable, from any external position, from authorisation that did not occur. The auditor’s question is not “do you have a governance system?” but “can you prove that policy was evaluated against this specific action under this specific delegation before this action executed?” The evidence required is concrete: the action that was proposed, the policy that was applied, the delegation under which it was authorised, the decision that resulted — bound together cryptographically so that the record cannot be silently edited, deleted, or backdated. Without that record, governance is a claim. With it, governance is a verifiable system property.
Governance has to be independent of the system it governs. A reasoning system that decides whether its own actions are authorised is a system with no external accountability. The pattern of independence appears in every regulated domain — financial auditors must be independent of the firms they audit; payment authorisation networks are operated separately from the merchants whose transactions they evaluate; in distributed systems, the routing controller is independent of the switches that execute its decisions. The same structural requirement applies. The governance layer for autonomous AI action has to be outside the model, outside the orchestration framework, outside the runtime — separable, replaceable, and operated under different organisational control than the systems whose actions it evaluates. If swapping the model changes what is allowed, the planes are not separated; you do not have governance, you have a feature of the model vendor’s product.
None of these four properties is novel. They are the structural shape every previous infrastructure layer eventually arrived at when its domain crossed the action threshold. The pattern is worth tracing.
The Pattern
Every major infrastructure layer in modern computing went through the same trajectory: a capability begins embedded in applications, separates into a standalone service, converges on a platform standard, and becomes infrastructure so foundational that serious systems cannot operate without it. The trajectory is observable, and the structural forces driving it are the same in every case.
Identity. In the early days, every application managed its own user database, its own authentication, its own session model. Each one reinvented the same wheel with slightly different security holes. LDAP and Active Directory externalised user stores into shared services in the 1990s. OAuth and OpenID Connect transformed authentication into a federated platform standard in the 2010s. Today, no serious application is built without an identity provider, and “do you have identity infrastructure?” is a rhetorical question — the answer is always yes.
Networking. Network behaviour was originally embedded in application logic; early networked applications managed their own routing, connection handling, and protocol negotiation. BGP and standardised routing protocols externalised network decisions into dedicated infrastructure. Software-defined networking made the network control plane structurally independent of the data path — the controller decides, the switches execute, and a routing decision is the same decision regardless of which switch happens to be forwarding the packet.
Container orchestration. Deployment began as bespoke shell scripts encoding operational knowledge in imperative procedures. Configuration management tools (Chef, Puppet, Ansible) externalised it into standalone products. Kubernetes turned it into declarative infrastructure: applications declare desired state, the orchestration layer converges actual state toward it, and Kubernetes is the layer underneath every serious cloud-native deployment today.
Payments. Each merchant once managed its own payment logic, fraud detection, and settlement. Payment gateways externalised processing into standalone products. Visa, Mastercard, and the card network infrastructure converged transaction authorisation onto a platform. Today, every transaction flows through an authorisation network that evaluates the transaction before it executes and produces a record that proves it was authorised. The merchant cannot execute the transaction without the network’s decision, and the receipt is the artefact that establishes the merchant’s authority to have taken the customer’s money.
Four transitions, four different domains, one shape. In every case the forces driving separation were the same. Heterogeneity demanded neutrality — the capability had to work across diverse systems, not be coupled to one. Audit demanded evidence — decisions had to be provable, not asserted. Scale demanded automation — manual processes could not keep pace. Trust demanded independence — the control mechanism had to be operated separately from the systems it controlled. Each transition happened when the cost of ungoverned action became structurally unacceptable.
Governance of autonomous AI action is currently in this trajectory. Today, governance is embedded in applications — guardrails inside the model orchestrator, ad-hoc checks in the agent loop, framework-specific permission systems. Three of the four forces press on it visibly: heterogeneity as enterprises deploy autonomous systems from multiple vendors, audit pressure as regulators and counterparties demand proof rather than assertion, scale that makes manual review impossible. The fourth force is doing the most structural work. A model vendor that also operates the governance mechanism controls both the actor and the rules that judge it — the kind of concentration that prior layers eventually pushed out to separate operators, because trust in the layer required independence from the systems being governed.
The pattern does not predict whether the layer will emerge. It predicts that it will be the same structural shape as every prior layer when it does.
Who Builds It
The structural requirement is settled. The shape of the layer is determined by the same forces that produced identity infrastructure, network control planes, container orchestration, and payment networks — neutrality across vendors, evidence as the basis of trust, deterministic evaluation, independence from the systems it governs. There is no path to “autonomous AI systems at enterprise scale, governed and accountable” that does not pass through this layer. The question is not whether it will exist. It is who builds it, and whether the organisations that need it recognise the requirement in time.
Software has become an actor. The next decade of computing will be defined by whether we govern that fact at the structural level it requires, or pretend the actor is still a tool and govern accordingly. The first path produces autonomous systems that organisations can deploy in consequential domains. The second produces incidents.
Ambit Systems exists to build this layer.
The argument made here continues in the rest of this programme. The Governance of Autonomous AI Action develops the conceptual foundations — the formal definition of consequential action, the Authority Composition Law that explains why composed systems exceed the authority of their parts, and the principles any execution-governance architecture must satisfy. From Confused Deputies to Execution Governance gives the architecture itself — the deterministic evaluation function, the delegation model, the evidence shape. The Ambit Infrastructure Thesis makes the convergence case in detail.