Organizations evaluating an AI system almost always start with a performance question: does it produce accurate outputs, does it save time, does it get adopted. These are reasonable things to measure, but they answer only whether the system works. But once it does, what do we actually know about whether the organization can still stop using it? That second question, reversibility, sits outside the standard evaluation because at the moment of adoption it appears to answer itself. Of course we could go back to the old process, the reasoning goes; we still have the contract, the data, the staff who ran it before.
Nobody forbids turning the system off. No policy makes it illegal, no clause blocks it. What makes it effectively irreversible is not permission but accumulated cost, and that cost is invisible at deployment because it has not yet accumulated. It only becomes visible at the moment someone tries to invoke it, whether the trigger is a vendor acquisition, a model retrained into different behavior, or a shift in regulatory expectations. By then, the conditions that made the fallback plausible have usually already changed.
Irreversibility rarely arrives as a single decision. Organizations do sometimes close a door deliberately, decommissioning a legacy system, or cutting the team that ran the old process. The more interesting case is the one nobody decided at all. It accumulates while everyone is watching the metrics that justified adoption.
The Ratchet, Not the Switch
The language organizations use to talk about AI systems assumes a switch: on or off, present or absent, a state that can be toggled back at will. The way these systems actually become embedded in an organization behaves more like a ratchet, a mechanism built to turn one direction and resist turning back.
The mechanism works in two stages. First the process changes: once a workflow is rebuilt around the system's outputs rather than merely supplemented by them, removing the system does not restore the old workflow, because the old workflow no longer exists as a defined sequence of steps anyone still follows. Then the organization changes around it. Judgment that isn't exercised decays, the way any practiced skill does. Sign-off and audit procedures get rebuilt around the assumption that the system's output is already there as an input, so removing the system can invalidate the procedure itself. Contractual terms, data formats, and integration points harden around the same assumption, until reverting to a manual process carries a cost nobody priced into the original decision.
None of this requires bad faith or poor planning. It is the predictable consequence of successful integration. The more useful a system is, the more the organization reorganizes around it, and the more that reorganization narrows the path back. Reversibility does not decline because the system performs badly; it declines because the system performs well enough to be trusted with more of the workflow than it was originally given. Performance evaluation is much better equipped to measure failure during operation than loss of capability once operation stops.
That matters because reversibility is closely related to resilience, though the two aren't the same. A resilient organization doesn't need the old process running in parallel forever, only some credible way to continue if the system becomes unavailable, and the more completely a workflow is rebuilt around the system, the less plausible that fallback becomes. A system can be operationally successful while quietly making the organization less resilient.
The asymmetry is obvious next to ordinary disaster recovery. Organizations routinely test whether their infrastructure survives a failure: whether backups restore, whether systems fail over. Far fewer test whether the organization can recover the judgment a system has displaced. There is a rehearsed plan for technical recovery. There is rarely an equivalent for judgment recovery.
Why the Question Goes Unasked
It is worth asking why this receives so little attention given how much attention AI governance receives generally. Much of that governance is concerned with what happens while a system is operating: the quality of its outputs, the conditions under which people may rely on them, the controls surrounding its use. What happens once the system is removed is less often treated as a property of the system itself, because removal is assumed to be a return to a known prior state rather than a transition with costs of its own.
System replacement is a recurring pattern in advisory work, and it predates AI by decades. A migration project can prove that the data migrated, the interfaces work, and the documented requirements were reproduced. None of that proves the organization retained the capability the old system supported, because what migrates from one system to the next is a representation of the workflow, not the workflow itself, and whatever was never made explicit in that representation is exactly what is most likely to be lost: the exceptions the old system accommodated, the workarounds that existed for reasons nobody wrote down because everyone simply knew them.
AI sharpens this problem rather than introducing it. Much of what an AI system replaces was never represented explicitly to begin with. It was judgment, exercised through practice, not written down anywhere a migration project could capture. The problem is not new. AI simply makes the missing specification harder to reconstruct.
Consider reimbursement decisions for medical aids within a statutory health-insurance system: wheelchairs, hearing aids, orthopedic supports. A prescription arrives, eligibility and medical indication are checked against it, a provider's contractual conditions apply, and the case is matched against the person's prior provision history to see whether enough time has passed since a comparable device was last supplied. Some cases go to independent medical review before a decision is communicated to the provider and the insured person. It is a chain of dependent judgments and checks, not a single approve-or-deny classification.
An AI system introduced somewhere in that chain typically arrives with a modest mandate: extract the prescription data, classify the product category, flag whether the provision interval has been met, identify missing documentation, recommend whether a case needs review. The formal decision stays with a human caseworker throughout. Three years later, though, the caseworker no longer has to reconstruct a case from the prescription up. The workflow assumes the system has already performed the first-pass interpretation. Staffing reflects the reduced manual workload. Escalation procedures are built around the exceptions the system flags, not the ones a caseworker might independently notice. Training teaches new hires how to review the system's output, not how to assemble a case without it. None of these changes is individually irreversible. Together, they are.
Now imagine the system is unavailable for a week, not permanently, just long enough to matter. The question is no longer whether the vendor can restore the service. It is whether the insurer has enough trained staff, current procedures, and operational capacity to process cases without it. A recovery plan for the infrastructure does not answer that question.
This is worth pulling apart into three separate questions rather than one. An organization can audit a system without being able to replace it, and switch a system off without being able to recover the work it performed. Auditability asks whether the system can be inspected. Reversibility asks whether it can be stopped. Recoverability asks whether the underlying function survives its removal. The first two can largely be answered technically. The third is an organizational question, and it is the one that becomes consequential when something actually forces the issue.
What Lock-In Actually Takes Away
Vendor lock-in is not a new problem. Organizations have lived with it since early enterprise systems made switching cost a familiar line in procurement risk assessments. The usual response treats it as economic, a cost that can be estimated and mitigated through contract terms, data portability, or dual sourcing. What AI adds is not a new kind of dependency so much as a harder version of the one system migrations already expose. Conventional software often automates rules or processes that can at least in principle be specified independently of the tool: bookkeeping rules, approval criteria, calculation logic. Judgment-like AI work can be different: the judgment it replaces was often never specified to begin with, only exercised through practice by people who could not fully have articulated their own decision rules if asked.
The lock-in an AI system produces is therefore not only economic. It becomes epistemic: the organization gradually loses both the capability to perform the work without the system and the knowledge of how that work is supposed to be done, which is a deeper loss than losing a tool. Reversibility calculations, on the rare occasions organizations make them, treat the pre-AI process as archived somewhere, intact, like a paper file in a drawer that can be pulled back out. In practice it was never a file. It was a capability that had to be exercised to stay alive.
There is, in principle, a way to slow this: preserve manual capacity deliberately, rotate staff through the old process, keep documentation current rather than filed away. Organizations rarely do this, not from oversight but from a conflict of purpose. The system's entire justification was to stop paying for that capacity, so preserving it as insurance means paying indefinitely for what the investment was meant to retire, a harder case to make every year the system runs without incident. Every decision delegated this way, without preserving the capacity to perform it independently, creates something close to capability debt: capacity saved now, against a reconstruction bill that comes due precisely when the organization can least afford to pay it.
This doesn't argue that organizations should avoid systems that are hard to reverse. Most valuable integrations are, almost by definition, hard to reverse; that is part of what makes them valuable. Someone can always revoke an API key. That was never really the question. The real one is whether the organization still knows how to do the work the software replaced, or whether anyone left could tell you.
