When Did Anyone Last Run This By Hand?

As you’ll recall from the past few posts in this AI in finance series, the phrase “human in the loop” actually cashes out to at least four different approaches; we’ve designated them as approval before action (where nothing lands until a person accepts it), exception-only escalation (where the routine clears and the anomalies route), spot-check sampling (where a fraction of the final output gets inspected), and post-hoc confirmation (where the output is used and checked afterward).

This piece will not cover a fifth arrangement, but is rather about an assumption underneath all four, namely: that someone on the team would still be able to execute the workflow unassisted if all the generative AI models were suddenly taken offline (perhaps in response to a rogue agent swarm executing an unauthorized hacking campaign).

But the ability to do “by hand” what had once been passed to machines is a capability, and capabilities are either maintained, or they aren’t, and almost nothing in a control description treats “be sure someone knows how to do this the long way” as a skill requiring upkeep.

Because this argument tends to be misread in a particular way, it’s worth plainly stating at the outset that AI in finance is arriving regardless of how anyone feels about it, the staggering volumes involved are not going back, and nothing I’m saying here is an argument for doing less of it.

The questions we’ll take up here are narrow, more practical ones about which specific abilities you should keep exercised, and at what attendant cost.

Why does finance already have a name for this situation?

Because it’s an aspect of continuity, and continuity is a discipline finance has been practicing, testing, and reporting on for decades.

The Cambridge Centre for Alternative Finance’s 2026 global survey gives the phenomenon a name: “collective forgetting,” which refers to organizations losing the institutional memory and the capability to execute processes manually should they need to. It ranks third among all AI risks in the survey at 51% among vendors, 55% among industry users, and 42% among regulators. The survey is cross-sectional and self-reported, so it measures where concern sits rather than where capability has actually decayed, and it can’t establish causality in either direction; still, it’s noteworthy.

What makes this actionable rather than merely worrying is that the manual fallback is already regulatory furniture. The Bank of England’s financial stability guidelines discuss workaround solutions for operational and financial contagion, naming manual processing where automated systems are impaired as one of them, alongside alternative funding sources. On the US side, the FFIEC’s Business Continuity Management booklet is more direct still: among the things exercises and tests should confirm is that systems can support critical business processes, and the examples given include the existence of manual workarounds.

While only some of these regimes say anything about AI-driven capability loss, what they do establish is that finance already owns the vocabulary, the cadence, the testing obligation, and the budget line for the kind of preparation that would mitigate the relevant loss. Filing this under continuity rather than under AI puts it somewhere with an owner.

Two starting positions, and they need different work

Whether the task is retention of manual-execution capabilities or the construction of such capabilities depends on which kind of firm you are, and the difference is larger than it might first appear.

The first position is having the muscle and not wanting to lose it. This is the situation the incumbents are in. There are people in the building who closed the books before the current system existed, who worked alert queues when the narrative had to be written from scratch, who can price an instrument without the provider’s model. The process was run by hand within institutional memory, and often the documentation from that era still exists somewhere. The work here is retention: exercising something that exists, before the people who hold it retire or the documentation stops matching anything.

The second position is never having built it. A (probably younger) firm that automated a process at inception has no prior state from which to decay. There was no manual close, no hand-worked queue, no era when someone reconciled the accounts in a spreadsheet. Ask “when did anyone last run this by hand,” and the honest answer is: “nobody ever has.” That isn’t a deficiency, per se, because building a process without a manual precursor is what greenfield looks like, and it’s a large part of why challengers move faster. It does mean the work is construction rather than maintenance, however.

Now, the survey tells us traditional FIs are the most concerned; it doesn’t tell us why, and it certainly doesn’t establish that challengers are less exposed. But what it does communicate is the usefulness of asking the question of your own processes so you can prepare appropriately.

What does retention look like, and what does each piece cost?

Retention can be broken up into different practices. Below, I’ve laid out five, sorted in rough ascending order of cost. None of them is exotic; the difficulty is that each produces something no business unit asked for.

Periodic manual execution means running one entity’s close, one account’s reconciliation, etc., by hand and on a schedule.

Hand-run sampling is the cheapest genuine test available, because it does double duty. Working a sample of items without model assistance is exactly the base-rate exercise the escalation and sampling pieces both concluded a firm needs and nobody budgets. Doing it once buys the base rate and the capability check together.

Drills against a named loss — a provider outage, a model withdrawn by its vendor, an extraction service that stops responding during close week — are the practice with the most direct regulatory precedent, since scenario testing is what various laws already require. Extending an existing severe-but-plausible scenario to include an AI dependency is a smaller step than standing up something new.

Documentation maintained against the manual process decays fastest, precisely because nobody consults it. It needs a named owner and an event-based review trigger that fires during a system migration, a new product launch, a provider change, etc.

Staffing and rotation is the expensive one and the one with a genuine tension in it, since keeping people fluent in work that is widely believed to be obsolete is a hard thing to offer as a career.

For a firm in the second starting position, the first two rows aren’t maintenance costs at all. They’re build costs: before you can run a process manually on a cadence, somebody has to work out how it would be run manually, and write that down. That’s a delimited project, and it’s considerably cheaper to do while the automated process is working than during the week it isn’t.

How would you know the capability had decayed?

You wouldn’t, and that’s the operative problem.

In the face of an atrophying ability to operate without AI agents, there are no changes in metrics, throughput, or the error rates. Coverage ratios sit exactly where they sat last quarter. The loss of manual capabilities produces no signal whatsoever until the day it’s needed, and on that day the measurement and the consequence arrive together. The only instrument that works is a test, which is a strong argument for thinking of this as a continuity issue (since testing is the one thing continuity programs are already built to do).

I’m curious about how people with more on-the-ground experience find these claims. We’re constantly investigating how AI in finance is evolving, so don’t hesitate to get in touch directly or skip straight to telling me in the M.A.S.E. Discord.

Frequently asked questions