Who is accountable when a system acts unattended?

A named person, and the arrangement should say who before anything runs. Accountability that is assigned to a team is assigned to nobody at three in the morning. The same person owns the alerts, which is why alerts that point at a shared inbox are a sign that the accountability was never actually placed.

What happens when it gets something wrong?

The design should assume it will. Sensitive actions wait for a person, anything that cannot be grounded in your own documents is handed over rather than guessed at, and every action is logged so a mistake can be found rather than merely suspected. Running the first process alongside your team, with everything visible and nothing sent, is how the wrong answers happen where they cost nothing.

Is this about replacing people?

It is about moving the reading, routing, chasing, re-keying and reporting off people who are also doing the judgement. A system scoped on the promise of removing headcount tends to be scoped without a line in it, because the line is exactly the part that keeps people in the work. The useful question is which decisions you would never want made without a person, and that question has answers.

Can we change the limits ourselves afterwards?

You should be able to, and it is worth making it a requirement in writing. Thresholds, approval rules, working hours and the list of actions that wait are configuration rather than code in a well built system. If tightening a limit requires a change request and a deployment, the limit belongs to the supplier rather than to you.

How do we know the audit trail is real?

Read yesterday. Not a sample, not a demonstration, and not with the supplier narrating. If somebody who does not work in software can read the log and say what happened, what it decided and why, the trail is real. If it is a sequence of technical traces that require interpretation, there is a record but there is no accountability.

A material study in bright daylight. A dense radial wheel of clear turquoise glass blades with one clear opening at its centre, and a single pearl silver ribbon curving down and away from that opening across the pale floor.

Home
Insights
Agentic Operating Systems
Where to draw the line

Decision guide

Where to draw the lineon what software decides.

The line is the list of decisions a system may never make on its own. It is written before the build, in the words of the business rather than the software, and it holds only because it is engineered in: limits per agent, gates on sensitive actions, answers grounded in your own documents, and a log a person can read.

Three tests at the end of this article establish whether a line is a control or a description. Each of them is something a buyer can run themselves, on a running system, in under an hour. Written 23 September 2026.

HUREAL / Material studies

Timing

Every system has a line.Most are found later.

A system that acts on behalf of a business has a boundary whether or not anybody wrote one. The only question is when it gets discovered, and the two possibilities are a scoping session and a complaint.

Writing it first is cheap because it is a conversation. Writing it afterwards is expensive because it is a change to something already running, in a business that has already adjusted to the behaviour. The people who would have had opinions about it have by then also adjusted, which means the boundary that gets written second is usually the one that happened rather than the one anybody wanted.

The line is also the artifact that makes the rest of a project scopeable. It determines which integrations are needed, which actions need an approval queue, what the audit trail has to contain, and how much of the work is reviewable. A quote produced before the line exists is a quote for an unknown quantity of software.

The axis

What it takes over.What stays yours.

Most business processes divide along one seam. On one side is the handling: reading, finding, checking, recording, drafting, chasing, reporting. On the other is the judgement, which is usually one sentence in the middle and is the only part that needed a person in the first place.

Suited to a system

Reading what arrivesEmail, forms, messages, documents, at any hour, without a queue forming overnight.
Classifying and routingAgainst criteria the business wrote, to the right person, with the history attached.
Retrieving the answerFrom your own documents, policies and price lists, with the source shown next to it.
DraftingReplies, quotes, summaries and reports, prepared and held rather than sent.
Chasing on a scheduleDocuments, approvals, renewals and unpaid invoices, until they land or until it stops.
RecordingSo the same information is not typed into a second screen by a person.
EscalatingWhich is a decision to stop, and is the single most important capability on this side.

Belongs to a person

PriceEvery exception, every discount, every negotiation. This one is rarely argued with.
Credit and riskWho gets terms, and on what. A judgement about a relationship, not a lookup.
The relationshipThe call, the site visit, the apology. Nothing here is a message that could have been drafted.
Anything irreversibleA payment, a deletion, a cancellation, a message to a customer that cannot be recalled.
Anything where being wrong is expensiveAnd where being human is the point rather than an overhead.
The final wordAnything reaching a customer can wait for a person, if that is what the business wants.
The off switchYours, and it works without calling anybody.

The seam is not always in the same place, and the point of writing it down is that the business decides where it is rather than inheriting somebody else's default. What does not vary is that the right-hand column is never empty.

Applied

Two questionsper action.

The seam is easier to place with a rule than with an instinct, and two questions do almost all of the work. Can the action be undone, and what does it cost if it is wrong? Gate on those, rather than on how impressive the action sounds.

Ten actions, tested on reversibility and on the cost of being wrong
The action	Can it be undone	Where it belongs
Reading an inbound message	Nothing to undo.	The system, without a gate.
Classifying and routing it	Yes, trivially, and a person sees it either way.	The system, without a gate.
Recording it against a customer	Yes, and the log says what changed.	The system, without a gate.
Drafting a reply	Yes. A draft is not an action.	The system, without a gate.
Sending that reply to a customer	No. It has been read.	A gate, at least until the agreement rate is known.
Booking an appointment in a real calendar	Yes, but undoing it means a second contact with a person.	The system, inside rules, with an exception path.
Quoting a price	No, in practice. A quoted number is a number the customer heard.	A person, or a gate with no exceptions.
Chasing an unpaid invoice	No. The message has been received.	A gate on the first, then rules once the wording is settled.
Issuing a credit or a refund	Technically yes. Commercially no.	A person.
Deleting a record	No.	A person, and preferably nobody at all without a retention rule.

Scroll the table sideways to read it.

How it holds

Four mechanisms,not four sentences.

Limits written into the build, not into a policy

Each agent gets a role, a list of tools it may use, a list of actions it may take, and an explicit list of things it may never do. The last list is the important one and it is the one usually missing. A limit that exists as guidance in a prompt is a suggestion; a limit that exists as a tool the system does not have is a limit.
Gates that a person actually works in

The system prepares, a person approves, then it acts. The quality of a gate is entirely the quality of the queue behind it: each item shows what is about to happen and why, the queue has an age on it, it can be worked with a keyboard, and the default of doing nothing is safe. A gate whose queue is a shared inbox has the cost of a gate and the effect of none.
Grounding, with the source shown

Answers come from the company's own documents, policies and records, and the answer carries a link to where it came from. The test of a well grounded system is not that it is right often. It is that it says it does not know, and hands over, rather than producing a confident sentence about a price list it never found.
A log written for a person

What it did, when, on whose behalf, why, and what it used to decide. In plain sentences. A technical trace is a record for an engineer and it is not accountability, because the person who has to answer for the system cannot read it. The two are often confused, and the confusion is only discovered on the day somebody asks what happened.

Two failures

Too tight is a failureas well as too loose.

Drawn too tight

Everything waits for approval. The queue becomes a second inbox, the approvals become a batch job somebody does on Friday, and the system has added work rather than removed it. The process is now slower than it was and has a subscription attached.

The tell is an approval rate near one hundred per cent with almost no edits. If a person approves nearly everything unchanged, the gate is not exercising judgement, it is providing reassurance, and the decision should move to the system with monitoring instead.

The honest response is to move one action type at a time, on evidence, and to say so in the arrangement rather than treating every gate as permanent.

Drawn too loose

Something irreversible happens that nobody reviewed. Usually it is a message to a customer, and usually it is correct in every particular except tone, timing or the fact that the account was already in dispute.

The tell is that nobody can name who would be told. Ask who finds out, and how long it takes. If the answer involves somebody noticing, the action is outside the line regardless of what the document says.

The other tell is a system that has never handed anything over. A month of operation with zero escalations does not mean it understood everything; it usually means it was never given a way to say that it did not.

Gate on irreversibility and on the cost of being wrong. Neither impressiveness nor difficulty is a reason for a gate, and both are common reasons one gets built.

Proving it

Three testsyou can run yourself.

Before any of this is trusted, the system should do the work alongside the people who currently do it, where everything is visible and nothing is sent. What matters about that period is that it has a written exit condition, agreed in advance and expressed as a number: the agreement rate between the system and the people, on a defined sample, per action type, over a defined period. Without one, shadow running ends when somebody loses patience with it, which is a scheduling decision dressed as a quality decision.

After that, three tests establish whether the line is a control or a paragraph. Each takes under an hour and none of them requires a technical background.

Tighten a limit, without calling anyone
Change a threshold, a working hour or an approval rule from an administration screen. If that needs a change request, the limits belong to the supplier rather than to you.
Use the off switch
Stop it, confirm it stopped, and start it again. Test this before launch and once afterwards. An off switch that has never been used is a configuration setting somebody believes in.
Read yesterday
Open the log and have somebody who does not work in software say what happened, what was decided and why. If that is not possible, the system has a record and the business has no account of itself.

What none of this establishes. These tests prove that the controls exist and work. They do not prove that the line is in the right place, and nothing external can, because that is a judgement about your business and your customers. What they do is make the judgement reviewable, which is the most that any supplier can honestly offer.

Questions

Questions peopleactually ask.

Who is accountable when a system acts unattended?

A named person, and the arrangement should say who before anything runs. Accountability that is assigned to a team is assigned to nobody at three in the morning. The same person owns the alerts, which is why alerts that point at a shared inbox are a sign that the accountability was never actually placed.
What happens when it gets something wrong?

The design should assume it will. Sensitive actions wait for a person, anything that cannot be grounded in your own documents is handed over rather than guessed at, and every action is logged so a mistake can be found rather than merely suspected. Running the first process alongside your team, with everything visible and nothing sent, is how the wrong answers happen where they cost nothing.
Is this about replacing people?

It is about moving the reading, routing, chasing, re-keying and reporting off people who are also doing the judgement. A system scoped on the promise of removing headcount tends to be scoped without a line in it, because the line is exactly the part that keeps people in the work. The useful question is which decisions you would never want made without a person, and that question has answers.
Can we change the limits ourselves afterwards?

You should be able to, and it is worth making it a requirement in writing. Thresholds, approval rules, working hours and the list of actions that wait are configuration rather than code in a well built system. If tightening a limit requires a change request and a deployment, the limit belongs to the supplier rather than to you.
How do we know the audit trail is real?

Read yesterday. Not a sample, not a demonstration, and not with the supplier narrating. If somebody who does not work in software can read the log and say what happened, what it decided and why, the trail is real. If it is a sequence of technical traces that require interpretation, there is a record but there is no accountability.

Three that followfrom this one.

The service behind this article

The line is drawnbefore anything is signed.

An Agentic Operating Systems engagement produces the line as a written document in the first phase, before the build, and it is signed rather than assumed. Nobody should discover a boundary after launch, and the discovery is expensive in exactly the cases where it matters.

The discovery call draws the line with the people who own the consequence. What the system takes over first, what stays with your people, which actions wait for approval, and who is told when something is held. The written proposal that follows carries it, along with a scope and a price built on it.

Book a discovery call

More on this subject in the Agentic Operating Systems index, and everything else at Insights.

A material study photographed in bright daylight. A tall fan of clear turquoise glass fins rises on the right of the frame with a polished pearl silver ribbon curving through it, standing in a shallow film of still water. The left of the frame is empty pale mint.

Software you own from day one.