Right now the default answer to most automation problems is: make it an agent. We nearly did that with churn, and I'm glad we stopped.

Churn at Tutero is mission critical — it touches revenue directly. The original plan was an agent that would handle the whole thing end to end. Then we looked at what that agent would actually be deciding. We already ask the customer a long form, so every detail we need is there, structured, before anything runs. There was nothing left to interpret. The rule was writable as a set of conditions.

So we built it as a deterministic workflow instead. No model in the path.

That's the question this talk is about. Not "can a model do this" but "is there anything left to decide?" If you've already structured the input, you've done the work the model would have done. Putting an LLM after that point adds cost and variance to something you could have written down.

Support and recruitment are different. Judgement is real there, so those are agents. The second half of the talk is about containing them: least-privilege tool access across nine internal systems, and a deny-by-default rule for what agents are allowed to hand to each other.

You'll leave with the test I use to make the call, and the patterns I use to stop an agent reaching further than I meant it to. This session is aimed at Constraining AI: holding probabilistic AI to deterministic rules and human oversight.

Jake Ranallo

The speaker's sessions