#19 The pause before action

Imagine you’re applying for a mortgage – you’ve finally saved up for a deposit, and you’re rewarded with weeks of back and forth with brokers, chasing payslips and pulling bank statements. Agentic finance promises a soothing alternative. Instead of you handling it all, an agent pulls your documents together, finds you the best rate, and handles your application on your behalf. All you need to do is say yes.

Attractive? Probably. But I’m not sure it’s realistic.

To let an agent act for you, you need to trust that it will act in your best interest, and be able to take control if it doesn’t. But how do you keep control of something designed to work without you? To answer this, our team has been working on a speculative project to design a trustworthy experience of agentic finance. We considered the different ways that trust could break down and explored why that is. In one of them, despite failing the user, the agent’s logic is hard to fault – which is exactly what makes it dangerous.

Misaligned intent describes the scenario where an agent does what you said, but not what you meant. If you ask an agent to “minimise what I spend on energy”, it might switch you to a plan with a lower rate – but also with worse terms and exit fees. 

The ‘agentic promise’ is an agent that can figure out the best way to achieve a goal and make it happen autonomously. But if the goal is ambiguous, the same agent could wind up taking the best path to the wrong thing.

There are initiatives already trying to mitigate this. Mastercard's Verifiable Intent and Google's Agent Payment Protocol capture users' intent in a tamper-proof record, creating an evidence trail so you can compare what you got with what you asked for. But this doesn’t address the risk of misunderstanding. It doesn’t matter if an agent acts in line with its record of what a user wants, if that record never reflected what the user actually wanted. To realise the potential of these services, we’ll need ways to verify alignment, not just intent.

We’re exploring a pattern for an agent that will check it’s understood you correctly before it acts. When you say “minimise what I spend on energy”, it will outline the considerations and tradeoffs, and help you understand their impact. It will only proceed on clear terms you’ve explicitly agreed to, instead of resolving ambiguity silently. 

The best agents won't just deliver for users, they'll preserve their agency. This could mean building in the pause before action, and the explanation before commitment. For the teams building them, this isn't a feature – it's a responsibility.

For 10 years, IF has been helping large organisations build customer-facing services that scale safely, earn trust from the start and deliver long-term impact. We prototype, test, and launch AI products and services that people believe in and want to adopt, while helping organisations change the way they work in the AI age.

Learn more or talk to us.

What we’ve been working on

Accelerating cities’ journey from AI prototyping to piloting

Our team is getting ready to support cities across the Americas taking part in the Bloomberg Philanthropies City Data Alliance, using data and AI to tackle critical citizen challenges. We’re establishing the right ways of working and rituals to accelerate the cities’ journey and move them quickly and effectively from prototyping to piloting.

Unlocking the value of public sector information

We’ve just wrapped up a project with The National Archives, helping people navigate the rules for reusing public sector information. 

The Open Government Licence is meant to make that simpler for everyone, from public servants to small businesses. But for most people who encounter it, the OGL raises more questions than it answers. 

We set out to understand who’s using the framework and why, and how to improve awareness, understanding, and use of it. Through rapid prototyping, we surfaced where the framework created uncertainty and found better ways to support the diverse needs of its users. Read the case study.

What we’ve been reading

I secured my driver’s licence in 2024 after managing dodgy gear-changing habits, a fear of magic roundabouts, and the biggest obstacle: a tiny trickle of test availability. Two years later, the DVSA has introduced new rules outlawing third-party test booking – putting a big blocker in the way of automated bulk-buying, which caused the limited test availability. As agentic technology advances, I wonder what we might learn from the driving test situation to prevent similar impacts of mass buying on public supply. Read the story.

DemocracyNext argues that data centres need social licenses – the ‘OK’ from the people living with the consequences of the infrastructure – to operate. The more formal requirements (like permits and approvals) often address the needs of institutional players, without capturing the voice of the public. So DemocracyNext offered citizen assemblies as a mechanism for community participation in deciding what gets built in their spaces. I’d love to see this implemented – if we’ll be impacted, we should have a say in what goes. Read the full newsletter.

A life without US tech infographic puts a sharp exclamation point to the global dominance of US technology, by showing what happens when we have a single point of failure. Check out the illustration and read the full piece (paywall).

Final thought

To my fellow agent-builders, cautious technologists and people who care about people: I’d love to know your thoughts on building agents you can trust – particularly in finance. How do we make sure we design agents that clarify ‘just enough’ to avoid misunderstanding - without turning delegation into a questionnaire?

With best wishes,

— Omotara and the IF Team

This month’s edition was written by Omotara Edu, Senior Creative Technologist at IF, interested in people, their needs and how tech can support them.

Find us on LinkedIn and Medium for more insights and updates.