Hi everyone, thanks for coming back to Customer Futures. Each week I unpack the fundamental shifts around Personal Agents, digital customer relationships and Empowerment Tech.
If you’re reading this and haven’t yet signed up, why not join a growing number of global executives, entrepreneurs, designers, regulators and other digital leaders by clicking below.
To the regular subscribers, thank you.
(p.s. did your email cut this newsletter short? You can find the full post here)
Hi folks,
Welcome to Part 5, the final post in my series unpacking Meta’s new personal AI agent, Muse. Looking at what it all means for the coming shift to ‘agentic commerce’ more generally.
Over the last four posts, we’ve explored what happens when personal agents get their own computers, tools and memory. How they might interact with businesses, why they need legitimate identities and permissions, and what happens when customers start sending agents instead of turning up themselves.
In Part 4, we explored the next difficult question:
What happens when the agent that’s supposed to represent the customer also works for a company whose business model depends on influencing that customer?
Today, we come to perhaps the most important question of all.
Who actually controls the agent?
Because personal AI agents could represent the biggest shift in customer empowerment since the invention of the internet itself. Giving individuals intelligent digital representatives that can understand their needs, coordinate their lives, negotiate with businesses and act on their behalf.
I’m not saying all this just so I can join the jazz-hands agentic commerce train coming out of Silicon Valley.
Some of the most serious, sober and smart people I know, working close to AI, digital trust and customer experience, are seeing the same thing coming. And are having the same concerns.
I’m now wondering if we’re about to build the most sophisticated walled garden we’ve ever seen.
Only this time, the walls won’t be around websites, apps or platforms.
They’ll be around us.
Here’s what I want to talk about in this final post:
The next generation of walled gardens
Who controls your agent’s memory?
The difference between security and independence
The liability problem (that nobody has solved)
The great customer empowerment test
Let’s go.
1. The next generation of walled gardens
Remember when the internet was supposed to be open? Anyone could publish a website. Anyone could build a service. Anyone could link to anything.
Then came the platforms. Social networks, app stores, messaging ecosystems and digital marketplaces. Each created extraordinary value, but also developed increasingly sophisticated ways of keeping customers inside their own environments.
We’ve spent the last twenty years building 'digital walled gardens’.
You can do anything you like, as long as it’s on their terms, in ways and places they pre-approve, and as long as they get a cut of the spoils: your rich personal data, your intent, and in most cases, a view and slice of every transaction on their platform.
Now imagine your personal AI agent becoming your main interface to the digital world.
It knows your preferences, remembers your decisions, manages your calendar, reads your emails, understands your relationships and increasingly makes decisions on your behalf.
It knows which airline you prefer, which hotel you stayed in last year, what you paid for your insurance and why you rejected that job offer six months ago.
You no longer need to open dozens of apps or visit hundreds of websites. Your agent can bring everything together, across the fragmented businesses and services you deal with every day.
Excellent stuff.
But what if you can’t move that accumulated personal context to another provider? What if your agent prioritises certain commercial partners? What if its provider decides which services, businesses and other agents are allowed to participate?
And what if you don’t even know those decisions are being made?
We’d end up with the same digital walled gardens of today, where all the activity is governed centrally, pre-approved and managed by the Tech Platform Du Jour (take your pick of the Mag7).
But now, the walled garden is no longer somewhere you visit. It’s something that follows you around.
And because your agent is likely to become your main interface to almost everything you do online, the potential for digital lock-in is not only enormous, it’s inevitable.
Or is it?
2. Who controls your agent’s memory?
Here’s a question that will become increasingly important.
If you spend five years using a personal AI agent, who controls what it learns about you?
Not just the files you’ve uploaded or the conversations you’ve had. I mean the accumulated understanding of your life. Your preferences, habits, relationships, priorities, past decisions and the thousands of little details that make an agent genuinely useful.
This is what makes personal agents so interesting. Their value won’t just come from the underlying AI models, but from the vast personal context they accumulate.
And that creates a rather powerful new form of lock-in.
Imagine you’ve spent years teaching your agent about your life, and then a better one comes along. Can you leave? Can you take your history, preferences, memories and permissions with you? Can another agent pick up where the first one left off? Or do you have to start again, teaching it everything from scratch?
We’ve already been through versions of this with social networks, photos, digital music and messaging. But an agent’s accumulated understanding of you will be far more valuable, and far more difficult to reconstruct than a collection of playlists or social media posts.
This is why data portability alone won’t be enough.
We also need context portability.
And ideally, a separation between the personal context we control, and the intelligence we choose to use.
Because the AI model that serves you best today might not be the one that serves you best tomorrow.
Think about being able to switch between different AI providers, or even use several specialised agents, while keeping your personal memory, preferences and permissions under your own control.
That’s a very different architecture from handing your entire digital life to a single platform and hoping it always acts in your interests.
3. The difference between security and independence
Now here’s where Meta’s technical paper on Muse gets interesting. (Check out Part 1 for a deeper dive into Meta’s technical research on Muse, including how they handle identity, permissions and trust).
Meta has designed a secure computing environment that the agent operates separately from sensitive credentials and permissions.
Its ‘Sentinel’ module acts as an independent layer of control, deciding whether certain actions should be permitted, denied or referred back to the customer for approval.
And this matters deeply. Because the agent can propose an action. But it doesn’t automatically have the authority to execute it.
As I wrote about in Part 2, Meta has designed five different levels of permission, from one-off approvals through to longer-lived authorisations. It’s a smart attempt to separate the intelligence that decides what to do from the controls that determine what is allowed.
And it’s helpful. But it also raises a new and different question.
Who controls the controls?
Because a platform can build an exceptionally secure agent, and enforce every permission correctly, but still have misaligned commercial incentives that don’t always act in the customer’s interests.
I’ve written before about two important types of trust.
Expertise Trust: Will you do what you say you will, safely and competently?
Motive Trust: Are your interests aligned with mine?
It’s precisely what’s happening here with Muse. A personal agent might have excellent Expertise Trust. It always follows instructions, protects credentials, respects permissions and performs tasks reliably.
But what about Motive Trust?
So you ask your agent to find the best insurance policy for your family. It compares the market, checks the exclusions, considers your circumstances and recommends three options. Wonderful.
But if one insurer pays the AI agent provider a commission, does the personal AI get bent out of shape? What kind of recommendation is made? And what if certain businesses now have commercial agreements with each AI platform?
Will the personal agent now simply ignore providers outside its own ecosystem?
And here’s the thing… you might never find out.
Unlike traditional advertising, where we can often spot the sponsored or contrived product placement, agentic recommendations will be woven directly into decisions being made on our behalf.
In other words, the influence might not even be visible.
Woof, we’re not in Kansas any more.
This is the problem of agent bias. Not necessarily an AI telling lies, but an AI deciding which options to consider, which information to exclude, which recommendations to prioritise and what best actually means.
And remember the ‘AI intimacy problem’ from Part 4. The more an agent knows about you, the more precisely it can tailor its recommendations, anticipate your objections and influence your decisions.
So now an agent can be technically secure, correctly authorised and entirely accurate… while still being commercially biased.
That’s a big deal. And is why independence cannot simply mean that an agent operates autonomously. It must also mean that the customer has meaningful control over whose interests the agent serves.
Here’s a simple way of looking at it: When your personal AI agent becomes the most important interface to your digital life, will it be working for you, or will you be working for it?
4. The liability problem (that nobody has solved)
And of course there’s one more snag: Liability.
Much of the emerging debate about agent liability is focused on agentic commerce and payments. Who is responsible when an agent buys the wrong product, makes an unauthorised payment or enters into a transaction the customer didn’t intend?
Some have drawn parallels with self-driving cars, where responsibility for a crash could be divided between the driver, manufacturer and the technology provider.
I suppose for personal agents, the equivalent would be the customer, the AI platform and the merchant.
But I think that’s far too narrow. Payments are only a tiny fraction of what personal agents will actually do.
Your agent might check your bin collection times, manage your calendar, book your car in for a service, challenge a parking fine, or file an objection to a proposed housing development.
No payment required.
Yet any of those actions could cause harm. An agent might miss a deadline, disclose sensitive information, make a false allegation, cancel an important appointment or submit something you never intended.
And the harm wouldn’t be financial. It could be reputational, operational, legal or even personal. It could affect the customer, the business, or an entirely unrelated third party.
So I wonder if we’re asking the wrong question.
Rather than focusing on who is liable for an agentic payment, we need to understand responsibility for an agentic action.
Which brings us to my thinking around ‘OBO’: the On Behalf Of problem.
Folks are getting excited about ‘Know Your Agent’ (KYA). But what we really need is a more complete view of the agent, its context and the identities and reputations of those involved in the transaction.
I call it the OBO 8: the eight things that a personal agent framework can describe:
The identifier of the ‘principal’ (the person, org or machine) doing the delegating
The identifier of the AI agent
Proof of the ‘binding’ between the principal and the AI agent (the relationship)
The scope of the delegation (what, where, when, who)
The intent of the delegated task (why)
The operator of the agent (e.g. SaaS provider vs. device owner)
The reputation of the agent
The terms on which data can be shared about the principal (a pointer to a machine-readable governance framework, and covering compliance with any particular regulations)
Importantly, these ‘OBO 8’ can be wrapped up into a single digital object that can be asked for, inspected and verified instantly. It will be machine-readable, and suitable for high-frequency, real-time transactions, with auditability baked in.
Right now, we don’t konw who authorised the agent. What exactly was it permitted to do. What instructions it received. What action it took. And what happened afterwards.
It’s why ‘verifiable delegation’, auditability and revocation are going to matter so very much. Not just to establish that an agent can act, but to record what happened when it did.
We certainly don’t have all the answers yet.
But there’s a core principle here, if you pay attention:
We cannot give customers the liability for autonomous decisions without giving them meaningful control over how those decisions are made.
And that takes us right back to our walled garden and Muse.
5. The Great Customer Empowerment Test
Look at the pattern running through the history of personal technology.
The personal computer gave individuals computing power that previously belonged to large organisations. The mobile phone put calls and information in everyone’s pocket. Apps gave us thousands of specialised tools to manage different parts of our lives.
And now personal AI agents promise something different again. More than just giving people access to new digital tools, we’re beginning to give them tools that can act.
For the first time, individuals will have persistent, smart digital representatives working across the businesses they deal with every day. Helping them understand their options, identify opportunities, protect their interests and coordinate increasingly complex parts of their lives.
I’ve been writing for years about this shift: from business-controlled customer experiences towards customer-controlled experiences.
About ‘Empowerment Tech’.
All grounded in the idea that the only genuine 360-degree view of the customer belongs to the customer themselves.
Personal agents will finally give individuals the ability to stitch together their fragmented digital lives, manage their relationships with organisations, negotiate better outcomes and make more informed decisions.
But only if we get the foundations right.
We need portable personal context. Interoperable services. Verifiable identities and delegated authority. Transparent commercial relationships. Meaningful customer control over permissions, memory and decisions.
And critically, the ability to leave.
What good is a personal agent that understands everything about you, if you’re effectively trapped inside the platform that provides it? What good is an agent that can negotiate on your behalf, if you can’t see the interests that influence its recommendations? And what good is autonomy, if you carry the liability for decisions you couldn’t meaningfully control?
Meta’s Muse is an important signal of where the market is heading. Its technical architecture shows how seriously the industry is beginning to take secure execution, permissions and agent autonomy.
But the questions it raises are much bigger than Meta. The next great competition between technology companies won’t be about who builds the smartest AI.
It’ll be about who controls the personal context, permissions and relationships that make those agents useful.
And whether individuals can ever take those things elsewhere.
Thanks for coming on this five-part journey with me. In today’s blizzard of newsletters, hot takes and AI-ridden social media posts, I know your time and attention is precious.
So I really appreciate the time you’ve taken to get this far.
Here’s my closing thought:
The next generation of personal agents will be judged by how smart they are. But perhaps we should judge them by how independent they allow us to become.
Because there’s a difference between an agent that knows you, an agent that can act for you, and an agent that genuinely works for you.
That’s the real customer empowerment opportunity.
And that’s the future I hope we’re building.
Thanks for reading this week’s newsletter.
If you’ve enjoyed it, and want to learn more about the future of Personal Agents, digital customer relationships and customer engagement, then why not subscribe: