Hi everyone, thanks for coming back to Customer Futures.
Each week I unpack the disruptive shifts around Empowerment Tech. AI Agents, digital wallets, Personal AI in and the future of the digital customer relationship.
If you haven’t yet signed up, why not subscribe:
Hi folks,
From the sublime to the ridiculous this week.
First off, we all knew it was coming. But I didn’t think it’d be here this fast.
This is a ‘hand with six fingers’ moment for real-time directed video. Where you can type any instruction into a prompt, and the next scene in the film is what you want.
Science fiction? It’s now here.
Given that these technologies are compounding, and these digital video capabilities are doubling every 9-12 months, you can imagine what will be available at the end of 2027.
Next, the security world fell apart with the latest open-weight LLMs being jailbroken. You need to be paying attention to Abliteration. Now anyone can get around the safeguards of the public models.
You can now demand that an AI does what you want. Not what the authors, owners, or the general public (and indeed lawyers) demand.
If that doesn’t make you feel all very Black Mirror, then grab a coffee and pour over the latest piece by Jakub Pachocki, the Chief Scientist at OpenAI. He’s not very optimistic about governing AI once it gets very much smarter. His piece is, uncomfortably, called ‘An Alien Mind.’
“I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.”
There’s a lot to say about OpenAI and its own hype machine, but this isn’t Altman on breakfast television. It’s the Chief Scientist saying that as models get smarter, it gets harder to track their logic and logging.
On this one, I don’t think this is hyperbole.
“Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established.”
Then, just to put a bow on the week, we’ve had a wave of Personal AI jazz hands:
Apple updates Siri with AI, bringing some interesting Personal AI stuff (yes, privacy privacy privacy, on device, access to messages and calendar etc.).
Meta launches a new Personal AI ‘Muse’, which is making waves (all the stuff you’d expect, bookings, calendar etc… but the twist is that a) they have billions of users already and b) they have Instagram feeds (so you can turn that social post into a recipe, create the shopping order, and pay with a credit card) and c) they have WhatsApp for Business (one tap on the newsfeed post about shoes becomes a WhatsApp message to contact the business directly… and Muse turn that message into an agentic request, and the shoes arrive the next day).
New Silicon Valley darling ‘Instinct’ explodes onto the scene, but hits the skids around passwords and credit cards (predictably, as TechCrunch notes, they have “A broad, perpetual and irrevocable license to “access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify” any of the user’s materials, including for training its AI models”. They also capture screens, cursor movements, and keyboard inputs…. ah).
Then enter state left, Stripe brings their ‘Link’ service to the party, a data store for your cards and passwords, with a pretty neat UX. Everyone is scrambling to make this stuff trusted, safe and scalable.
It’s all fun and games, if you are into this sort of thing (which I’m assuming you are, if you’re reading this far).
So into the new digital economy machine we march on.
More and more frothy tech announcements. A couple of fresh identity company hacks (100s of millions of identities once again spill onto the kitchen floor). A shiny Apple event. And then there’s the BigTech brotherhood driving with both feet jammed on the accelerator, while we’re in the backseat muttering about the handbrake.
And it’s only Thursday, folks.
All the more reason to start paying attention to the future of the digital customer.
So welcome back to the Customer Futures newsletter.
In this week’s edition:
Sugar lumps, tonnes of CO2 and A4 paper - we need a new way to look at AI agents
AI efficiency can’t be the only goal
We are in a real bind with AI Agents
… and much more
Let’s Go.
Sugar lumps, tonnes of CO2 and A4 paper - we need a new way to look at AI agents
It all started with a post about how much sugar is in chocolate spread.
This company is using sugar lumps, not ‘grams of sugar’ to make the point.
The very excellent Rory Sutherland jumped in on social to agree:
“If you think talking about grams of sugar is silly, think how dumb it is to talk about “tons of carbon dioxide”. Talking about gas by weight may be scientifically accurate, but it’s fundamentally incomprehensible to any normal person.”
Which got me wondering. Aren’t we doing the same with personal data and consent?
We’re describing the ‘risks’ and ‘consent’ and ‘terms’ and ‘data processing’ like we’re talking aobut grams of sugar. Nobody really understands what any of the words mean, nor the impact of making the ‘wrong’ choice.
So I wonder what the equivalent of a ‘sugar lumps image’ could be for personal data. What really happens when you click ‘I accept’.
Do folks really know how much ‘sugar’ (data) is in a digital ad, and what happens to that blob of personal information when it travels across the Real Time Bidding (RTB) network?
Johnny Ryan calls it the biggest data breach in history. And it happens billions of times a day.
If we can show people how many sugar cubes are in Nutella, could they be able to see how much personal data is in Facebook ad?
And if we think there’s already a problem with the ‘I accept’ confusopoly, just wait until the Personal AI platforms (Meta, Grok, Instinct and many more thousands coming) start telling us about the data they need to make the booking and why they need access to our emails and calendar.
Go read the privacy backlash around Instinct again.
Sugar in grams. Carbon dioxide in tonnes.
Maybe we could show the number of personal data attributes like a stack of A4 paper. Show how many specific data records are involved when you click ‘Consent’. Where each blob of data gets its own piece of A4 paper. This one has 2 pages, this one 17, and this one has 334.
One of your ‘A4 personal data files’ looks as thin as an envelope. Another looks as fat as a Harry Potter book.
We need a new way to talk about - and show people about - their personal data. And quickly, before AI agents make a real mockery of what consented data sharing looks like.
AI efficiency can’t be the only goal
It all feels rather grubby talking about AI in terms of ‘efficiency’.
All the excitement is about how much AI can do, for how many tokens, how fast, how cheap, and what tasks will be replaced. You get to a discussion about job losses pretty fast.
Jory Des Jardins has been concerned about this narrow view of ‘AI efficiency’, and has come up with an excellent framework to describe and govern human agency in the context of AI.
“AI adoption is being measured by one metric: efficiency.
“That metric can’t tell you whether your organization can still catch AI’s mistakes, or whether the people who’ll need to make hard calls in five years are actually building that judgment right now.
I built Human Agency Architecture (HAA) — a governing standard that treats human judgment as infrastructure.
It’s excellent, and worth looking at in detail:
I love this bit (bold mine):
“Where current frameworks measure what AI does to productivity, HAA measures what AI does to people: their decision rights, capabilities, and capacity to flourish.
“It doesn’t ask organizations to choose between productivity and human capability; it gives them a way to pursue the productivity gains they’re already chasing while protecting the judgment those gains ultimately depend on.”
With a Customer Futures hat on, you can see that this applies to the customer side, the citizen side, too. What does customer agency look like when we are automating stuff on the side if the individual?
How much of our autonomy will be taken away over time? Will we really have control or oversight of the things our Personal AI does for us? Like switching home energy supplier, or moving bank account, or buying insurance? Will we even care about those ‘commodity’ parts of our lives, or will we just ask the Personal AI to do it - to ‘fire and forget’?
Efficiency can’t be the only goal.
Agency - and empowerment - must be central to how we apply AI to our lives.
Recommended reading.
We are going to be in a real bind with AI Agents
Why do detectives at a crime scene collect fingerprints? It’s to ‘bind’ the person to the place. And why do bank robbers wear balaclavas? To avoid ‘binding’ their face to the scene.
If you want to borrow a large amount of money most banks will ask for two different sources of identity, both with address, and at least one with a photo ID. It’s so they can match the ID documents to your physical face.
It turns out that identity is all about binding.
A couple of years ago, a man in Mumbai took a single identity photo, and using different angles applied for multiple SIM cards.
Thousands of them.
“A total of 8,500 SIM cards were activated across Mumbai Metropolitan Region, using 62 different persons’ photos. While the names on Aadhaar were different, a single accused’s photo was used across documents.”
Why did the telcos not spot this fraud initially? Because those identities - those photos - weren’t bound to a real person.
Hacking and data breaches happen so frequently simply because our personal data is worth going after - it’s useful elsewhere. When bad actors present your stolen data to a business, for example to access your account or to open a new one, few companies check the binding of that data to a real person.
Most fraud happens because we don’t get the binding right.
Of people to companies. Of data to people.
If a business only needs to ask for my identifiers (mother’s maiden name, account numbers, date of birth, name of pet)… then anyone who has those pieces of information can pretend to be me.
Personal AI is going to make things much worse. And digital fraud and cyber attacks are already exploding.
For two reasons.
First, we’re feeding AI platforms vast amounts of personal data. Our financial transactions. Our social activity. Our contexts and needs. Our searches and interests and habits. But also our voices and faces.
In the wrong hands, these AIs will be able to create rich social media histories about you. Plausible online reputations. They can already generate life-like versions of your government documents, and your voice, in real time.
Without any ‘binding’ to the real you or your devices, that data can be presented to any business, anywhere online, and become a pretty realistic version of you.
But the second reason is a bigger deal.
As AI agents start managing your calendar, messages and interactions with businesses and other people. Grok Bot, Muse, Instinct are all sorted to be the next big thing in Personal AI.
But unless we can bind your Personal AI - your authorised agents and personal data - to the ‘real you’, we have a huge problem on our hands.
We won’t just need to prove that it’s you. We will need to prove that it’s your AI agent, your own bot, acting for you.
I’ve always liked the idea of ‘No data about me, without me.’
Without foundational digital identity infrastructure, available across sectors and countries, we’re going to find it difficult to manage the good (manage my calendar, email, messages and payments) from the bad (fake AI opening accounts, watching ads etc. in my name).
AI agents are not enough.
We need the full Trusted Agents triangle:
Delegation AND trust AND context.
And critically, we need the binding between them.
How will that work exactly? Well it’s being worked out. And it’s becoming clear that cryptography will play a big part. Trust registries, verifiable credentials, ZCAPs, UCANs and all the rest.
But we also need to bind these digital objects to the individual they belong to. Only then will we be able to trust it. Only then will we tell the real from the fake.
For example, my Personal AI can be bound to verifiable credentials. Those credentials can be bound to my digital wallet. And the digital wallet can be bound to me.
Very likely with a device. Probably your phone. Certainly with biometrics. A face or a finger to approve a transaction. Then that chain of trust - of cryptography - will kick in.
And once companies start checking the bindings - not just the data - we can unleash the true potential of AI agents.
Lower risk. Higher assurance. Lower cost. Higher trust. And more value.
Unless and until we can prove who is who, and what (bot) is what (bot), we are going to find it incredibly hard to trust these new personal agents.
‘No (agentic) data about me, without me’.
We are fast approaching the moment we need to separate out the ‘Customer-Present’ transactions from the ‘Customer-NOT-Present’ transactions.
But it all starts with the agentic triangle. And the binding across all three corners.
Without it, build digital trust from the ground up, we are going to be in a real bind with AI agents.
A longer list of things to track
There are SO many interesting and important Customer Futures things happening at the moment. Here are just a few from my open tabs:
Post: What happens when half of us are wearing smart glasses READ
Announcement: The Federal Reserve, OCC, FDIC and FinCEN issue guidance on the use of government-issued verifiable digital credentials (VDCs) READ
Post: Would you trust Zuck’s agent to manage your life? READ
Analysis: Siri Recap is kind of a big deal READ
Idea: Stripe’s president says the checkout page is dying READ
News: Anthropic launches AI commerce agents with Mastercard, Visa and Accenture READ
And that’s a wrap. Stay tuned for more Customer Futures soon, both here and over at LinkedIn.
And if you’re not yet signed up, why not subscribe: