Customer Futures Stories: The Enshittification of Amazon, connecting AI to my bank account and Zuck's AI agents

Plus: ChatGPT on privacy, JP Morgan on digital ID, and hacking the operating system of human civilisation

May 02, 2023

Hi everyone, thanks for coming back to Customer Futures. Each week I unpack the ever-increasing and disruptive shifts around digital customer relationships, personal data and customer engagement.

This is the weekly STORIES edition. Covering the important, and sometimes less obvious, updates from the market. If you’re reading this and haven’t yet signed up, why not join hundreds of executives, entrepreneurs, designers, regulators and other digital leaders by clicking below. To the regular subscribers, thank you.

STORIES THIS WEEK

They say that being around cutting-edge technology is like living in ‘Dog Years’. Things move 7 times faster than normal human life. But with AI and customer engagement, it feels more like Dog Weeks. There’s so much to track at the moment, let alone make sense of.

Recently I made a prediction: that by December 2023 we’ll see a real-enough looking, but actually fake, video circulate widely online. One that triggers some kind of national-level crisis. But I now believe this will happen within 6 months, possibly sooner. Because last week the Republican National Committee released a campaign ad against Biden using AI-created imagery. All based on future news events, and (they say) the terrible outcomes if Biden was re-elected.

You have to double-take what just happened. Post-truth politics is now AI-enabled and accelerating.

Will customer engagement become post-truth too? Perhaps. Business won’t know what or who to trust. Neither will individuals. Because it all boils down to data. And data just became all about AI.

We must reset our assumptions about customer engagement. Reset the same assumptions about data.

And most importantly, reset digital trust.

We’re going to need to challenge what we see and hear digitally. As I proposed in last week’s edition: Anything that’s not face-to-face should become Digitally Guilty Until Proven Innocent.

Welcome to the future of being a digital customer. And welcome back to the Customer Futures newsletter.

In this week’s edition:

The FTC on AI and the engineering of consumer trust
The Enshittification of Amazon
Assessing Web3 identity building blocks
How to respond to getting hacked
Connecting AI to my bank account
… and my personal health data on my iPhone?
Meta wants to ‘introduce AI agents to billions of people’
ChatGPT gets privacy features
… plus other links from around the web you don’t want to miss

Let’s go.

The FTC on AI and the engineering of consumer trust

This is a big deal. The FTC weighing in on regulating AI. Specifically around customer engagement, bias and persuasion.

“Among other things, it should always be clear that an ad is an ad, and search results or any generative AI output should distinguish clearly between what is organic and what is paid. People should know if an AI product’s response is steering them to a particular website, service provider, or product because of a commercial relationship. And, certainly, people should know if they’re communicating with a real person or a machine.”

This last point is The Big One. Yes, we need to know what or who we’re dealing with. But how?

The answer is to give customers Personal AI: their own digital tools, where they can prove anything about themselves, including proof of their identity and human-ness. And control over who they share things with.

Say it again: Digital wallets (and Smart Agents) will be central to the new digital economy. FTC’S NEW ‘LURING TEST’

The Enshittification of Amazon

For the last few months, Cory Doctorow has been describing one of the most important economic patterns of our time: the ‘Enshittification’ of closed platforms. Not only is this a brilliant take on why we’re in such a digital mess with Web2 platforms. But it helps us think about the impacts of these platforms applying AI to the market. His latest take on Amazon’s decline is a must-read.

“Here is how platforms die: first, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the value for themselves. Then, they die.” ENSHITTIFICATION, AMAZON.

Assessing Web3 identity building blocks

“Web3 is touted to be the next generation of the internet, promising a landscape where individuals not only have read and write capabilities on the internet, but also the ability to own and control their data, including digital creations, digital assets and digital identities.”

JP Morgan might just be the dark horse on digital identity in fintech. Much further ahead on decentralised approaches and self-sovereign identity than most realise. Their second paper on portable digital identity explores new ways for consumers and organizations to interact online. Every day getting closer to realising that digital wallets will become a new 5th channel. JPM PAPER, 5TH CHANNEL

How to respond to getting hacked

As Mike Tyson once said, “It’s all good and well to know articles 33/34 GDPR by heart… until you are hit with a data breach”. Courtesy of the brilliant Masters Of Privacy podcast. A deep dive into how to respond to a personal data breach. From the privacy lawyers that actually deal with the mess. LISTEN

Connecting AI to my bank account

What happens when you outsource your entire personal financial life to GPT4? Start by scanning 10,000+ transactions across your bank, financial statements, credit report, and emails.

Joshua Browder, the CEO of Do Not Pay, just did exactly that. He describes what’s now possible with AI and personal finance, how he’s already $217.85 better off, and how his bots wrote firm but polite letters to contest fees. In the innovation world of crawl then walk then run, these are very clearly baby steps towards the true potential of Personal AI. Tools that work with me and for me (rather than doing things to me). READ

… and my personal health data on my iPhone?

HealthGPT is yet another frontier in the ever-expanding AI landscape. They take Apple’s HealthKit and some other data sources, then overlay an interactive Personal AI. It’s an interesting take on how personalised insights might be deployed.

And indeed a new revenue stream for Apple?

As ever, we don’t just need new answers, but we need new questions. Is the data trustworthy? How does consent work? Where is the learning model? How (and on what) was it trained? How are potentially sensitive and impactful health insights revealed to individuals? Are they shared with others (think insurance)? Is natural language the best interface for making sense of complex areas of our lives like health?

Now that Microsoft and Epic are also looking for trends across our medical records, these innovators will need permission not forgiveness.

Your daily reminder that language models are trained to produce plausibility, not facts. HEALTHGPT, MEDICAL RECORDS

Meta wants to ‘introduce AI agents to billions of people’

What does one do with decades of AI expertise, deep tech resources, (almost) limitless cash and nearly 4BN monthly active users?

New chatbots and ads of course!

“We’re exploring chat experiences in WhatsApp and Messenger, visual creation tools for posts in Facebook and Instagram and ads […] and I expect that a lot of interest in AI agents for business messaging and customer support will come once we nail that experience.” READ

ChatGPT gets privacy features

“We've introduced the ability to turn off chat history in ChatGPT. Conversations that are started when chat history is disabled won’t be used to train and improve our models, and won’t appear in the history sidebar.”

This is part of OpenAI’s official response to the Italian regulator, who shut the service down in the country on privacy grounds. Italy has now lifted the ban, and the privacy pivot seems to have given ChatGPT some breathing space.

For now. Meanwhile, EU regulators are setting up a new task force to investigate ChatGPT more broadly. The regulatory scrutiny continues, and legal experts remind us that"There's no AI exemption to the laws on the books." CHATGPT PRIVACY, ITALY, AI & EXISTING LAW

OTHER THINGS

There are far too many interesting and important Customer Future things to include in this edition, so here are some more links to chew on:

Microsoft Edge is leaking the sites you visit to Bing READ
Klarna Is Not A Buy Now, Pay Later (BNPL) Company - It’s a Customer Engagement Platform Business READ
Are super apps coming to the US? READ
5 Questions to Consider Before Investing in LLM AI-powered Chatbots for Customer Experience READ
AI Drake just set an impossible legal trap for Google READ
AI has hacked the operating system of human civilisation READ (sans paywall here)
NSW Digital ID set to provide people greater control over their online privacy READ

Thanks for reading this week’s edition! If you’ve enjoyed it, and want to learn more about the future of digital customer relationships, personal data and digital engagement, then why not subscribe:

Customer Futures