Zuckerberg wants to personalise your digital future - but will it be yours?

The next decade will define whether AI actually empowers people or just the platforms

Hi everyone, thanks for coming back to Customer Futures.

Each week I unpack the fundamental shifts around Empowerment Tech. Personal AI, digital wallets, digital ID and customer relationships.

If you’re reading this and haven’t yet signed up, why not join the growing number of executives, entrepreneurs, designers, regulators and other digital leaders by clicking below.

To the regular subscribers, thank you.

---

Hi folks,

Mark Zuckerberg just pitched a future where everyone has their own AI.

It sounds empowering. But it might be the most important power grab of the decade.

This week, we dive into what Meta calls ‘Personal Superintelligence’. It’s a bold, polished attempt to frame the AI agent future as empowering, human-centred, and optimistic.

It’s filled with interesting and important points, but there are some massive gaps. Especially around how he thinks about ‘empowerment’.

So I’ve taken Zuck’s post and added some Customer Futures commentary.

Pointing out some of the critical pieces he’s missed out (or more likely chosen to ignore). Like personal data control, customer agency and the inevitable bending towards shareholder interests.

Because digital empowerment is going to need the right infrastructure, not just a great UI.

This is an important piece of writing, from an important person, at an important time. We can’t walk past it without trying to understand what Zuckerberg is doing, where he’s taking us, and what it’s going to mean for everyday people.

We dive into

• The promise of Personal Superintelligence

• What Meta leaves out

• The illusion of control

• What’s Meta’s role, really?

• Surveillance with a friendly UI

• The decisive decade

You can read the original post here.

Now grab a comfy corner, your favourite summer smoothie, and Let’s Go.

---

The promise of Personal Superintelligence

Zucks opens with a bang:

“Over the last few months we have begun to see glimpses of our AI systems improving themselves. The improvement is slow for now, but undeniable. Developing superintelligence is now in sight.”

Whilst it’s more of a dampened view of ‘AGI’ than Sam Altman would have us believe, there it is. Self-improving AI systems are coming soon apparently. Another voice joins the choir.

But then he lands the big question:

“It seems clear that in the coming years, AI will improve all our existing systems and enable the creation and discovery of new things that aren't imaginable today. But it is an open question what we will direct superintelligence towards.”

Yes, it’s an open question. But it’s not just one for policymakers or technologists. It’s one each of us will need to face.

What Meta leaves out

So now we need to ask some other important questions that Zuck’s leaves out:

• Who gets to decide? Who governs that trajectory?

• Will this superintelligence be aligned with our values? If so, whose values? And whose objectives?

• Whose data is fuelling it?

• What’s Meta’s role, really?

He continues:

“In some ways this will be a new era for humanity, but in others it's just a continuation of historical trends. As recently as 200 years ago, 90% of people were farmers growing food to survive. Advances in technology have steadily freed much of humanity to focus less on subsistence and more on the pursuits we choose.

“At each step, people have used our newfound productivity to achieve more than was previously possible, pushing the frontiers of science and health, as well as spending more time on creativity, culture, relationships, and enjoying life.”

He’s using a clever and familiar framing here: technology sets us free.

But we must be careful. History teaches us that with each wave of new productivity comes new dependencies.

What will those be in this new AI era? From a Customer Futures point of view, the biggest risk is that individuals trade control for convenience.

Remember, true empowerment isn’t just more free time. It’s also having the freedom to choose the infrastructure that supports us.

Let’s keep going:

“I am extremely optimistic that superintelligence will help humanity accelerate our pace of progress. But perhaps even more important is that superintelligence has the potential to begin a new era of personal empowerment where people will have greater agency to improve the world in the directions they choose.”

“A new era of personal empowerment”. Now we’re talking. And ‘greater agency’ is right on the money. But we must distinguish between agency as an outcome and agency as an input.

If the AI helps you pursue your goals, but you don’t control the system that gets you there, then it’s not really your agent. It’s not really working for you. It’s more like personalised automation.

Yes, the automated taxi can take you there. But who is behind the steering wheel? Is there even a steering wheel?

---

---

The illusion of control

He goes on:

“As profound as the abundance produced by AI may one day be, an even more meaningful impact on our lives will likely come from everyone having a personal superintelligence that helps you achieve your goals, create what you want to see in the world, experience any adventure, be a better friend to those you care about, and grow to become the person you aspire to be.

This is beautifully worded… but notice the quiet but clever switch. From agency to ‘assistance’. The AI becomes a coach, a creator, a friend. And soon, likely your filter for reality.

If the infrastructure is owned and controlled by someone else - in this case Meta - this is more like a dependency in disguise.

But now he gets to the juice:

“Meta's vision is to bring personal superintelligence to everyone. We believe in putting this power in people's hands to direct it towards what they value in their own lives.

This is his central idea. That it’s now Meta’s role to empower everyone to have their own AI. But it raises a critical question:

Do I actually own the intelligence? Or will I just use it under license?

Unless these new AI models are running in a way and place I can choose and control, unless the data is private by design, and unless the outputs are portable… this is a just another 3rd party service, not really Personal AI.

Which is fine. Look, I’m not saying I need to run these AIs on a private server under my desk. But if this is a service, we need to be clear about who’s managing the infrastructure. Who else can see the data. And who else benefits from the data.

It used to be about giving your consent so that a company could process your data for a specific task. Perhaps so they can put ads in front of you.

But this is new territory. It’s about helping you ‘achieve your goals, create what you want to see in the world… and grow to become the person you aspire to be.’

Let’s be clear-eyed, shall we? About what’s at stake and therefore who’s running the show.

What’s Meta’s role, really?

Next, he attempts to create a competitive edge. Because, of course, it’s all about the individual, not those dastardly villains trying to control the future of work from the centre:

This is distinct from others in the industry who believe superintelligence should be directed centrally towards automating all valuable work, and then humanity will live on a dole of its output. At Meta, we believe that people pursuing their individual aspirations is how we have always made progress expanding prosperity, science, health, and culture.

It’s a smart move. Zuck is positioning Meta as the individualist alternative to the OpenAI/Altman vision.

But look closely and you’ll see that both the Meta and OpenAI models have a similar flaw: they concentrate control at the top. If we want to understand true empowerment, it’s not just about things being personal. It’s about them running on permissionless personal infrastructure.

In the same way that the web and the internet before it were designed. Peer to peer, permissionless, and without monolithic data platforms in the middle.

Let’s keep going:

“The intersection of technology and how people live is Meta's focus, and this will only become more important in the future.

“If trends continue, then you'd expect people to spend less time in productivity software, and more time creating and connecting. Personal superintelligence that knows us deeply, understands our goals, and can help us achieve them will be by far the most useful.”

Well, this is the kicker.

Zuck wants to know us deeply, understand our goals, and help us achieve them.

Quick question. This superintelligence is going to need a LOT of personal data. Much of it is highly sensitive (think live location, financial profile, health information, not to mention your hopes and dreams). WAY more than Zuck already has in the META vault.

So where is all that personal data going to be stored? And who else has access?

What’s useful for the individual might also be useful for the platform.

Surveillance with a friendly UI

Then he drops another superintelligence grenade:

“Personal devices like glasses that understand our context because they can see what we see, hear what we hear, and interact with us throughout the day will become our primary computing devices.”

Now he’s getting specific. About how we’ll feed these new superintelligent platforms with data.

Context data. Personal data. Your data.

Let not forget: glasses that “see what we see” are only empowering if the data flow is under our control. Otherwise, we’re just building new surveillance infrastructure with a friendly UI and convenient pop-ups about the nearest bakery.

Hooray for empowerment!

But Zuck, what about the risks?

“We believe the benefits of superintelligence should be shared with the world as broadly as possible. That said, superintelligence will raise novel safety concerns. We'll need to be rigorous about mitigating these risks and careful about what we choose to open source.”

‘Novel safety concerns.’ That’s a cute way to put it. And notice more clever positioning. ‘Openness’ is now framed as a risk to be managed.

It seems Meta will share some of the AI capabilities openly. But the real question is whether the structure of the system is open. Its logic, its incentives, its training, its interfaces. That’s what makes real empowerment possible.

But there’s more:

“Still, we believe that building a free society requires that we aim to empower people as much as possible.”

Agreed. But empowerment without control is a façade. Facebook has made the world ‘More Open And Connected', right? We need wallets we own, agents that work for us, data we can port, and systems that don’t default to surveillance.

Empowerment must start with infrastructure, by design.

The decisive decade

This next part I can get behind:

“The rest of this decade seems likely to be the decisive period for determining the path this technology will take, and whether superintelligence will be a tool for personal empowerment or a force focused on replacing large swaths of society.

Great. But what’s Meta’s role?

“Meta believes strongly in building personal superintelligence that empowers everyone. We have the resources and the expertise to build the massive infrastructure required, and the capability and will to deliver new technology to billions of people across our products. I'm excited to focus Meta's efforts towards building this future.”

Zuckerberg is right on one thing. This is the decisive decade. But we shouldn’t confuse resources and reach (which he has) with permission (which he doesn’t).

Meta may deliver ‘superintelligence’ to billions. But unless it’s truly personal, truly portable, and truly private, then we’ll just be users again. (This is your regular reminder that the only industries that call people ‘users’ are software developers and drug dealers).

If we want a future of real Personal AI, of real superintelligence, we need more than grand promises. We need:

• Local agents - on device, or cloud environments, we can see and control

• Verified, portable data

• Digital wallets we own

• Interoperability and open standards

• Infrastructure that doesn’t ‘Phone Home’ by default

• And personal data policies that put people in charge

Superintelligence should empower the person, not just the platform. And the real test isn’t how powerful the model is. It’s whether you have agency and choice.

I’ll be honest, I’m not optimistic about this one, folks. In the end, Personal AI must answer to us. Not the platforms that claim to empower us. And Meta’s track record speaks for itself.

Even if Zuck can turn on superintelligence for billions of people. I’m just not sure I want to be one of them.

---

And that’s a wrap. Stay tuned for more Customer Futures soon, both here and over at LinkedIn.

And if you’re not yet signed up, why not subscribe: