Mark Zuckerberg’s unbelievably bleak AI vision

Of all the many famous Steve Jobs stories that tech industry folks like to share, perhaps the single most famous is his 1983 pitch to then-Pepsi president John Sculley to join Apple: “Do you want to spend the rest of your life selling sugared water or do you want a chance to change the world?”

Like many things Jobs said, the pitch was wildly arrogant, self-important and self-aggrandizing, but ultimately correct. What Sculley did at Apple (mostly after firing Jobs) to sell the Macintosh and popularize personal, graphics-centered computing changed the world more than his invention of the Pepsi Challenge had. There really was a huge difference between selling Macs and selling sugar water.

After listening to Meta CEO Mark Zuckerberg lay out his vision of how AI “superintelligence” would change the world, though, my main reaction was: man, this guy just wants to sell us sugar water.

“Personal superintelligence”? Maybe just “superintelligence,” it’s cleaner

In an Instagram video (of course) posted last week, Zuck explains that Meta’s goal is to develop “personal superintelligence for everyone,” accessed through devices like “glasses that can see what we see, hear what we hear, and interact with us throughout the day.”

“A lot has been written about the scientific and economic advances that AI can bring,” he noted. “And I’m really optimistic about this.” But his vision is “different from others in the industry who want to direct AI at automating all of the valuable work”: “I think an even more meaningful impact in our lives is going to come from everyone having a personal superintelligence that helps you achieve your goals, create what you want to see in the world, be a better friend, and grow to become the person that you aspire to be.”

The main reaction to this pitch I’ve seen from smart AI observers is: are you kidding? “Superintelligence,” by definition, means a system that performs better than a human, sometimes vastly better, across most if not all domains. And the most ambitious thing Zuck can think of to make with that is… VR glasses? As Fortune’s Sharon Goldman put it, while Steve Jobs called his computers “a bicycle for the mind,” “Zuckerberg, by contrast, imagines superintelligence as a pair of Ray-Bans that help you…be a better friend?”

The lack of ambition in Zuckerberg’s rhetoric is all the more striking when one considers the extreme ambition of his spending on AI. This year alone, he’s hired former GitHub CEO Nat Friedman and veteran AI founder Daniel Gross; Scale AI CEO Alexandr Wang (as part of a quasi-purchase of Scale, a massively important company whose training data is used by just about every AI company); Apple AI chief Ruoming Pang; and ChatGPT co-creator Shengjia Zhao, among several others.

His hiring spree, and the gargantuan amounts he’s willing to pay top talent, have roiled the sector for weeks now. At one competitor (former OpenAI CTO Mira Murati’s Thinking Machines Lab), Zuckerberg reportedly sent offers to more than a dozen of the company’s 50 staffers, one of which was for over $1 billion over a few years, while the rest ranged from $200 million to $500 million over four years.

Even for a company as rich as Meta, billion-plus offers for talent are unheard of. ($1 billion is how much Zuckerberg paid for all of Instagram in 2012.) It’s a pretty vivid sign that Zuckerberg sees AI as the future of his business.

But what does that future look like?

I’ve seen the best digital minds of my generation wasted on Reels

One could, very charitably, reason that Zuckerberg knows that a world of vastly superintelligent AI systems would lead to massive, far-reaching social ramifications that are not adequately summarized as “you get smart sunglasses,” but he has concluded that most of his investors and customers aren’t in a place to understand the gravity of those changes. Hence, talk about glasses.

That could be what’s happening, and I have some sympathy for his position if so. Trying to game out what a post-superintelligence world looks like is in fact extremely difficult, not least to those of us limited to mere human intellects. And it’s usually scary — even if the changes ultimately prove positive.

For all the uncertainty, there is no plausible world where people have access to “personal superintelligence” and they and businesses do not use that to automate huge numbers of tasks, and there are a number of conceivable scenarios where that leads demand for human wage labor to totally collapse. Other scenarios see wages skyrocket. It’s a tough situation for a CEO to message.

But it’s also worth considering the possibility that Zuckerberg means exactly what he’s saying: that the AI systems his team is building are not meant to automate work but to provide a Meta-governed layer between individual human beings and the world outside of them. Facebook and Instagram are, in a sense, very crude versions of that layer, synthesizing and compressing the outside world into a digestible and addictive form people can consume throughout their days, and Zuckerberg’s earlier obsession with the metaverse seemed a logical continuation. This approach has been immensely profitable. (Though, not so much the metaverse.) Imagine how much more profitable it’d be if a digital mind much smarter than Zuck’s was designing it.

Conversations like Zuckerberg’s with the business writer Ben Thompson in May give credence to this interpretation. Zuckerberg sees four opportunities with AI: improving his products’ recommendation algorithms to better target advertising, driving greater engagement on “consumer surfaces” like Instagram Reels, “business messaging” (i.e., businesses doing transactions through WhatsApp and Messenger, using AI), and lastly direct AI use à la ChatGPT.

The promise of AI, to Zuckerberg, is that it can help him sell you more ads and convince you to spend more time watching Instagram brainrot. My reaction to that pitch was the same as AI writer Zvi Mowshowitz’s: “It was like if you took a left wing caricature of why Zuckerberg is evil, combined it with a left wing caricature about why AI is evil, and then fused them into their final form. Except it’s coming directly from Zuckerberg, as explicit text, on purpose.” At least the sugar water from Pepsi tastes good.

That the sixth largest company on Earth is devoting billions of dollars toward this vision is not, y’know, great. But it has a silver lining.

One thing I’ve learned from talking to AI researchers over the years is that most of them are driven by a conviction that this thing they’re building is really, really socially important. Sometimes that comes with a safety tinge (“this thing could kill us, and we need to make it so it doesn’t”), sometimes with an accelerationist tinge (“this thing could liberate mankind from economic scarcity”), but either way it’s usually stated with real conviction. If they only wanted money they’d go work for a hedge fund. But they also want to build something they’re proud of.

That character trait will, I think, cause the “throw money at smart people until they all join” strategy that Zuckerberg is attempting to fail. If superintelligence is built, it will be built by a team that is productive due to a passionate, shared, optimistic vision for what a world with superintelligence will look like. It will be made because its makers want to change the world, not sell sugar water. A team of researchers joining primarily for the money, under a leader whose boldest vision is “what if we sold more ads on sunglasses,” is not going to make it.