Inside Google’s augmented reality reset

At its IO developer conference in May, Google turned heads when it showed off a demo of Project Astra, a real-time AI assistant that could see and perceive the world around it. It could understand jokes, read code, and even remember where the user left their glasses. At one point, the user switched from using Astra through their smartphone to running it on a pair of augmented reality spectacles.

The show-stopping moment led to some speculation that Google may have resurrected its Iris smart glasses, which we reported had been shelved last year. But the demo was merely a concept to whet appetites for what Google’s AI could do in a glasses-like device.

For Google, the AI boom’s arrival is perfectly timed for an AR revival. Internally, the company has been consolidating efforts to move faster, hoping to spark partnerships for a range of head-worn devices. At the same time, it faces an uphill battle in convincing customers, partners, and employees that it’s prepared to make it work.

Google has made several false starts in Extended Reality (XR) — an umbrella term for virtual, augmented, and mixed reality devices — but it knows these sorts of wearables are a logical home for the AI features it is developing. Google cofounder Sergey Brin even called glasses the “killer app” for AI.

Internally, Google teams are heavily focused on Android XR, a new software that Google hopes hardware partners will want to build devices on.

Meanwhile, Google and Samsung partnered on a headset that will be powered by Google’s new software, but leaders, fearing the device isn’t yet good enough to dazzle users, have continually pushed it back, according to people familiar with the timeline. The launch has been delayed to Q1 of next year, they said.

Tech giants are now setting the stage for an arms race as they build AR platforms and lure partners onto them. We spoke to half a dozen insiders who revealed how Google is regrouping on AR and the challenges it faces. These people, whose identities we have confirmed, asked to remain anonymous they were not permitted to speak to the press.

A Google spokesperson declined to comment on the details of this story.

Google senior vice president Rick Osterloh.

In April, Google merged its devices and Android groups. It put most of its AR and VR teams under Shahram Izadi, a former Microsoft director for HoloLens who has worked on AR and XR efforts at Google since 2017.

Last month, Google further shuffled the ranks to bring more teams in its devices and Android group closer together on the org chart. That included moving a team that has been developing new optical AR technology together with the groups working on Android XR under Izadi, according to two people familiar with the move. The changes could help the company move faster and more effectively.

Hugo Swart, formerly Qualcomm’s head of XR, joined Google’s AR group in April to lead ecosystem strategy and partner relationships.

“He’s very much about relationship management, about how you partner well,” said a person who previously worked with Swart and highlighted his move as significant for Google’s AR ambitions.

At the same time, Google continues to see a stream of exits following layoffs and leadership reshuffles. Jeffrey Margolis, a principal AR architect at Google who previously helped Microsoft build its HoloLens headsets, left the company last month for Meta to work on future AR glasses, according to a person with direct knowledge of the move.

Some other Google employees have also defected to Meta in recent months, that person said, adding that some of these moves happened due to frustration with Google’s constant pivoting and the fact it laid off employees working on AR back in January.

Mark Lucovsky, a senior director on Google’s AR software, left last July, citing “changes in AR leadership and Google’s unstable commitment and vision” to the product. Paul Greco, who oversaw the optical technology Google acquired with the startup Raxium in 2022, also left Google for a job in Meta’s XR group.

Barry, Betty, Harriet

Google has made encouraging progress with the Raxium optical technology, according to a person who saw a recent demo. It built the technology into a collection of internal development software used for building and showcasing Android XR, each with monikers such as “Betty,” “Barry,” and “Harriet” (within these names are buried clues about their abilities; Harriet includes the letters “et” because the prototype has eye tracking).

Such leaps of progress could help the company eventually reboot a first-party offering. For now, it is focused on landing partners like Samsung and Qualcomm to put Android XR on various types of headsets, from fully immersive to more glasses-like devices.

“Google is out of the first-party game for glasses for the foreseeable future,” said one current employee. “We’re only making prototypes to convince people to use our software.”

Some current and former employees speculated that Google’s decision not to build its own hardware offerings could help convince partners it won’t be playing second fiddle to any of Google’s own glasses.

“Doing anything first-party is a challenge relationship-wise,” said a former Google AR team member. “That leads you to compete with your partners. It’s a sensitive topic.”

Executives onstage at Samsung Unpacked in 2023

Google and Samsung are still building a joint device—a mixed-reality headset more akin to Apple’s Vision Pro, internally codenamed Moohan. The device, announced in February 2023, was initially planned to launch in Q1 of this year but has been pushed back. According to a person familiar with the internal timeline, a consumer version is now set for next March, with a developer version planned for release this October.

There is internal speculation that Samsung will be the first to also build glasses with Google’s AR software, according to two employees familiar with the matter.

Google is running prototypes of Moohan and other devices on chips built by Qualcomm, which it hopes will prove a key strategic partner as it ramps up its renewed efforts.

Employees have been told that Google expects Qualcomm to fully support Android XR and that it may even move away from its own Snapdragon Spaces development platform in favor of Google’s, according to an employee who heard the remarks from Google leaders.

A Qualcomm spokesperson pointed B-17 to the company’s senior XR lead, Said Bakadir, recently announcing that Spaces would be “forward-compatible” with Android XR, but declined to comment on plans beyond that.

Google’s XR graveyard

It’s been over a decade since Google Glass was launched. The device, which sold poorly and became something of a laughingstock for the company, is regarded as a symbol of peak Silicon Valley hubris (you’ll spot a pair on display in the touring Museum of Failure).

After licking its wounds, Google tried a more immersive virtual reality headset named Daydream and opened the software for partners to build on. Once again, it failed to resonate with consumers, leaving the initiative to slowly die off.

It later started work on Project Iris, a Google Glass reboot that looked more like regular spectacles and performed tasks like live translation. Last year, the company put Iris on ice, unconvinced the tech was good enough to shake off the specter of Google Glass. This caused frustration among Google employees who watched rival companies leap further ahead.

As Google kept zig-zagging, Apple forged ahead with the Vision Pro, while Meta launched several virtual reality devices and Ray-Ban spectacles that record video and play sound.

Without a device of its own and with its AR software still in development, Google now finds itself on the back foot. The company has been selling Android XR to partners and hopes the arrival of AI could help it reenter the market. Still, its inconsistent commitment to the space could make it difficult to convince partners to go with Google over building their own software offerings.

Google cofounder Sergey Brin wearing the Google Glass. 

There is also a lingering question over whether AR developer software like Android XR makes sense. One former Google employee said Android XR “could potentially unify this fragmented landscape and give developers a common platform.”

Apple, Meta, and others have so far chosen to build their own operating systems. Meta, with its Horizon OS platform, has said it plans to partner with companies such as Lenovo and Microsoft.

Google held talks with Meta to go all-in on Android XR, but Meta rejected the proposal because it already had its own platform, according to a person familiar with the matter. The pushback underscores how Google came late to the party.

Anshel Sag, principal analyst at Moor Insights & Strategy, said the company’s constant leadership changes and strategic pivots have let Google abdicate the market to rivals, although it could be an “enabler” if it can convince partners and original equipment manufacturers that Android XR is here to stay.

“Google has burned a lot of bridges with developers and OEMs,” said Sag. “I think it’s going to be very challenging for them to easily re-enter the market without some assurances that it won’t disappear from the market once again.”