Fridman and Zuckerberg record world's first Metaverse podcast

The Lex Fridman Podcast, hosted by Fridman, an AI researcher at the Massachusetts Institute of Technology, usually takes place in a face-to-face conversational format. And in his latest podcast episode starring Meta CEO Mark Zuckerberg, it seemed like the two were in the same room but not quite. 

Elatus (excellent communication articulate, precise (interesting that such intimacy of conversation can be achieved remotely)) conjunct Haumea (able to reconstruct, redefine, transform, especially regarding consciousness, portraits and or caricatures (codec avatars are a deep generative model of 3D human faces that achieve state-of-the-art reconstruction performance)). Mercury (speaking, self-expression of all kinds) trine Arrokoth (far-reaching, transcends borders, details (”breaks all barriers, geographic physical barriers”, sitting miles apart, Fridman and Zuckerberg’s avatars interacted with each other as though sitting in the same room, captured subtleties in expressions and showed minute details)), 

Sitting miles apart, in what was possibly the world’s first interview in the Metaverse – a virtual reality-based social media platform – Fridman and Zuckerberg’s avatars interacted with each other as though sitting in the same dark room. The real-life Fridman and Zuckerberg wore Meta Quest Pro headsets. All was possible due to Meta’s "codec avatars" developed by the company’s Reality Labs.

What are codec avatars?

Codec avatars are a deep generative model of 3D human faces that achieve state-of-the-art reconstruction performance. Both Fridman and Zuckerberg underwent extensive scans of their faces and their expressions, which were used to build computer models. The headset then helps transfer the real-time expressions that the user makes to the computer model. The outcome was super-realistic faces that don’t look cartoon-like.

“The realism here is just incredible… this is honestly the most incredible thing I have ever seen,” said Fridman.

In the interview, one can see side-by-side comparisons of each process. The three views involve a real Fridman/Zuckerberg wearing the headset, along with a rendered version of their faces and then the output generated.

A YouTuber wrote. “They should have recorded this as something that we could experience in the meta verse. I think it'd be dope to be able to basically be sitting there with them.”

Smallest of expressions captured

The generated output was impressive, for it captured subtleties in expressions and showed minute details like the 5 o'clock shadow on Fridman’s face, the freckles around Zuckerberg’s nose, and the crinkles around eyes.

Soon, Zuckerberg hinted, people will be able to create similar avatars using their phones. The user would be required to form a couple of sentences, make varied expressions, and wave the screen in front of their face for a couple of minutes to complete the scan. He talked about a similar project that Meta is working on, which would not require the tremendous amounts of scans that the current project requires.

Zuckerberg had earlier revealed a cartoonish metaverse avatar last year, for which the Meta CEO was trolled heavily on the internet. Users likened the graphics in the avatar to dated graphics in old video games and called the avatar ‘dewy-eyed’ and ‘creepy.’ Anticipating memes from the podcast, Fridman said, “That’s gonna be the meme, that the two most monotone people are in a metaverse together.”

Integration of AI and Metaverse?

Apart from Metaverse, the two discussed Meta’s new mixed reality headset Quest 3, artificial intelligence in the Metaverse, large language models, and the future of humanity.

Speaking about how Metaverse could be integrated with artificial intelligence, Zuckerberg said someone like Fridman could create an AI version of themselves so that his community of listeners could speak to him even after he dies. 

He also hinted that this technology won’t be only limited to Metaverse but will be pushed soon to other platforms like Instagram, WhatsApp, and Messenger. 

Minor planet keywords developed by Philip Sedgwick, used with permission http://philipsedgwick.com

Source: /interestingengineering.com/