Li Feifei’s space artificial intelligence startup World Lab launches first product

In January last year, Li Feifei, a well-known artificial intelligence researcher, took a leave of absence from Stanford University to abandon academia and enter the entrepreneurial life. Nearly two years later, her business World Labs launched its first commercial product: World Model Marble. Marble can create 3D virtual worlds from text, images, videos and even rough layouts. It builds on an earlier World Labs prototype that created 3D scenes from 2D images, but had some limitations, such as restricted interaction areas.

So-called world models like Marble are central to Lee’s vision for the future of artificial intelligence because these models can reason about and interact with complex environments, so they are critical to building artificial intelligence that understands not just language but the physical world itself. WorldLab aims to imbue its systems with spatial intelligence, teaching them physical concepts that humans intuitively grasp, such as parking without hitting a curb, catching a thrown object or pouring a drink without looking.

“Today, leading artificial intelligence technologies such as large language models (LLMs) have begun to change the way we acquire and use abstract knowledge,” Li wrote in a Nov. 10 blog post. “Yet they remain wordsmiths in the dark; eloquent but inexperienced, knowledgeable but lacking in foundation.”

An emphasis on visual and spatial intelligence has long been Li’s “North Star,” the researcher said. He released ImageNet in 2006, a database of 15 million images that spurred the rise of deep learning. Lee also co-directs Stanford University’s Institute for Human-Centered Artificial Intelligence and serves as an AI policy advisor to the United Nations.

Today, however, Li’s focus is on World Labs, which has raised $230 million to realize its vision of spatial intelligence. Its backers include Radical Ventures, Andreessen Horowitz and Nvidia, as well as prominent technology figures such as Geoffrey Hinton, Eric Schmidt, Marc Benioff and Reid Hoffman.

Marble has been in testing for several months and is now publicly available. It can create a complete 3D world from a single image or text prompt. Users can also merge multiple environments by uploading multiple images in the prompt. According to World Lab, the model can be combined with photos or short videos of real-world spaces to generate immersive and realistic virtual worlds.

The mockup includes a range of editing tools that allow users to customize their creations. A feature called Chisel allows users to draw rough sketches of 3D layouts, while other tools can expand the world or build entirely new scenes within the same environment. Going forward, WorldLab plans to develop world models with more interactive capabilities for humans and artificial agents.

While Lee may be the most prominent figure in the development of models in the world, she’s not the only one in the field. Google DeepMind and Nvidia have explored similar techniques with their Genie and Cosmos models respectively. Meta’s chief artificial intelligence scientist Yann LeCun is reportedly in the early stages of raising funding for his own world-modeling startup.

Applications of spatial intelligence tools like Marble will “span different timelines,” Li said. The model has been used by filmmakers, game designers, and architects to enhance creative workflows. In the medium term, Li expects such technologies to drive advances in robotics, while future applications in science, healthcare and education could lead to breakthroughs in experimental simulations, drug discovery and immersive learning.

“Spatial intelligence will transform the way we create and interact with real and virtual worlds—revolutionizing storytelling, creativity, robotics, scientific discovery, and more,” said Lee. “This is the next frontier of artificial intelligence.”