Meta Tests AR Furniture Try-On on Facebook Marketplace

Meta is rolling out an innovative augmented reality (AR) feature for Facebook Marketplace, allowing users to virtually 'try on' furniture and home decor items in their own living spaces before making a purchase. This new 'View in Room' option is powered by Meta's evolving Segment Anything (SAM) artificial intelligence process, aiming to transform the online shopping experience.

How Meta's SAM AI Enhances Virtual Try-On

At its core, Meta's SAM process excels at identifying specific objects within an image or video frame. This capability then unlocks new possibilities for editing and customization, tailored to each identified object. The 'View in Room' feature leverages the latest advancements in Meta's SAM collection, specifically SAM 3D. Meta is currently piloting this functionality with a select group of users.

Meta has been developing its SAM system over the past few years with a long-term vision: to create more realistic 3D depictions for augmented reality (AR) and virtual reality (VR) experiences. This ambition extends directly to AR applications within Marketplace listings, where the SAM system can now simulate the placement of various furniture and home decor items within a user's environment.

Comparing to Existing AR Solutions

This functionality bears a resemblance to Pinterest's 3D placement tools for home decor products, which often utilize LiDAR technology in newer mobile devices to enhance scene mapping, spatial understanding, and ensure accurate scaling and fit. While existing AR solutions can face limitations in fully accessing an item's spatial dimensions and accurately mapping them in real-world environments, Meta hopes SAM can automate much of this process at scale. Beyond Marketplace, SAM is also envisioned to power a range of new video editing features.

Future Implications for Shopping and AR/VR

Although the broader video customization potential of SAM is significant, its immediate application in Marketplace could prove to be one of its most valuable uses, offering a practical demonstration of Meta's evolving visual effects technology. While still in its early stages, a successful implementation of SAM could introduce an entirely new dimension to in-app shopping. It promises to allow users to virtually experiment with a wide array of items and objects in diverse environments, providing a realistic preview of how they would integrate into their real-life spaces.

This technology is also poised to extend its benefits to broader VR and AR experiences, streamlining the complex process of generating realistic 3D objects for Meta's next-generation immersive environments. As development continues, a wider release may take some time. Nevertheless, this represents a compelling advancement for both Facebook and Instagram listings, hinting at a more interactive future for online commerce.