Luma AI, the a16z-backed company specializing in AI video and 3D models, has launched Ray3 Modify, an innovative AI model designed to give creators unprecedented control over video generation and modification. Available through the company's Dream Machine platform, Ray3 Modify allows users to transform existing footage by providing character reference images while meticulously preserving the original human performance. A key feature also enables users to guide the model in generating seamless transitional footage using specified start and end frames.
Luma AI states that Ray3 Modify addresses a critical challenge for creative studios: maintaining authentic human performance when editing or generating effects with AI. The model is engineered to closely adhere to input footage, empowering studios to leverage human actors for creative or brand content. Luma highlights that the new model excels at retaining an actor's original motion, timing, eye line, and emotional delivery, even as it transforms the surrounding scene.
One of Ray3 Modify's core capabilities is the ability to transform a human actor's appearance into a specified character by providing a character reference image. This feature ensures that crucial details such as costumes, likeness, and overall identity are consistently maintained throughout the entire shoot, offering unparalleled creative flexibility.
Furthermore, Ray3 Modify allows users to define start and end reference frames to guide the creation of new video segments. This functionality is particularly valuable for creators needing to direct precise transitions or control character movements and behavior, ensuring seamless continuity between different scenes.
"Generative video models are incredibly expressive but also hard to control. Today, we are excited to introduce Ray3 Modify that blends the real-world with the expressivity of AI while giving full control to creatives. This means creative teams can capture performances with a camera and then immediately modify them to be in any location imaginable, change costumes, or even go back and reshoot the scene with AI, without recreating the physical shoot," said Amit Jain, co-founder and CEO of Luma AI, in a statement.
The Ray3 Modify model is now accessible to users via Luma AI's Dream Machine platform. The company, a competitor in the generative AI space alongside firms like Runway and Kling, had previously introduced video modification capabilities.
This latest model release follows a significant development for Luma AI: a recently announced $900 million funding round in November. This substantial investment was led by Humain, an AI company owned by Saudi Arabia's Public Investment Fund. Existing investors, including a16z, Amplify Partners, and Matrix Partners, also participated. Furthermore, Luma AI, in collaboration with Humain, is planning to develop a 2GW AI cluster in Saudi Arabia.
