Computer-generated imagery is hardly new to the movie industry. But, new advancements in generative AI video technology make it quicker, cheaper and more accessible than ever to create clips without an entire production studio and crew.
In the past, creative software powered by AI has attracted enthusiasts like tech nerds, marketing folks and amateur creators. However, professional creators have been more wary, voicing concerns over copyright, job security and AI slop replacing human creativity.
Now, AI videos are amplifying older concerns from folks like VFX artists about the impact of AI on their industry. It’s igniting questions over what role, if any, AI video tech should play in the future of moviemaking. One company is trying to get ahead of creators’ concerns and find a way to work with filmmakers and production companies.
Meet Moonvalley and its AI video generator, Marey 1.0, named after French cinema pioneer Étienne-Jules Marey.
Building AI with filmmakers
Moonvalley was built for professional filmmakers and directors. The company’s goal is to work with filmmakers, building out from their needs rather than aiming to replace them. This is one way that the Los Angeles-based company is philosophically (and physically) miles from the other Silicon Valley AI companies, according to co-founder and CEO Naeem Talukdar.
«We just don’t think the way that models are being built today are going to work in the [movie] industry,» said Talukdar in an interview with CNET. «Models are all text-to-video, primarily, which just doesn’t make sense in a production filmmaking environment. Text is such a limited form.» Filmmakers also want more control, like the ability to swing a camera around a virtual set.
Because it wanted its model to be different, Moonvalley collaborated with directors and filmmakers while building Marey. And it had visible impacts on how the model works and the features it includes.
Moonvalley said Marey is designed to create production-quality AI video clips. Marey can create clips up to 30 seconds long, which is six times longer than most models, but the sweet spot for the best results is the typical 5- to 10-second range. Like the majority of AI video tools, the clips don’t include audio. To get the high resolution that filmmakers would need, Marey natively generates in HD, with the ability to generate in 4K coming soon.
Marey launched in March to a private group of filmmakers, but the company plans to release the model publicly this summer. Moonvalley is partnered with Asteria Film Co, an AI production company.
The video creation process that Marey uses is inspired by the same method animated movies use. The program can help you storyboard and turn your storyboard into keyframe images, which Marey can animate. It’s designed to integrate with the existing movie-making process, rather than force directors, designers and animators to learn how to write effective prompts or use generative editing tools.
For example, a big part of the beginning stages of making a movie is developing aesthetics and other visuals. Talukdar heard from filmmakers that they wanted a way to draw using the program rather than writing the perfect prompt to get the model to produce what they were envisioning. So Marey lets you upload sketches and storyboards and turns them into video clips. It also includes a tool called mimic. You can upload a video of an actor acting out a scene, doing action sequences or having emotional reactions. Then, the tool can read and recreate that motion in an AI video.
Marey also includes a slew of camera and motion controls, along with the ability to edit specific regions of the shot. These controls and editing tools are important for any AI program, but they’re especially vital when aiding in professional productions.
Besides learning the nuts and bolts of using AI video models, creators have bigger questions when deciding whether to use AI or AI-enabled tools.
«The industry has also made it pretty loud and clear that they’re not happy about the way that these models are being built,» said Talukdar. «There’s a massive legal discussion, but also just from a practical standpoint, I think it’s very hard to get artists and creators to use models that have been trained on their own work.»
One of the most important questions is how a model is trained. That’s another aspect where Marey stands apart from other tech companies: It’s a so-called ‘clean’ AI.
«Clean» AI: What it is, possibilities and pitfalls
«Clean AI» is a term that describes AI programs whose underlying models are trained on high-quality, human-produced content that the company has the right to use for training.
The idea of clean AI is important because so many creators are worried about how their work is being used to train AI, including training done without their permission. The designation of «clean AI» can also indicate that the content an AI program creates is commercially safe — people can use it for business or marketing purposes without worrying about legal concerns around IP and copyright. For professional creators, that’s a vital element.
However, it’s important to note that the concept of a «clean» model is something of a misnomer. Generative AI models cannot function without being trained on existing content; it’s not creating something from nothing. «Clean» AI is as much of a marketing term as it is a promise on its ethics.
For Moonvalley, its «clean» model means the company has written consent to use the content in its datasets for the explicit purpose of AI training. Moonvalley worked with data brokers, creator collectives, solo creators and filmmakers to hammer out those paid agreements.
Moonvalley hasn’t published exactly what content it used. But the company hopes to build an open marketplace where creators can upload and remove their content as needed in the future.
Why does «clean» AI matter?
The idea of clean AI is so important because too many tech companies aren’t transparent about how their AI models are created. OpenAI, the company behind ChatGPT and video generator Sora, is in a heated legal battle with The New York Times over its training practices. Runway, another popular AI video company, is one of several AI companies facing a class action lawsuit from artists claiming the AI creative companies infringed on their copyright protections.
One of the biggest frustrations creators have with AI companies is a maddening mix of ambiguity and lack of control. OpenAI, Google, Runway and many others refuse to be open about what they use to train their models; Meta doesn’t let its US social media users opt out of having their posts used for training. When the bar is set so low it’s in hell, Moonvalley’s claim to have created «clean» AI can seem like a better path, or even a refreshing one.
The use of AI in creative fields is controversial and something of an ethical minefield. Moonvalley hopes that building a clean model will earn the company some trust with creators. For filmmakers considering whether or not to incorporate AI into their workflows, using a clean model like Marey might be one less thing to worry about.
«Our hope is that this can set kind of a precedent that it is possible to build state-of-the-art models while still respecting copyright law and respecting creators,» said Talukdar. «And our hope is that this is also the first model that creators themselves can actually get excited about using.»
