This is one of the first video-to-video AI tools in reaching the public. It can synthesize new video content and significantly transform existing clips from text prompts.
Intrigued? Read on!
RunwayML’s Gen-1: Making Videos with AI on your Phone
Runway’s Gen-1 is an AI-generative model that uses video, image, and text input to produce video-only output. In other words, you can feed it video clips and photos, enter a written description of what you want it to do with it, and the tool creates an entirely new video to match your request.
With it, you can do things like turn a regular daylight video into one with a pastel palette, or convert it into a claymation-looking clip, or recreate a video of a person talking but replacing the person’s face for your dog’s, for example.
The app comes with a number of preset filters you can apply, it lets you preview stills of what your generated content will look like and select the ones you wish to include in the final video, and it’s free to download and use – you only pay if you wish to remove the watermark from the final result.
As this model uses video clips as the foundation for generating new motion content, it’s considered a video-to-video AI video generating application, more than an AI video editor that only applies filters or performs editing tasks on a given piece of content – Runway already has very powerful AI video editing tools, by the way.
Besides the AI functionality, the highlight is that this is a mobile app –currently available for iPhone users only–; as Runway's CEO said in declarations to The Verge, this seeks to bring video generative capability to the devices in which most users capture their footage. The idea is that you can capture video and transform it or create something new from it, all right on your phone.
RunwayML’s Current Functionality and Expectations
Just like it happened with AI image generators that disrupted the creative industry less than 12 months ago, Gen-1 is for now intended mostly to play around and challenge your creativity, more than for professional use.
It has usability limitations; you can only upload short clips of up to 5 seconds in duration, and it takes up to 2 minutes to generate a video, for example. Plus, just as it happened with AI image generators when they just came out, the final results can often be a bit weird-looking, not quite what you expected, or simply wrong.
That said, the company is already planning to integrate Gen-2 –the evolution of this model, which is presumed a lot more powerful– into the app. And if AI video generation follows the path of AI image generation, the results will get better and better by the day.
For now, you can have fun using an innovative tool to explore new creative horizons in video. Still, you shouldn’t expect to be able to produce pro-quality videos with it, at least not without investing time and effort.
AI Video Generation is On
Though no one really doubted that video was the next frontier for generative AI to conquer, it did seem –and still does, in a way– that it was taking longer than it did for AI still images to surge.
However, Adobe has already announced important video functionality coming to its platform through Adobe Firefly. And a cutting-edge startup like Runway releasing an AI video generative model and doing it on mobile, is another big leap that puts AI-generated video one step closer to our hands.
Are you going to try RunwayML? Do tell!