Runway teases AI-powered text-to-video editing using text prompts

A still from Runway's
Enlarge / A still image from Runway’s “Text to Video” opening ad suggesting the possibility of visualisation.


In a tweet posted this morning, artificial intelligence company Runway tease a new feature Its AI-powered web-based video editor can edit videos from written descriptions, often referred to as “reminders”. A promotional video that appears to show very early steps towards commercial video editing or creation, echoes the hype over recent text-to-image fusion models like Stable diffusion but with some optimistic framework to mask the current limitations.

Runway’s “Text to Video” slide shows a text input box that allows editing of commands like “enter city street” (suggest already existing video clip) or “make it look more cinematic” (apply effects). It depicts someone typing “remove object” and selecting a street light with the drawing tool then disappearing (from our testing, Runway was able to achieve a similar effect using the tool). “inpainting”, with mixed results). The promo video also showcases what looks like a text-to-still image similar to Steady Diffusion (note that the video doesn’t depict any of these generated scenes in motion) and demonstrates text overlays, character masks (using the “Green Screen feature”, also available in Runway) and more.

The promise of video generation aside, the most novel thing about Runway’s Text to Video notifications is the text-based command interface. Whether video editors want to work with natural language prompts in the future, but evidence suggests that those in the video production industry are actively working toward a future in which Composing or editing videos is as easy as writing commands.

Runway's web-based video editor used AI to mask objects to create a "Green screen" effect.
Enlarge / Runway’s web-based video editor used AI to mask objects to create a “Green Screen” effect.

Ars Technica

AI-based raw video generation (sometimes referred to as “text2video”) is in its infancy due to high computational demands and lack of large open video training sets with metadata that can train generating models. video equivalent to LAION-5B for still images. One of the most promising public text2video models, called CogVideo, can create simple video in low resolution with variable frame rate. But consider the primitive state of text-to-image models just a year ago fight with todayIt seems reasonable to expect the quality of the composite video generation to increase rapidly over the next few years.

Runway is available as a commercial web-based product that runs in the Google Chrome browser for a monthly fee that includes cloud storage for approx. $35 per year. However, Text-to-Video is in closed beta “Early Access” and you can sign up for the waitlist on Runway’s website.

Source link


News5s: Update the world's latest breaking news online of the day, breaking news, politics, society today, international mainstream news .Updated news 24/7: Entertainment, the World everyday world. Hot news, images, video clips that are updated quickly and reliably

Related Articles

Back to top button