Meta unveiled a crazy artificial intelligence model that allows users to turn their typed descriptions into video. The system is called Make-A-Video and is the latest in a trend of AI generated content on the web.
The system accepts short descriptions like “a robot surfing a wave in the ocean” or “clown fish swimming through the coral reef” and dynamically generates a short GIF of the description. There are even three different styles of videos to choose from: surreal, realistic, and stylized.
An artist’s brush painting on a canvas close up
According to a Facebook post by Meta CEO, Mark Zuckerberg, translating written text into video is much harder because of how video requires movement:
“It’s much harder to generate video than photos because beyond correctly generating each pixel, the system also has to predict how they’ll change over time. Make-A-Video solves this by adding a layer of unsupervised learning that enables the system to understand motion in the physical world and apply it to traditional text-to-image generation.”
A young couple walking in a heavy rain
Meta’s AI Research team wrote a paper describing how the system works and how it differs from current text-to-image (T2I) methods. Unlike other machine language models, Meta’s Text-to-Video (T2V) method doesn’t use pre-defined text-video pairs. For example, it doesn’t pair “man walking” with a video of an actual man walking.
If this sounds a lot like DALL-E, the popular T2I application, you wouldn’t be far off. Other T2I applications have rolled out since DALL-E gained popularity. TikTok released a filter in August called AI Greenscreen that generates painting style images based on the words you type.
A fluffy baby sloth with an orange knitted hat trying to figure out a laptop close up highly detailed studio lighting screen reflecting in its eye
AI-generated content has become quite buzzworthy within the last few years. Deepfake technology, machine learning techniques to replace a person’s face with another, is even used by visual effects studios for big budget shows like The Mandalorian.
In July, The Times mistakenly reported on a Ukrainian woman in the midst of the Russia-Ukraine war. The problem is she wasn’t real.
The threat of AI probably isn’t a real threat, but projects like DALL-E and Make-A-Video are fun explorations into some of the interesting possibilities.