Text-to-Video

A small business wants a six-second clip of coffee being poured in slow motion for a product page. Instead of booking a shoot, someone writes that description and a model returns a clip. Google’s Veo and open models run through libraries such as Hugging Face Diffusers offer this, usually alongside image-to-video, where a still picture becomes the first frame.

Text-to-video grew out of text-to-image. Most systems are diffusion models that denoise many frames at once, so the model has to keep objects, lighting and motion consistent over time as well as matching the prompt. Google’s Imagen Video paper generates a short low-resolution clip first and then upscales it in space and time. Meta’s Make-A-Video paper reuses what an image model already knows and learns motion from unlabeled video. Video is the expensive end of multimodal AI: a few seconds of output is many images, so generation is slower and costlier per request than an image or a paragraph.

Why it matters (editorial synthesis): text-to-video makes storyboards, ad variations and explainer footage cheap to try, but the output is invented. Check clips for physics that looks wrong, garbled text and hands, and any resemblance to real people, places or brands before you publish. Realistic synthetic video of a real person can count as a deepfake under the EU AI Act when it would falsely appear authentic or truthful. The Act requires deployers to disclose deepfakes and providers to mark generated video in a machine-readable way.

No Chain of Thought guest has discussed text-to-video in depth yet, so this entry relies on the papers and documentation above.