Text-to-Image

A marketing team types “a red bicycle leaning on a brick wall at sunset, watercolor style” and gets four candidate images in seconds. Nothing was searched or copied from a stock library: the model produced new pixels that match the words. Products such as DALL-E, Midjourney, Stable Diffusion and Google’s Imagen all work this way.

Most of these systems are diffusion models. During training they learn to remove noise from images step by step, with a text encoder telling them what the caption says. At generation time they start from random noise and denoise it toward an image that fits the prompt. The latent diffusion approach (Rombach et al.) runs that process in a compressed latent space instead of on full-size pixels, which made high-resolution generation cheap enough to run on a single GPU. Text-to-image is one branch of generative AI and one direction of multimodal AI, which moves between text, images, audio and video.

In episode 54 at 9:02, Dan Klein makes a point that applies directly: in image generation, “the hallucination is actually the product.” You want a novel, invented image, not a copyrighted one handed back to you. That is a useful way to separate this task from fact-finding. The same habit of invention that is a feature here becomes an AI hallucination when a model is supposed to report facts. The episode was sponsored by Galileo (sponsor mention).

Why it matters (editorial synthesis): text-to-image cuts the cost of mockups, ad variations, storyboards and illustrations, but it raises questions a business has to answer before shipping the output. Check the provider’s licence terms, watch for images that closely resemble a real person, brand or artwork, and label synthetic images where your audience or the law expects it. The EU AI Act requires providers of generative systems to mark synthetic images in a machine-readable way and requires deployers to disclose deepfakes.