How Does AI Generate Images, Sound and Video? Diffusion, Sora, Veo and Choosing the Right Tools

How does AI generate images, sound and video?

A language model such as ChatGPT, Gemini or Claude does exactly one thing: it predicts the next piece of text, called a token, one at a time. So every image, voice and video you get from an AI product has to come out of one of three routes built around that limit.

The first route is hand-off: the language model writes a detailed prompt and calls a separate image, voice or video model to do the work. The second is media as tokens: images and sound are turned into codes the model can predict just like words. The third is diffusion: start from random noise and sculpt it, a little at a time, into a picture. The diagram below sets the three side by side.

Real products mix them. A chat model often plans and writes the prompt while a diffusion model renders it, and a model that generates images natively may still use a diffusion-style decoder for the final pass.

What is the hand-off route?

In the hand-off route the language model never draws anything itself. It writes a detailed instruction and calls another tool or model to produce the media. You type "make a poster for our Songkran sale"; the language model turns that into a long prompt specifying colours, layout and style, sends it to an image model, and shows the result back in the chat.

The well-known example is ChatGPT calling DALL·E 3 in 2023, and most agent tools still work this way today. The strength is that you can swap in the best specialist model for each job, test cheaply, and control exactly what is sent where.

The weakness is that the language model never sees the pixels. When you follow up with "same poster, just move the logo left", the image model tends to redraw everything, and the details you already liked drift.

How does AI turn images and sound into tokens?

AI turns images and sound into tokens with a learned compressor. Its encoder squeezes the media into a short list of codes picked from a fixed codebook, and its decoder rebuilds the media from those codes. Once media is a list of codes, the same transformer that predicts words can predict image or audio codes next.

Audio uses compressors called neural audio codecs, such as Google's SoundStream and Meta's EnCodec. Both cut sound into short slices and give each slice a stack of codes: the first layer captures the broad shape, each further layer the detail still missing. The technique is called residual vector quantization, or RVQ. The real numbers for EnCodec at 6 kbps are 75 slices a second with 8 codes each, so 600 codes for one second of audio.

Google's AudioLM research uses two kinds of token: semantic tokens for what is said, and acoustic tokens for how it sounds, the voice and tone. The model predicts the first kind, then fills in the second. The diagram below follows an image and a sound from the encoder, through the transformer, to the decoder.

• Good at text inside images and long instructions, because the prompt and the picture sit in one sequence and the model reads them together.
• Good at follow-up edits, because the model sees the image codes it produced and can change one area without redrawing the rest.
• Slower, because it predicts one code at a time and an image takes thousands of codes, while a text reply takes hundreds.
• OpenAI describes GPT-4o's image generation (2025) as native and autoregressive but has not published the architecture. Outside analyses suggest it predicts image latents and finishes them with a diffusion-style decoder; that is inference, not information from OpenAI.

What is diffusion, and how does it make a picture out of noise?

Diffusion makes an image by starting from pure random noise and removing a little of it at a time, over 20 to 50 steps, with the prompt steering each step, until the intended picture is left. In training, the model sees real images mixed with noise in varying amounts and learns which part is noise, so at generation time it can run the process in reverse.

Newer models such as Stable Diffusion 3 use rectified flow: noise and image sit on one straight line, and the model learns the direction along it. A straighter path allows fewer steps.

To save compute, none of this runs on full-size pixels. A compressor first shrinks the image into a latent that is 8 times smaller on each side in Stable Diffusion, the denoising happens in that small space, and a decoder expands the result back to a full image at the end.

A diffusion model does not understand language on its own. It relies on text encoders to turn the prompt into numbers that steer it, and Stable Diffusion 3 uses three: CLIP-L, CLIP-G and T5-XXL. The diagram below shows the picture emerging from the noise, step by step.

start: noise

noise 75%

noise 50%

noise 25%

done

When generating, the model walks left to right, removing noise a step at a time, with the text prompt setting the direction.

How does AI generate video, and how do Sora and Veo differ?

AI generates video with diffusion that runs across space and time at once. OpenAI's Sora technical report lays it out most clearly: compress the video into a latent, cut it into small blocks called spacetime patches that span a few pixels and a few frames, and let a diffusion transformer denoise all the patches together.

Because every patch can attend to every other one, a face in frame 1 can stay the same face in frame 200. That consistency is the hard part, and compute grows with width, height and length, which is why video is the most expensive medium of all. The diagram below shows a clip cut into blocks across space and time.

Google's Veo 3 added sound in the same pass: dialogue, effects and ambience generated together with the picture, so lips and footsteps line up. Veo 3's model card describes it as latent diffusion applied jointly to video and audio latents, though in less detail than the Sora report.

The practical difference today is that Sora is gone. OpenAI closed the Sora app and website on 26 April 2026 and shuts the Sora API on 24 September 2026. If you are starting video work now, look at options such as Veo 3.1 or Runway instead.

How does AI generate voice and music?

Most voice generation uses the token route: text or a sample voice goes in, the model predicts audio codec codes, and a decoder turns the codes back into a waveform. Microsoft's VALL-E research used EnCodec codes this way and could imitate a voice from a short sample.

Real-time voice modes in chat apps work speech-to-speech: they take your voice in as tokens and answer in audio tokens directly, without converting to text first, which keeps replies fast and preserves tone. The hard part is natural timing and emotion without adding delay.

Music is harder than speech, because it needs structure over minutes: verse, chorus, and the return of a melody. Research such as Google's MusicLM uses tokens, but most commercial music apps do not publish their architecture, so nobody outside can say for certain whether they use tokens, diffusion or both.

Which route does each medium use, and what makes it hard?

Images use either diffusion or tokens, speech mostly uses tokens, music is largely unpublished, and video uses a diffusion transformer. The table puts them side by side.

Which AI image, video and voice tools are good, and what do they cost?

The popular tools come in two kinds. Pay-as-you-go APIs suit wiring generation into a system; monthly web plans suit a marketing team working by hand. This table includes only tools whose prices and commercial-use terms we could verify on the vendor's own pages on 23 September 2026. Prices are in US dollars as published, before tax.

On rights, check three things before real use. First, most free plans forbid commercial use. Second, some vendors set conditions by company size: Midjourney requires companies with more than USD 1 million in gross revenue a year to be on Pro or Mega. Third, every vendor makes you responsible for ensuring that what you put in and what comes out infringes no one's rights. Terms change, so read the current ones yourself before paying.

Which tool fits which business? Four example scenarios

These four are composite scenarios drawn from problems Thai businesses commonly bring, not real clients. The figures come from the price table above and already allow for regenerating, because real work rarely gets a usable result on the first try.

What should a business think about before using AI for media?

Cost follows the token count or the number of passes. A text reply is a few hundred tokens, an image is thousands of codes or dozens of denoising passes, and video is all of that multiplied by the seconds. Budget most for video, and price by the usable piece, not by each click of generate.

If you are wiring AI media into your own product or system, letting a language model plan and call a specialist model (hand-off) is the practical default: easy to swap, cheap to test, simple to control. But if the main job is repeated edits to the same image, a model that generates natively keeps untouched areas intact far better.

Before using it on anything customers see, check these risks.

• Brand consistency: colours, logos and product details often drift between images, so someone must review every piece before it goes out.
• Rights to faces and voices: a face or voice that identifies a person is personal data under Thailand's PDPA. Cloning an employee's voice or using a customer's photo needs a lawful basis, and written consent is the sensible default.
• Telling customers what is AI: YouTube and TikTok have rules requiring labels on realistic AI-generated content, and saying so plainly also protects customer trust.
• Some advertising, such as for clinics, food and medicines, has its own rules in Thailand, and AI-made images must follow them just as photographs do.
• What leaves the building: unreleased product photos or customers' voices should not go into free plans whose vendors may use the data to improve their products.

How can SyncEdge help with AI for media?

We start from the work your team actually does, not from a list of tools. We look at which image, voice or video work costs the most time or money, and assess whether AI is worth applying there at all. If hiring a photographer or changing the process is cheaper, we say so.

Where it is worth it, we train your team on its own work and data, and set PDPA-sound rules on which photos, voices and customer data may go into external tools and which may not. You finish with a written summary of where AI applies, where it does not, and the next steps. Sessions run on-site or online, at a fixed price agreed after scoping.

See our AI consulting and training