Lessons0 of 20 done

Module 4: Beyond chat. Lesson 14 of 20, 9 minutes.

Images, audio and video.

How image models build a picture from noise, how to prompt them, where they fail, what audio and video tools can do, and how to tell what is real.

Type a lighthouse at dusk, oil painting and a few seconds later there is one. No photographer, no painter, no stock library. Read a paragraph aloud into your phone and the text appears, nearly perfect. Paste a script and a voice you have never heard reads it back.

This lesson is about how those tools work, how to direct them, and how much to trust what you now see and hear.

How an image model works

Most image generators are diffusion models, and the name describes the training trick. Take millions of real images, each paired with a caption. Add a little random noise to each one, then a little more, until every image is pure static. Now train the model to run that process backwards: given a noisy image and its caption, predict what the slightly less noisy version looked like. Do this enough times and the model becomes very good at removing noise in the direction of whatever the caption describes.

To make a new picture, you start with pure static and a description. The model removes noise step by step, often dozens of times, until a picture emerges that fits the words. It is not retrieving an image it has seen. It is building one that has the statistical shape of images with that kind of caption. This is why these models are superb at the look of things (light, texture, mood, style) and unreliable about anything that has to be exactly right.

How to prompt for an image

Describe the picture the way you would brief a photographer who has never seen your idea. Five things, in roughly this order:

  • Subject. What it is, specifically. Not a dog: a wet golden retriever mid-shake.
  • Setting. Where and when. The place, the time of day, the weather, the light.
  • Style. Photograph or painting, and which kind. A documentary photo, a watercolor, a 1970s film still, a flat vector illustration.
  • Composition. Close-up or wide, the angle, where the subject sits in the frame.
  • What to avoid. Some tools have a separate box for this (a negative prompt); others accept it in plain words.
Before

a dog on a beach

After

A golden retriever shaking water off, mid-shake, on an empty gray-sand beach at low tide. Overcast morning, soft light. Documentary photograph, slightly low angle, dog filling the left two thirds of the frame. No people, no text.

Plain, specific words beat lists of magic keywords. Expect to generate several and pick, exactly as with text.

Where image models fail

Three failures come up again and again, and knowing them saves a lot of regenerating.

Text. Letters and words inside an image (a shop sign, a book cover, a label) often come out scrambled. Models have improved, but check every letter before you use anything with words in it.

Consistency. Asking for the same character, product or room across several images is hard. The face drifts, the jacket changes color. Tools offer reference images and other aids, but expect to work for it.

Fine detail with a right answer. Hands and fingers, teeth, the keys on a keyboard, the numbers on a watch face, the reflection in a mirror. Anything with a correct count or geometry. Ask for a clock showing twenty to four and you may well get the ten past ten of a thousand product photos. The model learned what things usually look like, and usual is not the same as correct.

Speech to text and text to speech

Speech to text, also called transcription, turns audio into written words. These models learned from huge amounts of recorded speech paired with transcripts, and on a clear recording of one person they are now very good. They struggle more with people talking over each other, accents they heard less of in training, specialist terms, and above all names and numbers. A transcript of a meeting is a fine starting point for notes. It is not a record you quote from without checking the names and the figures.

Text to speech goes the other way. Synthetic voices have become natural in pacing and tone, and many tools can clone a voice from a short recording. That is useful for narration, accessibility and language learning, and it is also the reason the final section of this lesson exists.

Video, and what it is good for today

Video generators work on similar principles to image models, extended across time, and today they mostly produce short clips of a few seconds from a text description or a still image. What they do well: atmospheric shots, backgrounds and filler footage, bringing a still image to life, quick concept previews, short ads. What they do badly: long coherent scenes, the same character across several shots, and physics. Objects merge, hands pass through things, a person walks behind a pillar and comes out different.

A fair way to think of them for now: an unusual stock footage library that you direct loosely. Expect several attempts for each usable clip, and expect to cut around the flaws.

Deepfakes, and knowing what is real

A deepfake is a synthetic image, audio clip or video that shows a real person doing or saying something they did not. It used to need expertise. It now needs a few photos or a few seconds of audio and a consumer tool. The most common harm so far is personal rather than political: a cloned voice calling a relative in distress and asking for money is an established scam.

Two efforts try to help you tell what is what. Content credentials attach a signed record to a file saying what made it and what was done to it, a bit like a nutrition label; some cameras, editing tools and generators now add them. Watermarks are patterns hidden in the pixels or the audio that a detector can read even when a person cannot. Both have limits. A screenshot or a re-upload can strip the label. A watermark can be degraded. Not every generator participates. So the absence of a label proves nothing either way.

Practical habits: for anything surprising, find where it first appeared. For a call that asks for money or secrets, hang up and call back on a number you already have. Agree on a word with your family that no recording would know.

Try it

If your chatbot can make images, run the before prompt from this lesson, a dog on a beach, then the after prompt, and compare. Then ask for a small bakery storefront with the name Rosa's Bakery painted on the sign and check every letter. If you have no image tool, open dictation on your phone and read aloud a paragraph that contains two names and a phone number. See what it gets right and where it slips.

Key terms

Diffusion model
An image model trained to remove noise from a picture step by step, so it can build a new picture from pure static to match a description.
Negative prompt
The part of an image request that says what to leave out, sometimes typed in a separate box.
Transcription
Turning recorded speech into written text, also called speech to text.
Text to speech
Turning written text into a spoken voice, sometimes one cloned from a real person's recording.
Deepfake
A synthetic image, audio clip or video that shows a real person doing or saying something they did not.
Content credentials
A signed record attached to a file saying what created it and how it was edited.

Check yourself

1.How does a diffusion model produce an image?
2.Which request is most likely to come back wrong?
3.A video has no content credentials. What does that tell you?

Image, audio and video models are superb at the look and sound of things and unreliable about exact details, so direct them like a photographer and check them like an editor.