Station 7 · Multimodal
Every station so far treated AI as a thing that reads and writes words. But modern models can also see pictures and hear audio — and can make brand-new images and voices. That's your new toolbox.
What you'll walk away with
Concept 1
A "mode" is just a type of information: text is one mode, images are another, audio is another. A multimodal model can work with more than one of them at the same time.
Up to now you've mostly met text-only models — you type words, they type words back. A multimodal model breaks out of that box. You can hand it a photo and it will describe what's in it. You can play it a voice recording and it will understand the words. Some can even hand you back a picture or a spoken voice, not just typed text.
A text-only model is like a brilliant friend who can only communicate by passing notes — very smart, but blindfolded and wearing earplugs. Making it multimodal is like giving that friend eyes and ears. Now you can show it things and talk to it, not just slide it notes. Same brain — more ways in and out.
The big three chatbots — ChatGPT, Claude, and Gemini — are all multimodal in 2026: you can drag a photo into the chat and ask about it. Behind the scenes there are also specialist tools that do only images or only voice extremely well. As a builder, you'll mix and match.
Concept 2
The first superpower is reading a picture. You give the model an image plus a question, and it answers in words — just like it would about a paragraph of text.
What can you actually ask? A lot more than "what's in this picture?":
Vision doesn't replace text — it feeds the text brain. The model "looks" at the image, turns what it sees into an internal description, and then does its normal next-word prediction on top of that. So all your prompting skills from Station 4 still apply: a clear, specific question about an image gets a clearer answer.
Real builder uses show up fast: an accessibility tool that describes photos aloud, a study helper that reads a snapshot of a worksheet, an app that extracts the total from a receipt photo. None of these needed you to train anything — you just asked a model that can see.
Concept 3
The flip side of understanding is creating: you describe a picture in words, and the model paints a brand-new one that never existed before.
This is what tools like DALL·E, Midjourney, Stable Diffusion, and the image generators built into Google and Adobe products do. You write a sentence — a "prompt" — and get an image back. Want it different? Tweak the words and generate again. That loop of describe → look → adjust is the whole skill.
Vague prompts give vague pictures. Strong image prompts usually name four things:
The main thing in the frame. "A red fox," "a cozy reading nook," "a paper coffee cup."
The art direction. "Watercolor," "3D render," "vintage photo," "flat minimalist icon."
How it's lit. "Soft morning light," "neon glow," "dramatic shadows," "bright studio."
Angle and layout. "Close-up," "top-down view," "wide shot with lots of empty sky."
Put together: "A red fox curled asleep in a snowy forest, watercolor style, soft morning light, close-up composition." That specific sentence gives the model a real target instead of a shrug.
They're astonishing but not magic. Text inside images (signs, labels, logos) often comes out garbled or misspelled. Precise details — exact hand shapes, the right number of fingers, an exact product's look, or a specific real person — can be unreliable. Treat the output as a first draft to refine, not a guaranteed-correct final. And always respect that generated images can raise copyright and likeness questions — a topic you'll dig into next station.
Concept 4
The third mode is sound. Voice AI has two clean halves that mirror each other — one turns speech into text, the other turns text into speech.
Also called transcription. You feed in an audio recording; you get back the words as typed text. This is how captions, voice notes, and "type by talking" work.
Often shortened to TTS. You feed in text; you get back a natural-sounding spoken voice. This is how screen readers and audiobook voices work.
Neither half is an LLM by itself — they're specialist audio models. But the magic happens when you chain all three pieces together into a voice assistant loop:
Hear → Transcribe → Think → Speak. A microphone captures your voice, speech-to-text turns it into words, an LLM reads those words and writes a reply, and text-to-speech says that reply out loud. Four ordinary parts, wired in a line, and suddenly you're talking to an AI. That's exactly how Siri, Alexa, and voice-mode chatbots are built.
Concept 5
The real fun for a builder isn't any single mode — it's piping them into each other so the output of one becomes the input of the next.
Here's a classic multimodal pipeline written as plain pseudocode: a student snaps a photo of a math problem, and the app reads it, solves it, and explains the answer out loud.
# 1. A photo comes in from the phone camera photo = "math_problem.jpg" # 2. VISION: the model reads what the picture shows problem = ask_vision(photo, "Type out the math problem in this image.") # problem -> "12 x (4 + 3)" # 3. LLM: send that text to the language model to solve + explain answer = ask_llm("Solve this and explain each step simply: " + problem) # 4. TTS: turn the written explanation into a spoken voice audio = text_to_speech(answer) play(audio) # the student hears the walkthrough
Swap the pieces around and you get endless projects: audio in → transcribe → summarize with an LLM → done (a meeting-notes app). Text in → LLM writes a scene → image generator draws it (an illustrated story maker). Once you see modalities as Lego bricks, the ideas don't stop.
Checkpoint
You want to build an app where a user talks to their phone and hears a spoken answer back. In the right order, which three pieces do you chain?
First you need to hear the user, so speech-to-text turns their voice into words. Then the LLM reads those words and writes a reply. Finally text-to-speech turns that reply into a spoken voice. Hear → think → speak — in that order.
Try it yourself — 10 minutes, no code
Pick one of these two experiments in any free multimodal chatbot (ChatGPT, Claude, or Gemini):
Keep: write one sentence on where the AI's extra sense was impressive — and one place it clearly guessed. That instinct for "trust vs. verify" is exactly what Station 8 sharpens.
Recap. "Multimodal" just means an AI that works in more than text — it can see images and hear sound. Vision lets you ask questions about a picture; image generation turns a well-described prompt into a new one (mind the garbled text and shaky details). Voice splits into speech-to-text and text-to-speech, which chain with an LLM into a talking assistant. The builder's real craft is piping these modes into each other — photo → answer → spoken explanation — one simple box at a time.