Station 5 · Memory
An LLM is brilliant but forgetful — it never read your notes, and it stopped learning on a fixed date. RAG is the trick that lets it answer from your own documents, today's facts, that one specific PDF — without retraining anything.
What you'll walk away with
Concept 1
The model you met in Station 1 learned from a giant pile of text — but that pile was frozen at a certain date, and it never contained the things that matter most to you.
Two gaps show up the moment you try to build something real:
Because the model predicts likely text (Station 1), when it doesn't actually know, it doesn't stop — it produces a believable-sounding answer anyway. Asking a cutoff model about your private PDF is a fast path to a confident, wrong answer. We need to hand it the facts instead of hoping they're in its memory.
Concept 2
RAG stands for Retrieval-Augmented Generation. Long name, simple idea: before the model answers, go find the relevant facts and paste them into the prompt.
Break the name apart and it explains itself:
Here's the core loop, every single time a question comes in:
A closed-book exam forces you to answer from memory — and you'll bluff on anything you forgot. An open-book exam lets you flip to the right page first, then write the answer. RAG turns every question into an open-book exam: the model gets to read the correct page before it responds. Same student, dramatically more reliable answers.
Another way to picture it: RAG is a librarian. You ask a question, the librarian sprints into the stacks, pulls the two or three books that actually cover it, and sets them open on your desk. The model isn't smarter — it just isn't guessing anymore, because the right material is sitting right in front of it.
This ties directly back to Station 1: if you put real facts into the text-so-far, the next-word guesses lean on them. RAG is just an automatic, at-scale way of stuffing the right facts into the text-so-far every time.
RAG never changes the model. It changes what the model sees. You don't make the AI smarter — you make sure the exact facts it needs are already in the prompt when it starts predicting. Retrieve → stuff → answer.
Concept 3
Retrieval has one hard job — find the most relevant chunks. Not chunks with matching keywords, but chunks with matching meaning. To do that by computer, we first turn text into numbers.
An embedding is a list of numbers (a vector) that represents a piece of text's meaning. A model reads a sentence and outputs, say, a few hundred numbers. The magic rule: text with similar meaning gets similar numbers, so it lands nearby in this "meaning space."
Some intuition for what "nearby" means:
That last one is the whole point. Keyword search would miss it (no shared words), but embeddings capture that the two sentences mean the same thing. Here's the idea in pseudocode:
# Turn each piece of text into a vector (a list of numbers). # Real vectors have hundreds of numbers — shown short here. q = embed("How do I get my money back?") note_a = embed("Our refund policy allows returns within 30 days.") note_b = embed("The store opens at 9am on weekdays.") # similarity() is high when meanings are close, low when far apart. similarity(q, note_a) # high — both about refunds similarity(q, note_b) # low — unrelated topic # So to answer the question, retrieve note_a — not note_b.
Concept 4
Once every chunk of your documents is an embedding, you need somewhere to keep them — and a way to instantly find the closest ones to a question.
A vector database is exactly that: a store built to hold thousands (or millions) of vectors and, given a new question's vector, return the handful that sit nearest to it in meaning space — in milliseconds. That's the "memory" in "give AI a memory." It's not the model remembering; it's a searchable filing cabinet the model gets to consult.
Question goes straight to the model. It answers from frozen, general memory — knows nothing about your PDF, and may bluff on anything past its cutoff.
Question is turned into a vector, the database returns the closest chunks of your docs, and the model answers grounded in those real facts.
Notice what each piece does: the embedding model converts text to vectors, the vector database finds the nearest ones, and the LLM writes the final answer. Three different tools, one clean pipeline — and you, the builder, are the one wiring them together.
Concept 5
You don't embed a whole 40-page PDF as one giant vector. You cut it into pieces first. This step quietly decides whether your whole RAG system works or falls apart.
Chunking means splitting a document into smaller pieces — a paragraph, a few sentences, a section — and embedding each piece on its own. Why bother?
But chunking is also where RAG breaks. Retrieval only ever gives the model the chunks it fetched — so if those are bad, the answer is bad. This is the builder's favorite phrase for it:
Common failure modes: chunks cut mid-sentence so a fact is split in half and neither piece matches; chunks so huge they're vague; the truly relevant chunk ranks 6th but you only fetched the top 5; or your documents simply don't contain the answer, so retrieval hands over the "closest" irrelevant thing and the model confidently runs with it. When a RAG app gives a wrong answer, the bug is usually in retrieval, not the model.
So the builder's real craft here isn't the LLM at all — it's chunking documents sensibly, retrieving enough (but not too much), and checking that the fetched chunks actually contain the answer before trusting it.
Checkpoint
Your RAG chatbot gives a confidently wrong answer about your company handbook. Where should you look first?
RAG never changes the model — it changes what the model sees. If the answer is wrong, the most common cause is that retrieval handed over the wrong (or incomplete) chunks: bad chunking, too few results fetched, or the fact just isn't in your documents. Fix the retrieval step and the model has what it needs. A bigger model can't answer from facts it was never shown.
Try it yourself — 15 minutes, no code
You're going to run the RAG loop by hand, playing the part of the vector database yourself:
Reflect: that worked because you had one page. Now imagine 10,000 pages — you can't paste them all, and you can't read them all to find the right one. That's why real RAG automates retrieval with embeddings and a vector database: to be the fast, tireless you that always finds the right page.
Recap. Models have a knowledge cutoff and never read your private stuff, so left alone they guess. RAG fixes that without touching the model: retrieve the most relevant chunks of your own documents, stuff them into the prompt, and let the model answer from real facts — an open-book exam every time. Under the hood, embeddings turn text into vectors so similar meanings sit near each other, a vector database instantly finds the nearest chunks to a question, and smart chunking makes those matches precise. Get retrieval right and the model stops bluffing; get it wrong and no model can save you. It's all just Station 1's rule, automated: put real facts into the text-so-far, and the next-word guesses lean on them.