Phase 3 — Core AI Engineering · Lesson 29 · 20 XP
Chunking and the RAG pipeline
You can't just embed a whole document — it may exceed the embedding model's input limit, and even when it fits, one vector for an entire document is too blunt for retrieval: it can't tell a reader which part is relevant. Chunking splits documents into smaller, focused pieces first.
A common strategy is fixed-size chunks (e.g. ~500 tokens) with some overlap between consecutive chunks, so a fact sitting right at a chunk boundary doesn't get split apart from the context it needs. The full RAG pipeline: chunk documents → embed each chunk → store in a vector table (Lesson 28) → embed the incoming query → retrieve the closest chunks → include them in the prompt → generate the final answer.
Exercise
Write a chunker that splits a long document into overlapping ~500-token chunks. Embed each chunk and build a minimal retrieve-then-generate pipeline over 3-4 source documents.
Check yourself
1. Why include overlap between chunks instead of splitting on hard, non-overlapping boundaries?
2. What goes wrong for retrieval if your chunks are too large? What goes wrong if they're too small?
Vector search with pgvector
Answer the check-yourself questions to unlock this