Math: vectors, dot products, probability

A vector is just a list of numbers. The dot product of two vectors — multiply matching positions, sum the results — measures how much they point in the same direction. This single operation is the foundation of embeddings and similarity search, which you'll use constantly from Phase 3 onward.

import math

def dot(a, b):
    return sum(x * y for x, y in zip(a, b))

def cosine_similarity(a, b):
    return dot(a, b) / (math.sqrt(dot(a, a)) * math.sqrt(dot(b, b)))

Cosine similarity divides the dot product by both vectors' lengths, so it measures direction (meaning) while ignoring magnitude (e.g. text length). A probability distribution assigns a non-negative weight to every possible outcome, summing to 1. Softmax is the function that turns arbitrary scores into exactly that — it's what lets a model turn its raw next-token scores into token probabilities to sample from.