There are two ways to see a vector, and you need both:
A vector is an arrow in space. It has:
In 2D, a vector v = (3, 1) means "go 3 steps right, 1 step up." Draw it from the origin — it's an arrow pointing to the point (3, 1).
A vector is a list of numbers:
The number of entries is the vector's dimension. A vector in 2D lives on a flat plane. A vector in 768D lives in a space you cannot visualize — but the same rules apply. This is the core trick of AI: math that works in 2D works in 768D exactly the same way.
Geometrically: place the tail of w at the tip of v. The result is the arrow from origin to the new tip. Algebraically: add component by component.
[3, 1] + [1, 2] = [4, 3]
In AI terms: if one vector represents "king" and another represents a "gender direction," adding them might approximate "queen." This is not a metaphor — word embeddings literally work this way.
Geometrically: stretch or shrink the arrow by factor c. If c is negative, flip the direction.
2 · [3, 1] = [6, 2] (stretch)
−1 · [3, 1] = [−3, −1] (flip)
In AI terms: scaling changes the "strength" or "intensity" of whatever the vector represents.
Geometrically: measures how much two vectors point in the same direction. Positive = similar direction, zero = perpendicular, negative = opposite directions.
[3, 1] · [1, 2] = 3×1 + 1×2 = 5
In AI terms: the dot product is the fundamental measure of similarity. When an AI system tells you two things are "similar," it computed a dot product (or its normalized cousin, cosine similarity). This is how recommendation systems find songs you'll like, how search engines rank documents, and how attention mechanisms in transformers decide which words are relevant to each other.
| Domain | What Gets Vectorized | Dimension | What the Dot Product Measures |
|---|---|---|---|
| LLMs (GPT, Claude) | Words / tokens | 768–12,288 | Semantic similarity — "king" is close to "queen," far from "bicycle" |
| Image Generation | Image patches | Typically 512–1,024 | Visual similarity — which patches of an image relate to each other |
| Recommendation | Users & items | 50–200 | Preference match — how much a user's taste vector aligns with an item's feature vector |
In every case, the AI system is doing the same thing: representing things as vectors, then comparing them with dot products. The magic is in how the vectors are constructed — that's where learning (training) comes in — but the operations are just the three you learned above.
Q1: A vector v = [−2, 5] is multiplied by the scalar c = 0.5. Geometrically, what happens to the arrow?
A. It flips direction and doubles in length B. It shrinks to half its length, direction unchanged C. It stays the same length but changes direction D. It rotates 90 degrees clockwiseQ2: Two vectors have a dot product of zero. What does this tell you?
A. They are identical B. They point in exactly opposite directions C. They are perpendicular — as dissimilar in direction as possible in that plane D. One of them must be the zero vectorQ3: An AI company claims their new model represents documents as 1,536-dimensional vectors and finds similar documents by computing dot products. Based on what you know, is this claim plausible?
A. No — dot products only work in 2D or 3D B. Yes — dot products work in any number of dimensions, and this is exactly how modern semantic search works C. No — similarity between documents requires a neural network, not vector math D. Yes, but only for documents shorter than 1,536 wordsQ4 (Challenge): You have a word embedding where king = [0.5, 0.8, 0.1], queen = [0.5, 0.6, 0.1], and man = [0.2, 0.7, 0.1]. Without computing the exact number, which pair likely has the largest dot product?
A. king · man B. king · queen C. queen · man D. They are all equal