Vector operations

What you will be able to do

A model often turns a ticket’s feature vector into one score by multiplying and adding. This lesson shows the four operations behind that step. Then you can follow how a model makes a score, and find a shape error before it breaks your code.

NotebookOptional: check the numbers in a notebookColab · Kaggle · GitHub

The notebook is optional: this lesson works with a calculator. It calculates the lesson’s scores, lengths and shapes with plain Python lists, with checks that say if your answer is right without showing it. Then it uses numpy to show Tomás’s shape error and why a reshape is the wrong fix. Its first cell installs numpy 2.5.3; Colab and Kaggle already have numpy.

Open in ColabOpen in KaggleView on GitHub

Colab and Kaggle open your own copy. They need a free Google or Kaggle account. Stuck? See the solution notebook.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

• How a ticket becomes a feature vector, and Priya’s 3 × 4 matrix X, from Represent observations: Worked example: Priya builds the feature matrix. The feature order is length, refund words, past tickets, photo.
• The shape of a matrix (rows × columns), from Represent observations: From many tickets to a table of numbers.

What you need to have ready (the notebook needs none of it)

• A calculator. A square root key helps for the length.
• Your c03-notes.md file, with the vectors of T-4104 and T-4105 from Represent observations: Guided practice: add two tickets. The guided practice of this lesson scores them. If they are missing or wrong, copy them from the expected result in that section. T-4104 is (4, 2, 2, 0) and T-4105 is (1, 0, 0, 0).
• A new heading in c03-notes.md: ### Vector operations.

The notebook was tested in Jupyter with Python 3.14.6 and numpy 2.5.3.

Where you meet this in AI

• A linear model’s score. A simple spam filter or ticket classifier gives each feature a weight, multiplies, and adds. That sum is a dot product. Larger models repeat the same step millions of times.
• The mean of many embeddings. A search system can describe a long document by the mean of the embeddings of its paragraphs. A mean of vectors is an addition followed by a scaling.
• A training step. In Module 4, a model learns with the update “new weights = old weights − learning rate × gradient”. That is a scaling and an addition of vectors.
• A shape error. “Shapes (1200, 4) and (5, 4) not aligned” is one of the most common error messages in machine learning code. You can read it after this lesson.

Add and scale vectors

Two features are easy to draw, so start with two. Take two tickets from the same customer and count two kinds of words in each: refund words and delivery words (“parcel”, “late”, “tracking”).

• First ticket: a = (3, 1): 3 refund words, 1 delivery word.
• Second ticket: b = (1, 2): 1 refund word, 2 delivery words.

Addition. Grace wants one vector for the customer’s whole conversation. She adds the counts, position by position: (3 + 1, 1 + 2) = (4, 3). In a picture, you put the arrow of b at the tip of a. The sum is the arrow from the start to the new tip.

Vector addition: put b at the tip of a. The sum a + b goes from the start to the new tip.

Text description of the chart

The horizontal axis is refund words and the vertical axis is delivery words, both from 0 to 5. Arrow a goes from (0, 0) to (3, 1). Arrow b goes from (0, 0) to (1, 2). A copy of b starts at the tip of a, (3, 1), and goes 1 to the right and 2 up, to (4, 3). The sum a + b goes straight from (0, 0) to (4, 3). If you start with b and then add a, you reach the same tip: the order of addition does not change the sum.

Scaling. Multiply every number in a vector by the same number, called a scalar. 2 × a = (6, 2). The arrow points in the same direction and is twice as long. Think of a ticket that is twice as long, with the same mix of words. A scale factor between 0 and 1 makes the arrow shorter. A negative factor turns it to the opposite direction.

Scaling changes the length of a vector, not its direction: 0.5a, a and 2a lie on one line.

Text description of the chart

Three arrows start at (0, 0) and lie on the same line. The shortest, 0.5a, ends at (1.5, 0.5). The middle one, a, ends at (3, 1). The longest, 2a, ends at (6, 2). All three point in the same direction. Only their length is different.

The mean of vectors uses both operations. Add the vectors, then scale by 1 ÷ (how many there are). The mean of a and b is (4, 3) × 0.5 = (2, 1.5).

As formulas, for vectors with two positions:

$$\mathbf{a} + \mathbf{b} = (a_1 + b_1,\ a_2 + b_2) \qquad c\,\mathbf{a} = (c\,a_1,\ c\,a_2)$$

Read them as “add position by position” and “multiply each position by c”. You can only add two vectors with the same dimension. A 4-feature vector plus a 3-feature vector has no meaning.

The dot product: a weighted score

Priya wants an urgency score for each ticket. She uses the four features from Lesson 1 and gives each feature a weight: a number that says how much the feature should count.

Feature	Value for T-4101	Weight	Value × weight
Length (hundreds of words)	2	0.5	1
Refund words	2	2	4
Past tickets	0	1	0
Photo	1	0.5	0.5
Sum			5.5

The score of T-4101 is 5.5. This “multiply position by position, then add” is the dot product. With the weight vector w = (0.5, 2, 1, 0.5) and the feature vector x = (2, 2, 0, 1):

$$\mathbf{w} \cdot \mathbf{x} = w_1 x_1 + w_2 x_2 + w_3 x_3 + w_4 x_4 = 1 + 4 + 0 + 0.5 = 5.5$$

Read it as “w dot x”. The result is one number, a scalar.

The weights tell you what the score cares about:

• A large positive weight (refund words, 2) makes the feature raise the score a lot.
• A weight of 0 makes the model ignore the feature.
• A negative weight makes the feature lower the score.

Priya’s three tickets get these scores:

Ticket	Features x	Products with w	Score w · x
T-4101	(2, 2, 0, 1)	1 + 4 + 0 + 0.5	5.5
T-4102	(1, 0, 3, 0)	0.5 + 0 + 3 + 0	3.5
T-4103	(3, 1, 1, 1)	1.5 + 2 + 1 + 0.5	5.0

This is how a linear model scores: a dot product of weights and features. Priya chose these weights by hand. A real model learns them from past tickets (Module 4). It usually also adds one constant number, called the bias. The score is not a probability. It is only a number for ordering tickets until somebody tests what a score of 5.5 means.

What the dot product measures

For two arrows, the dot product also has a picture. It is large and positive when the arrows point in a similar direction. It is 0 when they are at a right angle. It is negative when they point in opposite directions. Try it.

Explore

The dot product of two arrows

a

(3, 1)

b

(1, 2)

Dot product a · b

5

Length of a

3.162

Length of b

2.236

Angle

45°

a · b = 3 × 1 + 1 × 2 = 3 + 2 = 5

The angle is less than 90°, so the dot product is positive.

Drag the tips of a and b. The aqua bar is the projection of b on a: the part of b that points along a. The dot product equals the length of a times the length of that bar.

Text description of the explorer

The explorer shows two arrows that start at (0, 0), in a plot from −3 to 5 on both axes. At the start, a = (3, 1) and b = (1, 2). The readouts show the dot product 5, the length of a 3.162, the length of b 2.236 and the angle 45°. The calculation line shows a · b = 3 × 1 + 1 × 2 = 3 + 2 = 5. A dashed line goes from the tip of b to the line through a, at a right angle. Where it meets that line, at (1.5, 0.5), the projection of b on a ends. The projection is 1.581 long, and 3.162 × 1.581 = 5, the dot product. If b is at a right angle to a, for example b = (−1, 3), the projection has length 0 and the dot product is 0. If b points away from a, for example b = (−2, 1), the projection points backwards and the dot product is negative (−5).

The projection of b on a is the shadow of b on the line of a, when the light comes at a right angle. The dot product is the length of a times the signed length of that shadow. So the dot product combines two things: how long the vectors are, and how much they point the same way. Lesson 3 separates these two things.

The length of a vector

The length of a vector, also called its norm, is the length of its arrow. For a = (3, 1), the arrow goes 3 to the right and 1 up. Pythagoras’ rule gives:

$$\lVert \mathbf{a} \rVert = \sqrt{3^2 + 1^2} = \sqrt{10} \approx 3.162$$

Read it as “the length of a is the square root of the sum of the squares”. The same rule works for any dimension. For T-4101, (2, 2, 0, 1):

$$\lVert \mathbf{x} \rVert = \sqrt{2^2 + 2^2 + 0^2 + 1^2} = \sqrt{4 + 4 + 0 + 1} = \sqrt{9} = 3$$

The length is the dot product of a vector with itself, under a square root: a · a = 9 + 1 = 10, and √10 ≈ 3.162.

A vector with length 1 is a unit vector. To make one, divide each number by the length: a ÷ 3.162 = (0.949, 0.316). It points the same way, but it is exactly 1 long. Search systems often store embeddings as unit vectors. You see why in Lesson 3.

Matrix multiplication: many dot products at once

Priya does not score one ticket. She scores the whole matrix X from Lesson 1. She writes the weights as a column, a matrix with shape 4 × 1. Then each row of X gives one dot product with that column.

$$\underbrace{\begin{pmatrix} 2 & 2 & 0 & 1 \\ 1 & 0 & 3 & 0 \\ 3 & 1 & 1 & 1 \end{pmatrix}}_{X:\ 3 \times 4} \times \underbrace{\begin{pmatrix} 0.5 \\ 2 \\ 1 \\ 0.5 \end{pmatrix}}_{\mathbf{w}:\ 4 \times 1} = \underbrace{\begin{pmatrix} 5.5 \\ 3.5 \\ 5.0 \end{pmatrix}}_{3 \times 1}$$

Each result is one row of X “dot” the column w. These are the same three scores as in the table above. This is matrix multiplication: every row of the first matrix with every column of the second, one dot product for each pair.

You will not multiply large matrices by hand. A library does it. What you must check is the shape:

• A row of X and a column of W must have the same number of values. So the columns of the first matrix must equal the rows of the second.
• The result has the rows of the first matrix and the columns of the second.

$$(n \times k) \times (k \times m) \rightarrow (n \times m)$$

Read it as: the two inner numbers must be equal, and the two outer numbers give the shape of the result.

First matrix	Second matrix	Fits?	Result	What it means
3 × 4	4 × 1	Yes, 4 = 4	3 × 1	One urgency score for each of 3 tickets
1,200 × 4	4 × 1	Yes	1,200 × 1	One score for each ticket of the week
1,200 × 4	4 × 5	Yes	1,200 × 5	One score for each of the 5 teams, for each ticket
1,200 × 4	5 × 4	No, 4 ≠ 5	none	The weight matrix has one row per team. Turn it to 4 × 5 first
32 × 784	784 × 10	Yes	32 × 10	A batch of 32 digit images, 10 scores each (one for each digit)
1 × 384	384 × 1,200	Yes	1 × 1,200	One new ticket’s embedding, dot product with 1,200 past embeddings

The last row is the start of a search: 1,200 dot products in one step. Lesson 3 explains how to read them.

The transpose turns a matrix so that its rows become columns. A 5 × 4 matrix becomes 4 × 5. Row 1 becomes column 1. It is the fix for the fourth row of the table.

Order matters. X × W and W × X are different. Take X with shape 1,200 × 4 and W with shape 4 × 5. Then W × X has inner numbers 5 and 1,200, so it does not fit at all.

Explore

Check the shapes of a matrix product

Shape of X

3 × 4

Shape of W

4 × 5

Shape of X × W

3 × 5

Work for one cell

4 products, then a sum

Multiplications in total

60

The inner numbers match (4 = 4), so the product exists. The outer numbers give its shape: 3 × 5. Cell (1, 1) is the dot product of row 1 of X and column 1 of W.

Change the shapes of X and W. Row 1 of X (blue) and column 1 of W (orange) make cell (1, 1) of the result (aqua).

Text description of the explorer

The explorer shows X × W = result as grids of small squares. At the start, X has 3 rows (tickets) and 4 columns (features). W has 4 rows and 5 columns (one for each team). The result has shape 3 × 5: 15 cells. Each cell needs 4 products and a sum, so the product needs 3 × 4 × 5 = 60 multiplications. The plot highlights row 1 of X, column 1 of W and cell (1, 1) of the result. That row and that column make that cell. If you set the rows of W to 3, the inner numbers are 4 and 3. Then there is no result. The plot shows a question mark and “4 ≠ 3”.

Worked example: Tomás’s shape error

Tomás, the AI engineer, wants a score for each team, not one urgency score. He has a weight matrix for 5 teams. He runs the product on one week of tickets and gets an error:

ValueError: shapes (1200,4) and (5,4) not aligned: 4 (dim 1) != 5 (dim 0)

Priya reads the message with him, one part at a time.

1. The first shape, (1200, 4). The feature matrix X: 1,200 tickets, 4 features. That is correct.
2. The second shape, (5, 4). The weight matrix W: 5 rows and 4 columns. So W has one row for each team and one column for each feature.
3. The rule. The inner numbers must be equal: the 4 columns of X and the rows of W. W has 5 rows. 4 ≠ 5, so the product does not exist.
4. The meaning. Each team’s weights are in a row of W, but the product needs them in a column.
5. The fix. Transpose W. Its shape becomes 4 × 5. Now (1,200 × 4) × (4 × 5) fits.
6. Check the result shape. The outer numbers are 1,200 and 5. The result is 1,200 × 5: one score for each ticket and each team. That is what Tomás wanted.
7. Check one value by hand. Priya takes T-4101, row 1 of X, and the returns team, column 1 of the transposed W. She calculates that one dot product on paper and compares it with cell (1, 1) of the result.

In numpy, a professional writes X @ W.T: @ is matrix multiplication and .T is the transpose.

Guided practice: score two tickets

Use Priya’s weights w = (0.5, 2, 1, 0.5). Use the two tickets from Lesson 1: T-4104 = (4, 2, 2, 0) and T-4105 = (1, 0, 0, 0).

1. For T-4104, multiply each feature value by its weight. Write the four products.
2. Add the four products. This is the urgency score of T-4104.
3. Do the same for T-4105.
4. Put all five tickets (T-4101 to T-4105) in order from the highest score to the lowest.
5. Calculate the length of T-4104.
6. Write the shape of X (5 tickets, 4 features), the shape of w as a column, and the shape of the result.

Show the expected result

1. T-4104: 4 × 0.5 = 2, 2 × 2 = 4, 2 × 1 = 2, 0 × 0.5 = 0.
2. Score = 2 + 4 + 2 + 0 = 8.
3. T-4105: 0.5 + 0 + 0 + 0 = 0.5.
4. T-4104 (8), T-4101 (5.5), T-4103 (5.0), T-4102 (3.5), T-4105 (0.5).
5. √(16 + 4 + 4 + 0) = √24 ≈ 4.899.
6. (5 × 4) × (4 × 1) → 5 × 1: one score for each ticket.

Check: T-4105 asks to change an address. A very low score is what Grace expects for it. If your score for T-4105 is high, check that you used the weights in the same order as the features.

Run Python in your browser

Optional: check the scores with code

This is optional. The code calculates the three scores from the table above with plain Python lists. Add T-4104 and T-4105 to the dictionary and check your answers from the guided practice.

Python code (you can change it)# Priya's urgency weights, in the order: # length, refund words, past tickets, photo weights = [0.5, 2, 1, 0.5] tickets = { "T-4101": [2, 2, 0, 1], "T-4102": [1, 0, 3, 0], "T-4103": [3, 1, 1, 1], } def dot(u, v): # Multiply position by position, then add the products. return sum(a * b for a, b in zip(u, v)) for name, features in tickets.items(): print(name, dot(weights, features))

Independent variation: change one weight

Grace says: “A customer who writes again and again is the most urgent case for us.” Priya changes one weight: past tickets from 1 to 3. The other weights stay the same.

1. Calculate the new scores of T-4101, T-4102 and T-4103.
2. Does the order of the three tickets change? Which ticket moves, and why?

Show the answer

New weights: (0.5, 2, 3, 0.5).

• T-4101: 1 + 4 + 0 + 0.5 = 5.5 (no change: it has 0 past tickets).
• T-4102: 0.5 + 0 + 9 + 0 = 9.5.
• T-4103: 1.5 + 2 + 3 + 0.5 = 7.0.

The order changes from T-4101, T-4103, T-4102 to T-4102, T-4103, T-4101. T-4102 moves from last to first, because it has the most past tickets (3), and that feature now counts three times as much. One weight changes which ticket a senior agent reads first.

Common mistake

Making the shapes fit with the wrong operation

Symptom: Tomás gets the error “shapes (1200,4) and (5,4) not aligned”. He changes W from 5 × 4 to 4 × 5 with a “reshape” command. The error goes away. The scores look like normal numbers, but the team suggestions are much worse than before.

Cause: a reshape does not turn the matrix. It reads the 20 numbers row by row and refills them into the new shape. The numbers of different teams get mixed. The shapes now fit, so no error stops the code, but most weights are in the wrong place. A transpose turns rows into columns and keeps each team’s weights together.

Fix: before you change a shape, say what each direction means (“rows are teams, columns are features”). Use the transpose to swap rows and columns. Then calculate one cell of the result by hand and compare it, as Priya did in step 7.

Check your understanding

Predict the sign of the dot product for each pair, without calculating: (3, 1) and (6, 2); (3, 1) and (−1, 3); (3, 1) and (−3, −1). Then check one of them.

Show the answer

(3, 1) and (6, 2) point in the same direction, so the dot product is positive: 18 + 2 = 20. (3, 1) and (−1, 3) are at a right angle, so it is 0: −3 + 3 = 0. (3, 1) and (−3, −1) point in opposite directions, so it is negative: −9 − 1 = −10.

Check your understanding

A batch X has shape 64 × 784 and a weight matrix W has shape 784 × 10. Does X × W exist? If yes, what is its shape and what does one row mean? Does W × X exist?

Show the answer

X × W exists, because the inner numbers are equal (784 = 784). The result has the outer numbers: 64 × 10. One row is the 10 scores for one image of the batch. W × X has inner numbers 10 and 64. They are different, so W × X does not exist.

Check your understanding

In Priya's score, the weight of 'length' is 0.5. What happens to the scores if she sets it to 0? Explain why a weight of 0 can be a useful test.

Show the answer

Each score falls by 0.5 × length: T-4101 by 1, T-4102 by 0.5 and T-4103 by 1.5. The new scores are 4.5, 3.0 and 3.5. With weight 0, the model ignores length completely. If the scores and the order hardly change, length did not matter much for these tickets. Setting one weight to 0 shows how much one feature adds to a decision.

Recap

• Add vectors position by position. Scale a vector by multiplying every position by one number: the direction stays, the length changes. A mean of vectors is both.
• The dot product multiplies position by position and adds. With weights and features, it is the score of a linear model. It is positive for similar directions, 0 at a right angle and negative for opposite directions.
• The length (norm) of a vector is the square root of the sum of its squares. A unit vector has length 1.
• Matrix multiplication is many dot products at once. (n × k) × (k × m) → (n × m): the inner numbers must match. Fix a mismatch with a transpose, not a reshape, and check one cell by hand.

Next, you use dot products and lengths to find past tickets that are similar to a new one.

Next: Similarity and distance