I’ve had to draw pictures of the operations more times than I’d care to admit. I hope that this post and the visuals here will help myself and others understand and remember these rules, if not serve as a reference. Please note that I’m assuming you have some understanding of what vectors and matrices are.

notation

Before we start talking about anything else, let’s talk about notation:

`m \times n` is a common way to describe a matrix or vector size.

• `m` (the leading dimension) is the number of rows in the matrix.
• `n` (the second dimension) is the number of columns in the matrix.

A `2 \times 3` matrix looks like this:

However, I might sometimes use an HTML table to illustrate my matrices. A `2 \times 3` matrix would then look like this:

rule of size

For any of the following multiplication operations, this rule must hold true:

Given `A` and `B`, where `A` and `B` are both a matrix or a vector, in order for `AB` (the product of `A` and `B`) to be valid (defined), then the number of columns in `A` must match the number of rows in `B`.

That is to say, if `A` is of size `m \times n`, and `B` is of size `p \times q`, then `n` and `p` must be equal.

Think of it this way:

The “inner” terms describing the sizes of the operands (`n` and `p` above) must match.

The size of the product then contains the number of rows in the leading operand, and the number of columns in the second operand. It’s easier to think of this as the combinations of the outer size terms:

So the product matrix (or vector) is going to be `m \times q` in size.

example

Let’s use a vector-matrix multiplication as an example. I’ll use lowercase `v` for our vector and uppercase `A` for our matrix.

Vectors are just `n \times 1` single-column matrices.

transposition

Noice if we want the product of `A` and `v` that we’re in trouble…

`v` is `(3 \times 1)` and `A` is `(3 \times 2)` in size. We don’t have a valid match of sizes here.

Neither `(3 \times 1)(3 \times 2)` nor `(3 \times 2)(3 \times 1)` put the inner size terms (cols to rows) at the same value.

In order to get this to work, we’ll have to transpose one of our operands. Basically all this means is that we “flip” or rotate the operand.

The transpose of `v` is:

We can now perform product of a `1 \times 3` vector and an `3 \times 2` matrix.

Note that we could have transposed, `A`, too:

resulting size

The result of a $(\boldsymbol 1 \times 3)(3 \times \boldsymbol 2)$ sized matrix multiplication will be $(\boldsymbol 1 \times \boldsymbol 2)$ in size.

Or, look at our alternative approach…

The result of a $(\boldsymbol 2 \times 3)(3 \times \boldsymbol 1)$ sized matrix multiplication will be $(\boldsymbol 2 \times \boldsymbol 1)$ in size.

multiplication

Now we need to define the actual rules of multiplication. This is where I always have to stop and draw a picture…

Let’s take the multiplication of two matrices:

The `(2 \times 2)` product matrix is defined as follows:

That’s pretty ugly to look at. This is where I tend to get lost trying to match up subscripts and superscripts. We can break down the problem a little bit to make it clearer.

dot-products

In the notation above, I’ve added two vector-vector multiplications (aka dot-products) next to each value of the resulting `(2 \times 2)` matrix.

For example,

Although this looks extra confusing, this will help us break up the problem into its smaller sub-problems.

Let’s look at it without the subscripts:

Much clearer… This is basically the definition of dot-product. There’s a lot of magical properties and limitations related to dot products, but this is all we need to know for now.

visualization

Let’s go back to our original example:

Click on the question marks in the table below…

Or we can try a bigger example:

Click on the question marks in the table below…

For further exploration, or additional calculations, I recommend matrix.reshish.com. I find this to be one of the better online matrix calculators.