Matrices

đź‘‹ About the exercise session

Last week, we talked about how to translate a problem into a system of linear equations, and then by converting the system into a matrix, how to find a solution (if exists) of the given problem. This week, we will look more closely into matrices and how we can interact with them via different operations. Without further ado, let’s get into the topic!


đź““ Notes

Working with matrices using matrix algebra

From a general perspective, not many things change when it comes to manipulating matrices. I believe book gives a good summary of all possible operations as well rules. (chapters 2.1 and 2.2) However, you should especially remember that, generally speaking, matrix multiplication is not commutative, i.e., \(AB \not = BA\). One of the other highlights is to understand what are the dimenions of the given matrix. For instance if we say \(A\) is \(4 \times 2\), it means it is a matrix with 4 rows and 2 columns.

Matrix multiplication

Matrix multiplication can be viewed from two perspectives:

  1. The computation itself (described below and must know for this course)

  2. Visual intuition (not focus of this course, but nice addition)

Let’s start with the computation:

  1. Can given two matrices be multiplied? To answer this question, write their dimensions down: \((n \times m) @ (k \times p)\). Now, my intuition is that I essentially see 4 numbers, two inner ones (\(m, k\)) and two outer ones (\(n, p\)). To check whether I can multiply the two matrices together, I look at the inner numbers - if they match continue to the step 2, if not it is not possible. Here I would like to note that in practice, it is sometimes possible to take transpose of a matrix which swaps its dimensions, which might enable the matrix multiplication.

  2. At this point, I want to see what output I should expect - for this look at the outer numbers. Based on step 1, the output should be \(n \times p\) matrix.

  3. Finally, the computation itself. You can follow this pattern:

  • take first row (left matrix) and do a dot product with first column (second matrix), this gives you the very first number in the output matrix, i.e., in the left upper corner

  • if the output matrix has more than one column, simply continue by again doing dot product between first row (left matrix) and second column (right matrix). Continue like this until you fil the entire first row of the output matrix.

  • Hopefully, now you can see the pattern. To fill the second row of the output matrix, start by doing dot product between the second row (left matrix) and first column (right matrix), and so on…

What is dot product you may ask? Given two n-dimensional vectors \(u, v\), you compute dot product between these two as follows:

\[u \cdot v = u_1 \times v_1 + \dots + u_n \times v_n\]

where for instance \(u_1\) corresponds to the first element in vector \(u\).

Now the visual part, for this I suggest you take a look at the exercise 2 from this week. Here I explain, how matrices can be viewed as linear transformations. Last, but not the least, one of the most important things to remember about


Inverse of a matrix

In the first lecture, we talked about solving system of linear equations via matrices. More specifically, we can write the problem as follows:

\[Ax = b\]

To solve for \(x\), we need to first separate it on the left side. To do this, we can multiply both sides by inverse of matrix A:

\[\begin{aligned} A^{-1}Ax = A^{-1}b \\ x = A^{-1}b \end{aligned}\]

The natural question arises which is how do we actually find inverse of a matrix? There are essentially two ways:

  • If the matrix is \(2 \times 2\) dimension wise, we can use closed formula. Given some matrix \(E\):
\[\begin{aligned} &E=\left[\begin{array}{ll} a & b \\ c & d \end{array}\right] \end{aligned}\]

Then:

\[\begin{aligned} &E^{-1}=\frac{1}{a d-b c}\left[\begin{array}{cc} d & -b \\ -c & a \end{array}\right] \end{aligned}\]


Elementary matrices

Elementary matrices are a special kind of matrices that are obtained by a single row operation from an identity matrix. For instance \(A\) is not an elementary matrix:

\[A=\left[\begin{array}{ll} 0 & 2 \\ 1 & 0 \end{array}\right]\]

This is because it was obtained via more than one row operation (swap rows, row multiplication). On the other hand, \(B\) can be an example of an elementary matrix (we just swapped rows):

\[B=\left[\begin{array}{ll} 0 & 1 \\ 1 & 0 \end{array}\right]\]

Why do we care about elementary matrices? Well, they allow us to decompose some complex linear transformation into elementary linear transformations. Given \(A\) is a square matrix representing a complex linear transformation, we can write:

\[E_k \dots E_1 A = I\]

In words, by applying row operations (represented by elementary matrices), we can transform \(A\) into an identity matrix \(I\) (reduced row echelon form). However, we are interested in how to express \(A\) as a product of elementary matrices. Therefore we can write:

\[A = E_1^{-1} \dots E_k^{-1}\]

As you know from the book, inverse of an elementary matrix is still an elementary matrix. Therefore we have expressed \(A\) (some complex linear transformation) as a series of elementary linear transformations. See more in exercise 6.


⛳️ Learning goals checklist

Amazing, second week is behind you! This week’s in-class exercises were quite theoretical and included a lot of ideas to digest, so do not worry if you do not understand everything. Main takeaways for this weeks are more practical:

  • Be able to multiply matrices
  • Be able to compute the inverse of matrices using Gauss-Jordan elimination
  • Be able to compute the inverse of a \(2 \times 2\) matrix using the closed formula
  • Apply the equations of matrix algebra to reason about matrices (see exercise 2.3.66 for an example)
  • Construct the elementary matrix corresponding to a given row operation

Looking forward to see you next week!