Coordinates

👋 About the exercise session

In this session, we will talk about coordinates which is another way of how to represent vectors. More specifically, so far we have represented vectors in their actual form, for instance if \(v \in R^3\), then we would write it as:

In this example, we need 3 numbers to represent the given vector. Let’s assume that to represent each of these numbers in computer memory, we need 4 bytes. Therefore, each such vector takes 12 bytes of memory in our computer.

Now, imagine that you are game developer. And you need to describe vectors that are part of some plane in the 3D space. You can stick to the above representation, and spend 12 bytes of memory per vector. Or you realize that the given plane is a subspace of 3D and has 2 basis vectors. More specifically, all vectors in this plane can be obtained by a linear combination of these two basis vectors:

where \(B = \{v_1, v_2\}\) represents the basis and \(u\) is any vector on the given plane. We can now introduce a new representation of \(u\) which we call coordinates of \(u\) with respect to the basis \(B\):

Notice that now we only need 2 numbers, or 8 bytes to represent the given vector \(u\). Therefore, we save \(4\) bytes per vector. If you now have \(10^6\) vectors, then this means you save \(4*10^6 \approx 4 Mb\) of memory. Hopefully, this got you at least a bit excited about this topic!

Therefore, in this exercise, we will first cover how to compute the coordinates of the given vector with respect to given basis. And also the other way around. We then look into special kind of bases called orthonormal which allow us to speed the computation of given coordinates even more compare to just the case where we have normal bases.

If you feel unsure about the concept of basis vectors, please revisit the previous session notes.

📓 Notes

How to compute the coordinates

We are in some vector space \(V\) with corresponding set of basis vectors \(B\). We know that for any \(v \in V\), we can write \(v\) as:

In words, we can write vector \(v\) as a linear combination of basis vectors. The scalars in the linear combination represent the coordinates of vector \(v\) with respect to the basis \(B\) in the given vector space \(V\):

But how do you find such linear combination in practice? Well, we can find them by solving the following system of linear equations:

where \(A\) is a matrix whose columns are basis vectors from \(B\), \(x\) are the coordinates of the vector that we are looking for (\([v]_B\)), finally \(b\) is the vector for which we want to find the coordinates (\(v\)).

Now, how about the other way around? How do I obtain the actual vector \(v\), given its coordinates \([v]_B\) and also the set of basis vectors \(B\)? Well following the above equation:

See the first exercise to see a concrete example. This is the most important skill from this session, therefore make sure you understand this well.

Interesting thing to notice: One thing, I want you to notice is that from a computer perspective, you need to find the inverse of \(A\) to find \(x\). Inverse is computationally heavy operation compare to just matrix multiplication which is actually very fast. This is because nowadays all computers have GPUs which are super good at matrix multiplication. In summary, we would like to avoid taking the inverse if we can… And turns out we can in certain cases - see the below section.

Orthonormal bases and orthogonal matrices

We are given a set of basis vectors \(B\). Our question is rather simple: is this set of vector orthonormal? Clearly, to be able to answer this question, we need to know what orthonormal means:

(1) All pairs of vectors from \(B\) are orthogonal to each other. In other words, for any \(b_1, b_2 \in B\), it holds true that \(u_1 \cdot u_2 = 0\).

(2) All vectors are normal, i.e., for any \(b \in B\), it holds true that \(|b| = 1\). In other words the size of the given vector must be 1 where size can be computed as:

and we define \(b\) as \(b = [b_1, \dots, b_n]\). So why is this actually useful? Well, it leads to the following property: \(A^{-1} = A^T\) where \(A\) is the matrix whose columns are vectors from the orthonormal basis set \(B\). Therefore, to find coordinates of \(v\) relative to orthonormal basis \(B\), we no longer have to use \([v]_B = A^{-1}b\), but instead, we can write:

Finding a transpose of \(A\) is for a computer very simple. Therefore, we just found yet another speedup of our program. 😎

⛳️ Learning goals checklist

From this week, the most important skills are:

• Be able to compute the coordinates of a vector relative to a basis, also in the case of orthonormal bases

• Be able to use the rules for the dot-product and use this in small proofs (see exercise 6)

Next, week we will finally see again some more practical applications when we will talk about Least squares method. Stay tuned and see you next week!