What Are Orthogonal Matrices and Gram-Schmidt?

TL;DR

Orthogonal matrices simplify linear algebra by offering an orthonormal basis, where their vectors are at right angles and of unit length. The Gram-Schmidt process converts independent vectors into an orthonormal set, enhancing numerical stability in calculations. This video explains key properties and examples, highlighting the importance of orthogonal matrices in mathematical transformations.

Transcript

OK, here's the last lecture in the chapter on orthogonality. So we met orthogonal vectors, two vectors, we met orthogonal subspaces, like the row space and null space. Now today we meet an orthogonal basis, and an orthogonal matrix. So we really -- this chapter cleans up orthogonality. And really I want -- I should use the word orthonormal. Orthogo... Read More

Key Insights

• ❓ Orthonormal bases are essential in numerical calculations due to their simplicity and stability.
• 😫 The process of transforming a set of independent vectors into an orthonormal basis is known as the Graham-Schmidt process.
• 💨 Orthonormal matrices are a type of matrix with orthonormal columns, providing a convenient way to represent orthogonal transformations.
• 👾 The projection matrix onto a column space simplifies to Q(Q^T), where Q is the orthonormal basis matrix.

Questions & Answers

Q: What are orthogonal matrices and Gram-Schmidt?

An orthogonal matrix is a square matrix whose columns are orthonormal, so Q^TQ = I and Q^T = Q^-1. Gram-Schmidt is the process used to turn independent, non-orthonormal vectors, such as the columns of A, into an orthonormal basis.

Q: What is the difference between orthogonal and orthonormal vectors?

Orthogonal vectors have an inner product of 0, meaning they point at 90-degree angles to one another. Orthonormal vectors are also normalized: each has unit length, so Qi^TQi = 1.

Q: Why are orthonormal bases useful in numerical linear algebra?

Orthonormal bases simplify calculations because their pairwise inner products are 0 and their individual squared lengths are 1. The lecture also explains that orthonormal vectors do not get out of hand through overflow or underflow, which makes them valuable in numerical linear algebra.

Q: Why does Q transpose times Q equal the identity matrix?

The entries of Q^TQ are dot products between the columns of Q. A column dotted with itself gives 1, while its dot product with every other orthonormal column gives 0, producing ones on the diagonal and zeros elsewhere.

Q: Does Q have to be square for Q transpose times Q to equal the identity?

No. Q may have only two orthonormal columns whose vectors contain many components, and Q^TQ still equals the identity matrix of the appropriate size. Squareness is required only for the lecture's conventional use of the name “orthogonal matrix.”

Q: What special property does a square orthogonal matrix have?

If Q is square and has orthonormal columns, Q^TQ = I shows that Q^T is its inverse. Therefore, Q^T = Q^-1.

Q: What is an example of an orthogonal matrix?

A permutation matrix is an example because its columns are unit vectors placed in different positions and are perpendicular to one another. Its transpose is also an orthonormal matrix, and multiplying Q^T by Q gives I.

Q: How does an orthonormal basis simplify a projection matrix?

For a matrix Q with orthonormal columns, the projection matrix onto its column space simplifies to QQ^T. If the orthonormal basis matrix is square, that projection becomes the identity matrix.

Summary & Key Takeaways

• The lecture begins by introducing orthogonal vectors and the concept of orthogonality in a basis, emphasizing the importance of orthonormal bases.

• The Graham-Schmidt process is then explained as a method for transforming a set of independent vectors into an orthonormal basis.

• The lecture highlights the benefits of orthonormal bases in terms of numerical calculations, as well as the relationship between orthogonal matrices and square matrices.

• Examples of orthogonal matrices and the projection matrix onto a column space are provided to further illustrate the concepts.

• The lecture concludes with the observation that orthonormal matrices can simplify calculations and the connection between the original matrix and the orthonormal matrix is represented by a triangular matrix.