Lecture 5 Part 1: Derivative of Matrix Determinant and Inverse

TL;DR

The video discusses the importance of norms in defining derivatives and gradients, and the relationship between the gradient and the cofactor matrix in the context of calculating determinants.

Transcript

[SQUEAKING] [RUSTLING] [CLICKING] STEVEN G. JOHNSON: OK, so last time I talked about how in order to define a gradient, you need an inner product. So that way, if you have a scalar function of a vector, the gradient is defined-- basically the derivative has to be a linear function that takes a vector in and gives you a scalar out. So it turns out t... Read More

Key Insights

• 🍉 Norms are essential in defining derivatives and gradients, as they help measure the length or distance of vectors and determine the smallness of terms.
• 😚 The cofactor matrix is the gradient of the determinant, establishing a close relationship between these two mathematical concepts.
• ❓ Different proofs, including Laplace expansion and perturbations near the identity matrix, can be used to demonstrate the gradient of the determinant.
• ⛔ Determinants are useful for determining exact singularity of matrices, but their practicality is limited in numerical computations due to accuracy issues.

Questions & Answers

Q: Why is a norm necessary to define a derivative and gradient?

A norm is needed to measure the length or distance of a vector, which is crucial for determining the "smallness" of terms in derivative calculations. Without a norm, it is impossible to define what it means for a term to be small compared to a given value.

Q: What is the relationship between the cofactor matrix and the gradient of the determinant?

The cofactor matrix is equal to the gradient of the determinant. This can be proven through various methods, including Laplace expansion and analyzing small perturbations near the identity matrix.

Q: Why are determinants of limited usefulness in numerical computations?

Determinants can be challenging to compute accurately in finite precision calculations. Additionally, they do not provide a comprehensive understanding of matrix singularities, as the concept of conditioning numbers is more suitable for evaluating how close a matrix is to being singular.

Q: How are eigenvalues computed on a computer?

The characteristic polynomial is not commonly used to compute eigenvalues. Instead, algorithms such as the QR algorithm are employed, which involve iterative factorizations of the matrix into orthogonal and upper triangular forms. The eigenvalues are obtained through this process.

Summary & Key Takeaways

• The video explores the relationship between gradients, norms, and determinants, highlighting the need for norms to define derivatives and gradients.

• The cofactor matrix is shown to be the gradient of the determinant, providing a simple proof for this relationship.

• Two different proofs for the gradient of the determinant are presented, one based on Laplace expansion and another using small perturbations near the identity.

• The video also mentions the limited usefulness of determinants in numerical computations and introduces the concept of conditioning numbers for evaluating matrix singularities.