How Do You Change Basis in Linear Algebra?

TL;DR

Translating a vector from another basis into yours means multiplying it by the matrix whose columns are that person's basis vectors written in your coordinates; the inverse of that matrix goes the other way. To express a transformation in someone else's language, sandwich it: A inverse times M times A, where A is the change of basis matrix.

Transcript

If I have a vector sitting here in 2D space, we have a standard way to describe it with coordinates. In this case, the vector has coordinates 3, 2, which means going from its tail to its tip involves moving three units to the right and two units up. Now, the more linear algebra-oriented way to describe coordinates is to think of each of these numbe... Read More

Key Insights

• A coordinate system is any way to translate between vectors and sets of numbers, and the basis vectors are what give the numbers meaning. In the standard system, i-hat has length 1 pointing right and j-hat has length 1 pointing up.
• Coordinates work as scalars, not positions. The coordinates 3, 2 mean scale i-hat by 3, scale j-hat by 2, and add the results tip to tail, so the same numbers describe a different vector under a different basis.
• The grid drawn on 2D space is a construct, not a property of space itself. Space has no intrinsic grid, so the direction of the axes and the spacing of the grid lines depend entirely on the choice of basis vectors.
• The origin is the one thing every coordinate system agrees on. Everybody agrees that 0, 0 means the vector you get when you scale any vector by 0, so Jennifer's origin lines up with the standard one.
• The change of basis matrix has the other person's basis vectors, written in your coordinates, as its columns. With Jennifer's b1 at 2, 1 and b2 at negative 1, 1, that matrix converts her coordinates into yours.
• Applying the change of basis matrix is matrix-vector multiplication, which is the same as applying a linear transformation. Geometrically it moves i-hat and j-hat onto b1 and b2, transforming your grid into hers.
• The direction of the change of basis feels backwards because geometry and numbers point opposite ways. The matrix turns your misconception of what she means, the same coordinates read in your system, into the vector she actually meant.
• The inverse change of basis matrix translates your coordinates into hers by playing the first transformation backwards. For Jennifer's basis it has columns 1 third, negative 1 third and 1 third, 2 thirds, sending 3, 2 to 5 thirds, 1 third.
• A transformation matrix in another basis is built from three matrices applied in order: change of basis, then the transformation, then the inverse change of basis. The product takes in a vector in her language and returns the transformed vector in her language.
• The expression A inverse times M times A represents a shift in perspective. The middle matrix is a transformation as you see it, the outer two carry the empathy, and the full product is that same transformation as someone else sees it.

Questions & Answers

Q: What is a change of basis in linear algebra?

A change of basis is the translation of a vector's description from one coordinate system to another when the two systems use different basis vectors. Any way to translate between vectors and sets of numbers is called a coordinate system, and the basis vectors are the vectors that the coordinates are meant to scale. Since everyone is looking at the same vectors in space but using different words and numbers, a change of basis converts between those languages while leaving the underlying vector untouched.

Q: How do you convert coordinates from another basis into the standard basis?

Build a matrix whose columns are the other person's basis vectors written in your coordinates, then multiply that matrix by their coordinate vector. If Jennifer's basis vectors b1 and b2 look like 2, 1 and negative 1, 1 in the standard system, and she describes a vector as negative 1, 2, you compute negative 1 times b1 plus 2 times b2, which gives negative 4, 1. That is the same vector expressed in standard coordinates.

Q: Why does the change of basis matrix feel backwards?

It feels backwards because the geometry and the numbers move in opposite directions. Geometrically the matrix transforms your grid into the other person's grid, but numerically it translates a vector described in her language into yours. The way it clicks is to think of it as acting on your misconception of what she means, the vector you would get by reading her coordinates in your own system, and transforming that into the vector she actually meant.

Q: How do you go from the standard basis into someone else's basis?

Take the change of basis matrix that translates her language into yours, then use its inverse. The inverse of a transformation is a new transformation that corresponds to playing the first one backwards. For Jennifer's basis, that inverse works out to have columns 1 third, negative 1 third and 1 third, 2 thirds. Multiplying it by the vector 3, 2 gives 5 thirds, 1 third, which is how she describes that vector. Beyond two dimensions you would use a computer to compute the inverse.

Q: Why is matrix-vector multiplication the same as a change of basis?

Scaling each of her basis vectors by the corresponding coordinates and adding the results is exactly the operation matrix-vector multiplication performs, with a matrix whose columns represent her basis vectors in your language. Understood as applying a linear transformation, that matrix moves i-hat and j-hat, the vectors you think of when you say 1, 0 and 0, 1, onto her basis vectors, the vectors she thinks of when she says 1, 0 and 0, 1.

Q: How do you express a linear transformation in a different basis?

Start with a vector written in her language. First apply the change of basis matrix to translate it into your language. Then apply the transformation matrix on the left, which tells you where the vector lands, still in your language. Finally apply the inverse change of basis matrix on the left to express the result back in her language. Since this works for any vector, the composition of those three matrices is the transformation matrix in her language.

Q: What does A inverse times M times A mean?

It represents a mathematical sort of empathy. The middle matrix M is a transformation of some kind as you see it, and the outer two matrices carry the shift in perspective between coordinate systems. The full matrix product represents that same transformation but as someone else sees it. Whenever you encounter an expression in this form, it is a signal that a transformation is being viewed through a different set of basis vectors.

Q: Why can't you just translate the columns of a rotation matrix into another basis?

Because those columns record where your basis vectors i-hat and j-hat go, not where hers go. The standard 90 degree counterclockwise rotation matrix has columns 0, 1 and negative 1, 0 because i-hat lands at 0, 1 and j-hat lands at negative 1, 0. The matrix she wants must represent where her own basis vectors land, and it must describe those landing spots in her coordinates, which requires the full three-matrix composition instead.

Summary & Key Takeaways

• Coordinates only have meaning relative to a choice of basis vectors. Writing a vector as 3, 2 means scaling i-hat by 3 and j-hat by 2 and adding them tip to tail. Those two special vectors encapsulate the implicit assumptions of the coordinate system: that the first number means rightward motion, the second upward motion, and how far one unit of distance goes.

• Someone using different basis vectors describes the same vectors in space with different numbers. Jennifer's b1 and b2 look like 2, 1 and negative 1, 1 in standard coordinates, but in her own system they are 1, 0 and 0, 1. The vector described as 3, 2 in the standard system is 5 thirds, 1 third in hers, so the two systems are effectively different languages for the same space.

• To convert Jennifer's coordinates into standard ones, multiply by the matrix whose columns are her basis vectors expressed in standard coordinates. Her negative 1, 2 becomes negative 4, 1. Geometrically this matrix transforms the standard grid into her grid, while numerically it translates her description into the standard one. The inverse matrix performs the opposite conversion.

• Transformations, not just vectors, are described relative to a basis. A 90 degree counterclockwise rotation is written with columns 0, 1 and negative 1, 0 because those are the landing spots of i-hat and j-hat. Simply translating those columns into Jennifer's language is wrong, since her matrix must track where her basis vectors land and record the answer in her coordinates.

• The composition change of basis, transformation, inverse change of basis produces the transformation matrix in the other person's language. For Jennifer's basis and the 90 degree rotation, the product has columns 1 third, 5 thirds and negative 2 thirds, negative 1 third. Any expression of the form A inverse times M times A signals this shift in perspective.