Lecture 4: Fixed Optical Flow, Optical Mouse, Constant Brightness Assumption, Closed Form Solution

TL;DR

Fixed optical flow can be estimated by assuming constant brightness and a single velocity pair, u and v, for the whole image. The method accumulates spatial brightness gradients and temporal brightness changes across all pixels, then solves two linear equations with a symmetric 2×2 coefficient matrix. It can fail when that matrix is singular, so read on to understand the constraints, solution, and failure cases.

Transcript

[SQUEAKING] [RUSTLING] [CLICKING] PROFESSOR: We started off talking about the two aspects of image formation, where and how bright. And where we talked about perspective projections. And in a camera-centric coordinate system, it's very easy to just get with that, and the extent of that to be able to talk about motion. And so we just differentiated ... Read More

Key Insights

• 🎥 Perspective projection involves transforming 3D objects into 2D images using a camera-centric coordinate system.
• 🎥 Vanishing points can be used to determine camera rotation and calibrate cameras.
• 🎮 Camera calibration is important for accurate perception and control in robotics and autonomous systems.
• 😥 Calibration objects like cubes can be used to determine vanishing points and calibrate cameras.

Questions & Answers

Q: How is fixed optical flow estimated for an optical mouse?

Assume that every pixel moves with the same velocity components, u and v, as in the optical-mouse case. Estimate the spatial gradients Ex and Ey and the temporal brightness change Et across the image, accumulate the resulting totals, and solve the final two linear equations for u and v.

Q: What is the constant brightness assumption in optical flow?

The assumption says that the brightness of an environmental point's image does not change as that point is followed through successive frames. This lets motion be related to spatial brightness gradients and the time rate of brightness change.

Q: What is the brightness change constraint equation used for?

It provides a relationship between image velocity, the brightness gradient, and the temporal rate of brightness change. At one pixel it is only one linear equation in the two unknowns u and v, so it defines a line in velocity space rather than a unique motion.

Q: Why can’t optical flow be recovered from a single pixel?

A single pixel supplies one linear constraint on the two velocity components u and v. Unlike the one-dimensional case, that is insufficient to recover both components locally, so an additional constraint is required.

Q: Why does the fixed-motion assumption make the optical-flow problem highly overconstrained?

Under this assumption, the same u and v apply to every pixel in the image. That yields one constraint per pixel, potentially millions of equations, while there are only two unknowns, making the estimation setting very favorable.

Q: What closed-form system is solved for fixed optical flow?

After accumulating terms formed from Ex, Ey, and Et over the image, the method produces two linear equations in u and v. Their coefficients form a symmetric 2×2 matrix, so the two motion components can be obtained by solving that system when it is nonsingular.

Q: When does the fixed optical-flow solution fail?

The solution fails when the 2×2 coefficient matrix is singular, meaning its determinant is zero. Examples include a black image and cases where Ex or Ey is zero, because the image then lacks enough independent gradient information.

Q: Why does an image of the form f(ax + by) cause motion estimation to fail?

For this image, Ex and Ey remain in the same ratio everywhere, with Ey equal to b/a times Ex. Its constant-brightness contours are straight lines, and the dependent gradient measurements make the coefficient matrix singular, so the method cannot recover a unique motion.

Summary & Key Takeaways

• Perspective projection involves the transformation of 3D objects into 2D images using a camera-centric coordinate system.

• Vanishing points can be used to calibrate cameras and determine the rotation of the camera relative to the objects in the image.

• Camera calibration is important for various applications, such as robotics and autonomous vehicles, and can be done using calibration objects and vanishing points.