Lecture 3 | Convex Optimization I (Stanford)

Transcript

this presentation is delivered by the Stanford center for professional development um so actually first of all embarrassingly I guess I have to introduce myself even though it's the second week so um and I have to apologize for not being here last week I've actually for the record I've never done that before so I've never missed an entire first wee... Read More

Summary:

This video is a lecture on convex functions delivered by the Stanford center for professional development. The lecturer introduces himself and announces some administrative updates, including office hours and changes to the grading scale for homework assignments. He then explains the concept of convex functions and explores various examples. The lecturer covers first-order and second-order conditions for convexity and introduces the idea of Taylor approximations. He concludes by discussing the extension of functions to include infinity values for points outside the domain.

Questions & Answers:

Q: What are convex functions?

Convex functions are mathematical functions that have a domain which is a convex set and satisfy an inequality that demonstrates upward curvature or "convexity".

Q: How can one determine if a function is convex on the real number line?

Determining convexity on the real number line is relatively simple: plot the function and observe if it curves upward. If it does, then the function is convex.

Q: What are some examples of convex functions?

Examples of convex functions include affine functions, exponential functions (with a positive coefficient), power functions (depending on the value of the exponent), and negative entropy.

Q: How can one determine if a function is convex on matrices?

To determine if a function is convex on matrices, one can use the first-order condition for convexity, which involves testing if the gradient of the function is less than or equal to zero.

Q: What is the significance of the Taylor approximation in convexity?

The first-order Taylor approximation provides a global underestimate of a convex function. This allows for the use of local information, such as gradients, to make global statements about the function.

Q: What is the second-order condition for convexity?

The second-order condition for convexity states that a function is convex if and only if its Hessian matrix (second derivative matrix) is positive semi-definite.

Q: Are there any known examples of functions that are convex in each variable separately but not jointly convex?

Yes, functions can be convex in each variable separately but not jointly convex. The second derivative fails to be positive at the origin in such cases.

Q: Can a function be convex when restricted to any line and still not be convex in general?

No, if a function is convex when restricted to any line, then it is convex in general. However, this requires evaluating the function on all possible lines, which is not always practical.

Q: What is the purpose of encoding the not domain of a function as infinity?

Encoding the not domain of a function as infinity allows for the definition of an extended valued extension of the function. This makes it easier to analyze and work with the function, as it allows one to consider points outside the domain.

Q: How can one characterize the convexity of a quadratic function?

A quadratic function is convex if and only if the matrix P in its expression is positive semi-definite.

Q: Can the log sum exponent function be considered a convex function?

Yes, the log sum exponent function is a convex function due to its smoothness and the fact that it can be seen as a soft maximum calculation. It is often called the "soft max" function.