How Does the Logistic Regression Decision Boundary Work? — #33 Machine Learning Specialization [Course 1, Week 3, Lesson 1]

TL;DR

A logistic regression decision boundary separates predictions of 1 and 0 where w dot X plus b equals zero. The model first computes Z, applies the sigmoid function, and commonly predicts 1 when the output is at least 0.5. With two basic features the boundary is a straight line, while polynomial features can produce circles, ellipses, and more complex curves. Read on to see how parameters and features determine these boundaries.

Transcript

in the last video you learned about the logistic regression model now let's take a look at the decision boundary to get a better sense of how logistic regression is Computing is predictions to recap here's how the logistic regression models outputs are computed in two steps in the first step you compute Z as w dot X plus b then you apply the sigmoi... Read More

Key Insights

• 🖱️ Logistic regression uses the sigmoid function to compute predictions based on the calculated value of Z.
• 😃 The decision boundary separates the predicted values of 0 and 1, determined by whether w dot X plus b is greater than or equal to 0 or less than 0.
• ❓ Polynomial features can be introduced to create more complex decision boundaries in logistic regression.
• ❓ The threshold of 0.5 is commonly used to determine the predicted value of 0 or 1.
• ✋ Logistic regression can fit complex data by including higher-order polynomial terms.
• 🫥 Without higher-order polynomials, the decision boundary in logistic regression will always be a straight line.
• 🫥 The decision boundary can be visualized as lines or curves in two-feature examples.

Questions & Answers

Q: What is the decision boundary in logistic regression?

The decision boundary is the line or curve where w dot X plus b, or Z, equals zero. On one side, where Z is greater than or equal to zero, logistic regression predicts 1; where Z is less than zero, it predicts 0.

Q: How does logistic regression compute a prediction?

Logistic regression first computes Z as w dot X plus b. It then applies the sigmoid, or logistic, function: 1 divided by 1 plus e to the negative Z. The result is interpreted as the probability that Y equals 1 given X and parameters W and B.

Q: How does logistic regression decide whether to predict 0 or 1?

A common threshold is 0.5. If the sigmoid output f of x is greater than or equal to 0.5, the model predicts 1; if it is below 0.5, the model predicts 0.

Q: Why does a sigmoid threshold of 0.5 correspond to Z equals zero?

The sigmoid function is greater than or equal to 0.5 whenever Z is greater than or equal to zero. Because Z equals w dot X plus b, the same prediction rule can be written as predicting 1 when w dot X plus b is at least zero.

Q: What is the decision boundary when W1 and W2 are 1 and b is negative 3?

With two features, Z becomes X1 plus X2 minus 3. Setting Z equal to zero gives the decision boundary X1 plus X2 equals 3. The model predicts 1 to the right of this line and 0 to its left.

Q: Can logistic regression have a nonlinear decision boundary?

Yes. Adding polynomial features allows logistic regression to form curved and more complex decision boundaries. The transcript gives circles, ellipses, and irregular curves as examples of shapes produced by different features and parameters.

Q: How can polynomial features create a circular decision boundary?

Set Z to W1 times X1 squared plus W2 times X2 squared plus b, with W1 and W2 equal to 1 and b equal to negative 1. The boundary Z equals zero then becomes X1 squared plus X2 squared equals 1. The model predicts 1 outside or on the circle and 0 inside it.

Q: When is a logistic regression decision boundary always linear?

The boundary is always linear when the model uses only the original features X1, X2, X3, and so on, without higher-order polynomial terms. In a two-feature example, that linear boundary appears as a straight line.

Summary & Key Takeaways

• Logistic regression models compute outputs in two steps: computing Z as w dot X plus b, then applying the sigmoid function to Z.

• The sigmoid function, or logistic function, determines the probability that Y is equal to 1 given X and parameters W and B.

• A threshold of 0.5 is commonly used to decide whether to predict 0 or 1, with values above the threshold predicting 1.