Logistic Regression - THE MATH YOU SHOULD KNOW!

TL;DR

Logistic regression is a statistical method used for classification problems in supervised learning, in which the goal is to predict the probability of the response variable being either 0 or 1.

Transcript

hey guys in this video we're gonna take a look at a statistical learning method for classification called logistic regression in a nutshell I would say logistic regression is linear regression but for classification problems in supervised learning I created a video on linear regression not too long ago check it out if you already haven't the link w... Read More

Key Insights

• ❓ Logistic regression is a statistical method used for classification problems.
• 😒 It uses the logistic function to model the probability of the response variable being 1 or 0.
• ❓ Parameter estimation in logistic regression is done using the maximum likelihood method.
• 🏛️ Logistic regression is not suitable for multi-class classification using linear regression.
• 🧑‍💻 The log-likelihood function is used to optimize the parameters.
• ❓ The Newton-Raphson method is one approach to approximate the coefficient vector.
• 🏛️ Logistic regression requires a threshold to determine the class of a given feature vector.

Questions & Answers

Q: Why can't we use linear regression for classification problems?

Linear regression cannot handle multi-class classification, as it assumes a continuous range of values. It also predicts values outside the range of 0 and 1, making it unsuitable for binary classification.

Q: How does logistic regression model the probability of the response variable?

Logistic regression uses the logistic function, which maps any real-valued number to a value between 0 and 1. This allows it to model the probability of the response variable being 1 or 0.

Q: How does logistic regression estimate the parameters?

Logistic regression uses the maximum likelihood method, which aims to find the coefficients that maximize the likelihood of the data. It splits the training data into two groups based on their labels and estimates separate coefficients for each group.

Q: Does logistic regression have any limitations?

Logistic regression assumes a linear relationship between the predictors and the log odds of the response variable. It may not perform well if the relationship is non-linear or if there are interactions between predictors.

Key Insights:

• Logistic regression is a statistical method used for classification problems.
• It uses the logistic function to model the probability of the response variable being 1 or 0.
• Parameter estimation in logistic regression is done using the maximum likelihood method.
• Logistic regression is not suitable for multi-class classification using linear regression.
• The log-likelihood function is used to optimize the parameters.
• The Newton-Raphson method is one approach to approximate the coefficient vector.
• Logistic regression requires a threshold to determine the class of a given feature vector.
• Other numerical methods like the secant or Moeller's method can also be used for approximation in logistic regression.

Summary & Key Takeaways

• Logistic regression is a technique used for classification problems, where linear regression is not suitable.

• It uses the logistic function to model the probability of the response variable being 1 or 0.

• Parameter estimation in logistic regression uses the maximum likelihood method to find the coefficients that maximize the likelihood of the data.