Statistical Learning: 7.4 Generalized Additive Models and Local Regression

TL;DR

Local regression is a popular method for fitting non-linear functions by focusing on the data region, while generalized additive models allow for fitting non-linear functions in multiple variables while retaining the additivity of linear models.

Transcript

okay well so we've already got a long list of of of ways of of fitting non-linear functions there's another whole family of of methods called local regression in fact rob wasn't there a phd thesis from stanford on local regression i don't remember rob's phd pieces was called local likelihood and was an extension of these local regression ideas to a... Read More

Key Insights

• 😥 Local regression is a widely used method for fitting non-linear functions by focusing on a local set of points and using weighted least squares.
• 🥺 Generalized additive models allow for fitting non-linear functions in multiple variables while retaining the additivity of linear models, leading to interpretable models.
• 👾 GAMs can be fitted using tools like natural splines in R, and the contributions of each variable can be visualized.
• 👾 GAMs can also be used for classification by fitting the logit of the probability as an additive model.
• 👾 The "mgcv" package in R provides an alternative approach to fitting GAMs using smoothing splines.
• 👾 GAMs are a useful tool for finding non-linearities in variables and their interactions.
• 👾 GAMs can be compared and tested using methods like ANOVA.

Questions & Answers

Q: What is local regression and how does it differ from other methods of fitting non-linear functions?

Local regression focuses on fitting a linear function to a local set of points, with weights determined by a kernel. This method provides better extrapolation at boundaries compared to other local fitting methods.

Q: How do generalized additive models (GAMs) retain the additivity of linear models while fitting non-linear functions?

GAMs fit non-linear functions in multiple variables by considering each variable's contribution separately. The additivity constraint allows for an interpretable model, where the effects of each variable can be visualized.

Q: Can GAMs be used for classification as well?

Yes, GAMs can be used for classification by fitting the logit of the probability as an additive model. This allows for the visualization of the contributions to the logit, providing insights into the model's effects.

Q: What tools can be used to fit GAMs in R?

GAMs can be fitted using the "gam" function in the "mgcv" package or using natural splines with the "lo" or "s" symbols in the "gam" function. The resulting functions can be plotted using the "plot.gam" function for a better understanding of the model's effects.

Summary & Key Takeaways

• Local regression is a family of methods that fit non-linear functions by focusing on a local set of points and using weighted least squares to determine the weights for fitting the function.

• Generalized additive models (GAM) allow for fitting non-linear functions in multiple variables while maintaining the additivity of linear models, making them interpretable. GAMs can be easily fit using tools like natural splines in R.

• GAMs can also be used for classification, where the logit of the probability is fit as an additive model. The contributions to the logit are plotted to visualize the model's effects.