Applied Machine Learning: Prediction vs. Estimation

TL;DR

Machine learning is powerful for prediction, but not reliable for estimation and inference in applied econometrics.

Transcript

Hi and welcome back to the third part of this webinar, where I give an introduction to machine learning from an applied perspective. Specifically targeted at researchers and practitioners of applied econometrics, who think about how to add machine learning to the econometric toolbox. And want to understand how novel tools based on machine learning ... Read More

Key Insights

• 🎰 Machine learning excels at prediction problems but may not be reliable for estimation and inference.
• 🎰 Biases can arise in machine learning models due to variable selection and regularization.
• ✋ High-dimensionality and correlations between variables can affect the stability and interpretation of machine learning output.

Questions & Answers

Q: What is the main focus of this webinar?

The webinar aims to provide an introduction to machine learning in the context of applied econometrics, focusing on its ability to improve prediction in empirical analysis.

Q: How does machine learning address the trade-off between model complexity and overfitting?

Machine learning uses regularization techniques and data to determine the optimal level of model complexity to avoid overfitting, balancing flexibility with generalization to new data.

Q: Can machine learning models provide reliable estimates of coefficients in linear regression?

No, the selection of variables by machine learning models like Lasso can lead to biases in coefficient estimation. Variables may be excluded or included based on correlations with other variables, leading to compactification or expansion biases.

Q: Is it possible to perform inference on machine learning output?

Inference on machine learning output is challenging due to the complexity of the models and data-driven tuning. Traditional methods like the bootstrap may not be suitable for assessing statistical uncertainty in machine learning predictions.

Summary & Key Takeaways

• Machine learning is effective for solving prediction problems and improving empirical analysis.

• Machine learning uses flexible functional forms, regularization to avoid overfitting, and data to determine the trade-off between model complexity and prediction accuracy.

• The output of machine learning models may not provide reliable estimation of coefficients or causal relationships due to biases in variable selection and regularization.