# Statistical Learning: 9.3 Feature Expansion and the SVM | Summary and Q&A

2.1K views
October 7, 2022
by
Stanford Online
Statistical Learning: 9.3 Feature Expansion and the SVM

## TL;DR

Soft margin alone may not be sufficient for effective separation of data, but feature expansion and kernels provide a solution.

## Install to Summarize YouTube Videos and Get Transcripts

### Q: How does feature expansion help overcome the limitations of soft margin?

Feature expansion involves transforming variables, such as polynomials, to create additional dimensions. This increases the likelihood of achieving separation in a higher-dimensional space, leading to better classification results.

### Q: What is the impact of projecting the enlarged space back to the original space?

When the enlarged space is projected back, the decision boundary appears non-linear in the original variables. This allows for more complex and accurate classification, as demonstrated by the example with conic sections of a quadratic polynomial.

### Q: Why are polynomials not the ideal choice for non-linear support vector machines?

Polynomials can result in a large, unwieldy feature space, especially in high dimensions. This complexity can lead to overfitting and computational challenges. Hence, there is a need for a more controlled and elegant approach.

### Q: What is the role of kernels in support vector classifiers?

Kernels are bivariate functions that compute inner products between vectors, allowing for efficient calculations in high-dimensional feature spaces. They provide a more abstract and effective way to introduce non-linearities in support vector classifiers.

## Summary & Key Takeaways

• Soft margin may not be effective in situations where data cannot be separated linearly.

• Feature expansion can be used to transform variables and create a higher-dimensional space for improved separation.

• Projecting the enlarged space back to the original space results in a non-linear decision boundary.

• Kernels are functions that compute inner products in high-dimensional feature spaces, allowing for efficient fitting of support vector machines.