SVM: Maximum Margin with Noise in Machine Learning

Linear SVM Formulation

 
Limitations of previous SVM formulation

• What if the data is not linearly separable?
• Or noisy data points?

Extend the definition of maximum margin to allow no-separating planes.

Objective to be minimized

- Minimize
   w.w
   + C (distance of error points to their correct zones)
- Add slack variable ੬i

Maximum Margin with Noise

Lagrangian

 𝛼i's and 𝛽i's are Lagrange multipliers (≥ 0).

Dual Formulation

Find 𝛼1,𝛼2,.....,𝛼m   s.t

Solution to Soft Margin Classification

• xi with non-zero 𝛼i will be support vectors.
• Solution to the dual problem is:

(no need to compute w explicitly)