# Calculating the hard-margin SVM by hand

In this blog post, I will show how to calculate the hard-margin SVM by hand. If you are interested in a computational solution, refer to my last post.

## Setting up the optimization problem

There are different ways to write the hard-margin optimization problem. We will be using here the following formulation:

The data for the calculations is $X = \{(3, 4), (4, 1), (6, 6)\}$ with $Y = \{1, 1, -1\}$.

## Calculations

• Replace $\left\lVert \mathbf{w}\right\rVert^2 = w_1^2 + w_2^2$.
• Plug the data $X$ and $Y$ into $y_i(\mathbf{w}^T\mathbf{x}_{i} - b)$
• Simplify
• Change signs
• Setup Lagrangian
• Calculate gradient and set equation $0$
• Simplify equation
• Set $\mu_2 = 0$ and solve the linear system of equations with Gaussian elimination.
• The separating hyperplane and the two supporting hyperplanes are

## Plot

We write the hyperplanes in slope-intercept form in order to plot them.

In the following plot $f(x)$ is the separating hyperplane and $g(x), h(x)$ are the supporting hyperplanes (support vectors).

## Remarks

When solving the system of equations, I assumed $\mu_2^* = 0$, $\mu_1^* > 0$ and $\mu_3^* > 0$. Then according to complementary slackness, the inequality constraints $g_1(\mathbf{w}^*, b^*)$ and $g_3(\mathbf{w}^*, b^*)$ must equal $0$.

After Gaussian elimination, we can check whether the assumptions were correct.

Primal feasibility:

Dual feasibility:

Complementary slackness:

All KKT conditions are satisfied. In practice, you won’t find immediately the optimal solution.

Categories:

Updated: