_{*to be continued and implemented as module....}

The lagrangian multiplier tells that to optimize such $f(w)$ subjected to some constraint $g(w)<=a$ takes $L(w,\lambda) = f(w)-\lambda(g(w)-a)$

Then the optimization is later continued with the operations $\partial{L}/\partial{w}$ & $\partial{L}/\partial{\lambda}$ that is respecting to both the lagrange multiplier and the weight.

The objective of the function is the optimized-margin ($\gamma_{optimized}$)

, that is maximizing $\gamma = 1/|w|$

It turns out that

• maximizing $1/|w|$ is equal to minimizing $\|w\|$
• and, minimizing $\|w\|$ is equal to minimizing $\|w\|^2$

also introduce the factor $1/2$ to the process so that

• instead of minimizing $\|w\|$ , minimize the $\frac{1}{2}|w|^2$

Why not maximizing 1/||w|| directly? Because maximizing 1/|w| directly is complicated (it will takes several more steps during the computation which in turn, inefficient(that is slower))

Overall, the constrained optimization defined as

$$\text{minimize}_{\mathbf{w}, b} \quad \frac{1}{2} \|\mathbf{w}\|^2$$

subject to

$$t_n (\mathbf{w}^\top \mathbf{x}_n + b) \geq 1, \quad \forall n$$