Mathematics for Machine Learning with Python- CST284 KTU Minor - Dr. Binu V P -9847390760

Convex optimization problems are a particularly useful class of optimization problems, where we can guarantee global optimality. When $f(.)$ is a convex function, and when the constraints involving $g(.)$ and $h(.)$ are convex sets, this is called a convex optimization problem. In this setting, we have strong duality: The optimal solution of the dual problem is the same as the optimal solution of the primal problem. A set $C$ is a convex set if for any $x, y \in C$ and for any scalar $\theta$ with $0 \le \theta \le 1$, we have  $\theta x + (1 - \theta)y \in C $. Convex sets are sets such that a straight line connecting any two elements of the set lie inside the set. Figures 7.5 and 7.6 illustrate convex and non convex sets, respectively. Convex functions are functions such that a straight line between any two points of the function lie above the function.  Non convex function Convex Function Definition Let function $f : \mathbb{R}^D \to  \mathbb{R}$ be a function whose...

  In the unconstrained optimization we considered the problem of solving for the minimum of a function $min \quad f(x) $ where $f : \mathbb{R}^D \to  \mathbb{R}$ In the constrained optimization we have additional constraints. That is, for real-valued functions $g_i : \mathbb{R}^D \to \mathbb{R}$ for $i = 1,\ldots,m$, we consider the constrained optimization problem  $min\quad f(x)$  subject to $g_i(x) \le 0$ for all  $i = 1,\ldots,m$. In general the functions $f$ and $g_i$ could be non-convex in general. One obvious, but not very practical, way of converting the constrained problem into an unconstrained one is to use an indicator function $J(x)=f(x)+ \sum_{i=1}^{m} 1(g_i(x))$ where $1(z)$ is an infinite step function $1(z)=\left\{\begin{matrix} 0 & if z \le 0\\ \infty & otherwise \end{matrix}\right\}$ This gives infinite penalty if the constraint is not satisfied and hence would provide the same solution.However, this infinite step funct...