2.7 Orthogonal Compliment

Having defined orthogonality, lets now look at vector spaces that are orthogonal to each other.This play a major role in dimensionality reduction.Consider $D$ dimensional vector space $V$ and an $M$ dimensional subspace $U \subseteq V$. then its orthogonal compliment $U^\perp$ is a $(D-M)$ dimensional subspace of $V$ and contains all vectors in $V$ that are orthogonal  to every vector in $U$.Furthermore, $U \cap U^\perp={0}$ so that any vector $x \in V$ can be uniquely decomposed into

$x=\sum_{m=1}^{M} \lambda_mb_m +  \sum_{j=1}^{D-M} \psi_ib_j^{\perp}$ $\lambda_m,\psi_j \in \mathbb{R}$

where $(b_1,\ldots,b_M)$  is a basis of $U$ and $(b_1^\perp,\ldots,b_{D-M}^\perp)$ is a basis of $U^\perp$.

Therefore, the orthogonal compliment can also be used to describe a plane $U$(two dimensional subspace) in a three dimensional vector space.More specifically, the vector $w$ with $||w||=1$, which is orthogonal to the plane $U$, is the basis vector of $U^\perp$.All vectors that are orthogonal to $w$ must lie in the plane $U$.The vector $w$ is called the normal vector of $U$.

Generally orthogonal compliments can be used to describe hyperplane in n-dimensional vector and affine spaces.

Example:

The row space of a matrix and Null space are orthogonal compliments.The left null space and column space are orthogonal.Their dot product is zero.sin and cos are orthogonal functions. Integral from $-\pi$ to $\pi$ of their product evaluates to 0.

Python code

from sympy import *

M = Matrix([[1, 2,3], [2, 1, 3]])

#print("Matrix : {} ".format(M))

display(M)

# Use sympy.nullspace() method

M_nullspace = M.nullspace()

print("Nullspace of a matrix : ")

display(M_nullspace)

print("Dot product of row space with null space")

print(M.row(0).dot(M_nullspace))

print(M.row(1).dot(M_nullspace))

O/P

$\displaystyle \left[\begin{matrix}1 & 2 & 3\\2 & 1 & 3\end{matrix}\right]$

Nullspace of a matrix :

[Matrix([ [-1], [-1], [ 1]])]
Dot product of row space with null space

0 

0