Vectors and Vector Arithmetic,dot product,vector norm

Vectors are a foundational element of linear algebra. Vectors are used throughout the fi eld of machine learning in the description of algorithms and processes such as the target variable (y) when training an algorithm

A vector is a tuple of one or more values called scalars.Vectors are often represented using a lowercase character such as v; for example:
v = (v1; v2; v3)
Where v1, v2, v3 are scalar values, often real values.
Vectors are also shown using a vertical representation or a column; for example:
v =v1
      v2
      v3
Defining a Vector

We can represent a vector in Python as a NumPy array. A NumPy array can be created from a list of numbers. For example, below we defi ne a vector with the length of 3 and the integer values 1,2,3.

from numpy import array
# define vector
v = array([1, 2, 3])
print(v)

[1 2 3]

Vector Arithmetic

Vector Addition

Two vectors of equal length can be added together to create a new third vector
# vector addition
from numpy import array
# define first vector
a = array([1, 2, 3])
print(a)
# define second vector
b = array([1, 2, 3])
print(b)
# add vectors
c = a + b
print(c)

o/p:
[1 2 3]
[1 2 3]
[2 4 6]

Vector Subtraction
One vector can be subtracted from another vector of equal length to create a new third vector.
# vector subtraction
from numpy import array
# define first vector
a = array([1, 2, 3])
print(a)
# define second vector
b = array([0.5, 0.5, 0.5])
print(b)
# subtract vectors
c = a - b
print(c)

o/p:

[1 2 3]
[ 0.5 0.5 0.5]
[ 0.5 1.5 2.5]

Vector Multiplication
Two vectors of equal length can be multiplied together.
# vector multiplication
from numpy import array
# define first vector
a = array([1, 2, 3])

print("vector a")
print(a)
# define second vector
b = array([1, 2, 3])

print("vector b")
print(b)
# multiply vectors
c = a * b

print("vector c")
print(c)
o/p:

vector a
[1 2 3]

vector b
[1 2 3]

vector c
[1 4 9]
Vector Division
Two vectors of equal length can be divided.
# vector division
from numpy import array
# define first vector
a = array([1, 2, 3])
print(a)
# define second vector
b = array([1, 2, 3])
print(b)
# divide vectors
c = a / b
print(c)
o/p:
[1 2 3]
[1 2 3]
[ 1. 1. 1.]
Vector Dot Product

We can calculate the sum of the multiplied elements of two vectors of the same length to give a

scalar. This is called the dot product, named because of the dot operator used when describing

the operation.

The dot product is the key tool for calculating vector projections, vector decomposition,and determining orthogonality. The name dot product comes from the symbol used to denote it.

# vector dot product
from numpy import array
# define first vector
a = array([1, 2, 3])
print(a)
# define second vector
b = array([1, 2, 3])
print(b)
# multiply vectors
c = a.dot(b)
print(c)
o/p:
[1 2 3]
[1 2 3]
14

Vector outer product

import numpy as np
a=np.array([1,2])
b=np.array([3,4,5])
c=np.outer(a,b)
print("vector a")
print(a)
print("vector b")
print(b)
print("vector c - outer product")
print(c)

o/p

vector a 

[1 2] 

vector b 

[3 4 5] 

vector c - outer product 

[[ 3 4 5] 

[ 6 8 10]] 

Vector Cross Product

import numpy as np

a=np.array([2,3,4])

b=np.array([5,6,7])

c=np.cross(a,b)

print("vector a")

print(a)

print("vector b")

print(b)

print("vector c - cross product -vector product")

print(c)

o/p

vector a 

[2 3 4] 

vector b 

[5 6 7] 

vector c - cross product -vector product

 [-3 6 -3]

Vector-Scalar Multiplication

A vector can be multiplied by a scalar, in e ect scaling the magnitude of the vector. To keep notation simple, we will use lowercase s to represent the scalar value.

c = s x v
# vector-scalar multiplication
from numpy import array
# define vector
a = array([1, 2, 3])
print(a)
# define scalar
s = 0.5
print(s)
# multiplication
c = s * a
print(c)
o/p:
[1 2 3]
0.5
[ 0.5 1. 1.5]
Vector Norms

Calculating the length or magnitude of vectors is often required either directly as a regularization method in machine learning, or as part of broader vector or matrix operations.The length of the vector is referred to as the vector norm or the vector's magnitude.The length of a vector is a nonnegative number that describes the extent of the vector in space, and is sometimes referred to as the vector's magnitude or the norm.The length of the vector is always a positive number, except for a vector of all zero values.

It is calculated using some measure that summarizes the distance of the vector from the origin of the vector space. For example, the origin of a vector space for a vector with 3 elements is (0; 0; 0).

Vector L1 Norm

The length of a vector can be calculated using the L1 norm, where the 1 is a superscript of the L. The notation for the L1 norm of a vector is ||v||1, where 1 is a subscript. As such, this length is sometimes called the taxicab norm or the Manhattan norm.

L1(v) = ||v||1 The L1 norm is calculated as the sum of the absolute vector values, where the absolute value of a scalar uses the notation |a1|. In effect, the norm is a calculation of the Manhattan distance

from the origin of the vector space.

||v||1 = |a1| + |a2| + |a3|

The L1 norm of a vector can be calculated in NumPy using the norm() function with a parameter to specify the norm order, in this case 1.
# vector L1 norm
from numpy import array
from numpy.linalg import norm
# define vector
a = array([1, 2, 3])
print(a)
# calculate norm
l1 = norm(a, 1)
print(l1)
o/p:
[1 2 3]
6.0
The L1 norm is often used when fitting machine learning algorithms as a regularization method, e.g. a method to keep the coefficients of the model small, and in turn, the model less complex.
Vector L2 Norm
The length of a vector can be calculated using the L2 norm, where the 2 is a superscript of the . The notation for the L2 norm of a vector is ||v||2 where 2 is a subscript.
L2(v) = ||v||2

The L2 norm calculates the distance of the vector coordinate from the origin of the vector space. As such, it is also known as the Euclidean norm as it is calculated as the Euclidean distance from the origin. The result is a positive distance value. The L2 norm is calculated as the square root of the sum of the squared vector values.

||v||2 = sqrt(a1^2+ a2^2+a3^2)

The L2 norm of a vector can be calculated in NumPy using the norm() function with default parameters.

# vector L2 norm
from numpy import array
from numpy.linalg import norm
# define vector
a = array([1, 2, 3])
print(a)
# calculate norm
l2 = norm(a)
print(l2)
o/p:
[1 2 3]
3.74165738677

Like the L1 norm, the L2 norm is often used when fitting machine learning algorithms as a regularization method, e.g. a method to keep the coefficients of the model small and, in turn, the model less complex. By far, the L2 norm is more commonly used than other vector norms in machine learning.

Vector Max Norm

The length of a vector can be calculated using the maximum norm, also called max norm. Max norm of a vector is referred to as Linf where inf is a superscript and can be represented with the in finity symbol. The notation for max norm is ||v||inf , where inf is a subscript.

Linf (v) = ||v||inf 

The max norm is calculated as returning the maximum value of the vector, hence the name.

||v||inf = max a1; a2; a3 

The max norm of a vector can be calculated in NumPy using the norm() function with the  order parameter set to inf.

# vector max norm
from math import inf
from numpy import array
from numpy.linalg import norm
# define vector
a = array([1, 2, 3])
print(a)
# calculate norm
maxnorm = norm(a, inf)
print(maxnorm)
o/p:
[1 2 3]
3.0
Max norm is also used as a regularization in machine learning, such as on neural network weights, called max norm regularization.