Vector and matrix mathematics
In the previous chapter, you practiced introductory operations with vectors and matrices. In this section, you will practice more advanced vector and matrix operations that are heavily used in linear algebra. Let's remember the dot product perspective on matrix manipulation and how it can be done with different methods when you have 2-D arrays. The following code block shows alternative ways of performing dot product calculation:
In [1]: import numpy as np
a = np.arange(12).reshape(3,2)
b = np.arange(15).reshape(2,5)
print(a)
print(b)
Out[1]:
[[ 0 1]
[ 2 3]
[ 4 5]]
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
In [2]: np.dot(a,b)
Out[2]: array([[ 5, 6, 7, 8, 9],
[15, 20, 25, 30, 35],
[25, 34, 43, 52, 61]])
In [3]: np.matmul(a,b)
Out[3]: array([[ 5, 6, 7, 8, 9],
[15, 20, 25, 30, 35],
[25, 34, 43, 52, 61]])
In [4]: a@b
Out[4]: array([[ 5, 6, 7, 8, 9],
[15, 20, 25, 30, 35],
[25, 34, 43, 52, 61]])
Inner products and dot products are very important in ML algorithms such as supervised learning. Let's get back to our example about tumor detection. Imagine we have three images (MRIs): the first with a tumor (A), the second without a tumor (B) and the third one an unknown MRI that you want to label as with tumor or without tumor. The following graph shows a geometric representation of a dot product for vector a and b:
As a very simple example, the dot product will show you the similarity between these two vectors, so if your unknown MRI's direction is close to vector A, then your algorithm will classify as MRI with tumor. Otherwise, it will classify as without tumor. If you want to multiply two or more arrays in a single function, linalg.multi_dot() is very convenient. Another way to do this operation is by multiplying the arrays nested by np.dot() but you need to know which computation order will be fastest because in linalg.multi_dot() this optimization occurs automatically. The following code block does the same dot product operations, with different methods, as we mentioned previously:
In [5]: from numpy.linalg import multi_dot
a = np.arange(12).reshape(4,3)
b = np.arange(15).reshape(3,5)
c = np.arange(25).reshape(5,5)
multi_dot([a, b, c])
Out[5]: array([[ 1700, 1855, 2010, 2165, 2320],
[ 5300, 5770, 6240, 6710, 7180],
[ 8900, 9685, 10470, 11255, 12040],
[12500, 13600, 14700, 15800, 16900]])
In [6]: a.dot(b).dot(c)
Out[6]: array([[ 1700, 1855, 2010, 2165, 2320],
[ 5300, 5770, 6240, 6710, 7180],
[ 8900, 9685, 10470, 11255, 12040],
[12500, 13600, 14700, 15800, 16900]])
As you see in the following code, the multi_dot() method decreases running time by 40% even with three small matrices. This time gap can increase tremendously if the matrix amount and size increase. Consequently, you should use the multi_dot() method in order to be sure of the fastest evaluation order. The following code will compare the execution time of these two methods:
In [7]: import numpy as np
from numpy.linalg import multi_dot
import time
a = np.arange(120000).reshape(400,300)
b = np.arange(150000).reshape(300,500)
c = np.arange(200000).reshape(500,400)
start = time.time()
multi_dot([a,b,c])
ft = time.time()-start
print ('Multi_dot tooks', time.time()-start,'seconds.')
start_ft = time.time()
a.dot(b).dot(c)
print ('Chain dot tooks', time.time()-start_ft,'seconds.')
Out[7]:
Multi_dot tooks 0.14687418937683105 seconds.
Chain dot tooks 0.1572890281677246 seconds.
There are two more important methods in NumPy's linear algebra library, namely outer() and inner(). The outer method computes the outer product of two vectors. The inner method behaves differently depending on the arguments it takes. If you have two vectors as arguments, it produces an ordinary dot product, but when you have a higher dimensional array, it returns the sum product over the last axes as similarly in tensordot(). You will see tensordot() later in this chapter. Now, let's focus on the outer() and inner() methods first. The following example will help you to understand what these functions do:
In [8]: a = np.arange(9).reshape(3,3)
b = np.arange(3)
print(a)
print(b)
Out[8]:
[[0 1 2]
[3 4 5]
[6 7 8]]
[0 1 2]
In [9]: np.inner(a,b)
Out[9]: array([ 5, 14, 23])
In [10]: np.outer(a,b)
Out[10]: array([[ 0, 0, 0],
[ 0, 1, 2],
[ 0, 2, 4],
[ 0, 3, 6],
[ 0, 4, 8],
[ 0, 5, 10],
[ 0, 6, 12],
[ 0, 7, 14],
[ 0, 8, 16]])
In the preceding example for the inner() method, the ith row of the array produces a scalar product with the vector and the sum becomes the ith element of the output array, so the output array is constructed as follows:
[0x0+1x1+2x2, 0x3+1x4 +2x5, 0x6 +1x7+2x8] = [5, 14, 23]
In the following code block, we perform the outer() method for the same array but with one-dimension. As you noticed the result is exactly the same as we do with 2-D array:
In [11]: a = np.arange(9)
np.ndim(a)
Out[11]: 1
In [12]: np.outer(a,b)
Out[12]: array([[ 0, 0, 0],
[ 0, 1, 2],
[ 0, 2, 4],
[ 0, 3, 6],
[ 0, 4, 8],
[ 0, 5, 10],
[ 0, 6, 12],
[ 0, 7, 14],
[ 0, 8, 16]])
The outer() method computes the outer product of vectors, in our example, a 2-D array. That does not change the functionality of the method but flattens the 2-D array to a vector and does the computation.
Before moving to decomposition, the last thing this subsection covers is thetensordot() method. In a data science project, we mostly work with n-dimensional data which you need to do discovery and apply ML algorithms. Previously, you learned about vectors and matrices. A tensor is a generic mathematical object of vectors and matrices that can keep the relationships of vectors in high-dimensional spaces. The tensordot() method is used for the contraction of two tensors; in other words, it reduces the dimensionality (tensor order) by summing the products of two tensors over the specified axes. The following code block shows an example of tensordot() operation for two arrays:
In [13]: a = np.arange(12).reshape(2,3,2)
b = np.arange(48).reshape(3,2,8)
c = np.tensordot(a,b, axes =([1,0],[0,1]))
print(a)
print(b)
Out[13]:
[[[ 0 1]
[ 2 3]
[ 4 5]]
[[ 6 7]
[ 8 9]
[10 11]]]
[[[ 0 1 2 3 4 5 6 7]
[ 8 9 10 11 12 13 14 15]]
[[16 17 18 19 20 21 22 23]
[24 25 26 27 28 29 30 31]]
[[32 33 34 35 36 37 38 39]
[40 41 42 43 44 45 46 47]]]
In [14]: c
Out[14]: array([[ 800, 830, 860, 890, 920, 950, 980, 1010],
[ 920, 956, 992, 1028, 1064, 1100, 1136, 1172]])