Linear Algebra for Machine Learning
Contents
Linear Algebra for Machine Learning¶
“Why should we study mathematics?” - This is the single most common question I get. If you are reading this math textbook, you have probably asked this yourself.
There are two strong reasons. First, mathematics teaches you how to think. Contrary to common belief, mathematics is not only about the objects you study: sets, functions, matrices, and whatnot. Mathematics is equally about learning and understanding thinking via engaging in problem solving.
Second, mathematics is the lingua franca of science and engineering. The ability to formulate quantitative models in any discipline has become the driver of technological innovation for centuries, and we have mathematics to thank for it.
Linear algebra plays a special role in all of this. If mathematics is the lingua franca of science, then linear algebra is the lingua franca of mathematics.
The reason is simple: vectors and matrices are the principal methods for storing and manipulating data. Images, text, music, and scientific measurements are all translated to vectors, and their transformations are expressed in terms of matrices. (Among other building blocks, such as nonlinear functions.)
This book is about the structure of matrices, vectors, and the vector spaces around them. By the end, you’ll understand
how data is stored and manipulated,
what is the underlying geometric structure of the ambient spaces of data,
how matrices represent transformations between vector spaces,
and how all of these are represented inside a computer.
Studying mathematics is a long-term investment with a steep learning curve. Even though I put an extra effort into making the technical details as accessible as possible, don’t expect to breeze through it in a single afternoon. Rather, it’s best to go slow and use it to augment your day-to-day machine learning practice, even revisiting sections multiple times. Learning is a spiral, and each time you come back, you’ll gain a higher perspective on the subject.
(The current trends in online education dictate course creators and textbook writers to quickly create a superficial and easily digestible material, then rely on marketing to bring in sales while moving on to the next project. This book goes against these trends.)
With that, I wish you a great time with this book! Linear algebra is both useful and beautiful. Your investment will return multiple times throughout your career.
Tivadar
How to read this book¶
Mathematics follows a definition-theorem-proof structure that might be difficult to follow at first. If you are unfamiliar with such a flow, don’t worry. I’ll give a gentle introduction right now.
In essence, mathematics is the study of abstract objects (such as functions) through their fundamental properties. Instead of empirical observations, mathematics is based on logic, making it universal. A correct mathematical result is set in stone, remaining valid forever. (Or, until the axioms of logic change.) If we want to use the powerful tool of logic, the mathematical objects need to be precisely defined. Definitions are presented in boxes like this below.
Definition 1 (An example definition.)
This is how definitions are presented in this book.
Given a definition, results are formulated as if A, then B statements, where A is the premise, and B is the conclusion. Such results are called theorems. For instance, if a function is differentiable, then it is also continuous. If a function is convex, then it has global minima. If we have a function, then we can approximate it with arbitrary precision using a single-layer neural network. You get the pattern. Theorems are the core of mathematics.
We must provide a sound logical argument to accept the validity of a proposition, one that deduces the conclusion from the premise. This is called a proof, responsible for the steep learning curve of mathematics. Contrary to other scientific disciplines, proofs in mathematics are indisputable statements, set in stone forever. On a practical note, look out for these boxes.
Theorem 1 (An example theorem.)
Let \( x \) be a fancy mathematical object. The following two statements hold.
(a) If \( A \), then \( B \).
(b) If \( C \) and \( D \), then \( E \).
Proof. This is where proofs go. \( \square \)
To enhance the learning experience, I’ll often make good-to-know but not absolutely essential information into remarks.
Remark 1 (An exciting remark.)
Mathematics is awesome. You’ll be a better engineer because of it.
The most effective way of learning is building things and putting theory into practice. In mathematics, this is the only way to learn. What this means to you is need to read through the text carefully. Don’t take anything for granted just because it is written down. Think through every sentence, take apart every argument and calculation. Try to prove theorems by yourself before reading the proofs.