The zoo of multivariable functions

Tivadar Danka small portrait Tivadar Danka
The zoo of multivariable functions

Understanding math will make you a better engineer.

So, I am writing the best and most comprehensive book about it.

In calculus, going from a single variable to millions of variables is hard.

Understanding the three main types of functions helps make sense of multivariable calculus. They all share a deep connection. Let's see why!

In general, a function assigns elements of one set to another.

The general definition of a function

This is too abstract for most engineering applications. Let's zoom in a little!

As our measurements are often real numbers, we prefer functions that operate on real vectors or scalars. There are three categories:

  1. vector-scalar,
  2. vector-vector,
  3. and scalar-vector.

When speaking about multivariable calculus, vector-scalar functions come to mind first.

Vector-scalar functions

Instead of a graph (like their single-variable counterparts), they define surfaces. You can think about a vector-scalar function as a topographic map.

Topographic map. Source: (Image source: Wikipedia,

Although often not denoted, the argument of a vector-scalar function is always a vector. Most frequently, we write out the components - a.k.a. the variables - explicitly.

Vector-scalar function notation

Want a practical example of a vector-scalar function? The loss of a predictive model maps the vector of parameters to a single scalar.

Below, you can see the mean-squared error of a simple linear regression model.

Mean squared error of linear regression

Next up, we have the vector-vector functions.

Vector-vector functions

You can imagine them as a force field, putting a vector to each point.

The most important example of vector-vector functions is the gradient.

The gradient as a vector-vector function

We call this a gradient field. Let's visualize an example! This is how the vector field given by the gradient of f(x,y)=x2+y2 f(x, y) = x^2 + y^2 looks.

Gradient field example

It is important to note that not all vector-vector functions are gradient fields! For instance, f(x,y)=(xxy,xyy) f(x, y) = (x - xy, xy - y) cannot be a gradient.

A vector field that is not a gradient field

Can you figure out the reason why? (Hint: take a look at the partial derivatives of f(x,y) f(x, y) .)

Next up, we have scalar-vector functions, that is, curves.

Scalar-vector functions

Think about the scalar-vector function f(t) f(t) as the trajectory of a particle at time t t .

Technically, there is only a single variable involved. Yet, curves play an essential role in multivariable calculus.

Remember how vector-vector functions define force fields? Scalar-vector functions describe the trajectories of particles moving through them.

Scalar-vector functions as solutions of a differential equation

Gradient descent connects all of this. In essence, gradient descent

  1. takes the surface of the loss function,
  2. computes the vector field given by the gradient,
  3. and finds the trajectories given by the gradient vector field by a discrete approximation.

This is just the tip of the iceberg. Multivariable calculus is one of the most powerful tools in machine learning, helping us to optimize functions in millions of variables.

That is quite a feat.

Having a deep understanding of math will make you a better engineer.

I want to help you with this, so I am writing a comprehensive book that takes you from high school math to the advanced stuff.
Join me on this journey and let's do this together!