Complex Numbers

Complex numbers demystified

Complex numbers are often introduced in an unnecessarily mystic and arcane way using the 1=i\sqrt{-1}=i property. And suddenly multiplication breaks and looks nothing like what we're used to.

This leads to many dismissing complex numbers thinking they are too, well, complex to understand. However, in reality, complex numbers are a beautiful and simple generalization of the real numbers which you're already familiar with, and they can be introduced without using square roots at all! In fact, 1=i\sqrt{-1}=i is just a neat consequence of complex numbers, and has nothing to do with the way they're defined.

But before we can introduce complex numbers, we have to understand another mathematical object: The field.

Fields

The classical definition of a field is a set[1] FF with addition and multiplication defined as you're intuitively used to.

Formally, this means for all aa, bb and cc in FF:

  • Associativity:

    a+(b+c)=(a+b)+ca(bc)=(ab)ca+(b+c)=(a+b)+c \land a\cdot (b\cdot c)=(a\cdot b) \cdot c

  • Commutativity:

    a+b=b+aab=baa+b=b+a \land a\cdot b = b \cdot a

  • Distributivity of multiplication over addition:

    a(b+c)=(ab)+(ac)a\cdot (b+c) = (a \cdot b) + (a\cdot c)

  • Additive and multiplicative identities 0,10,1 both in FF are defined:

    a+0=aa1=aa+0 =a \land a\cdot 1 =a

  • Additive and multiplicative inverses are defined

    aF aF:a+(a)=0\forall a \in F \space \exists -a \in F : a + (-a) = 0

    aF:a̸=0 a1F:aa1=1\forall a \in F : a\neq 0 \space \exists a^{-1} \in F : a \cdot a^{-1} = 1

Let's look at RR the real number line. We can easily see that it's a field because associativity, commutativity, and distributivity all hold which we learned when we first learned the rules of multiplication. Furthermore, we also know that the inverses, and identities exist.

Vector spaces

We can generalize the real numbers to a kk-dimensional vector space with kk tupled numbers.

Here, addition is defined as vector addition, and multiplication is defined as scalar multiplication.

This way, the real numbers become R1\mathbb{R}^{1} which, per our definitions of vector addition and multiplication, collapses to our usual real number field.

The complex numbers would then just two dimensional tuples of numbers, R2\mathbb{R}^{2}, right?

Well, let's look at how many of the field properties complex numbers satisfy then:

Let aR2=[a1,a2]\vec{a} \in \mathbb{R}^2 = [a_1, a_2], bR2=[b1,b2]\vec{b} \in \mathbb{R}^2 = [b_1, b_2], and cR2=[c1,c2]\vec{c} \in \mathbb{R}^2 = [c_1, c_2]:

  • Associativity:

    For addition, we get:

    a+(b+c)=(a+b)+c \vec{a}+(\vec{b}+\vec{c})=(\vec{a}+\vec{b})+\vec{c}

    By expanding this using vector addition, we see that:

    [a1+(b1+c1)a2+(b2+c2)]=[(a1+b1)+c1(a2+b2)+c2] \begin{bmatrix} a_1 + (b_1 + c_1) \\ a_2 + (b_2 + c_2) \end{bmatrix} = \begin{bmatrix} (a_1 + b_1) + c_1 \\ (a_2 + b_2) + c_2 \end{bmatrix}

    Which gives us two equations:

    a1+(b1+c1)=(a1+b1)+c1a2+(b2+c2)=(a2+b2)+c2 \begin{aligned} & a_1 + (b_1 + c_1) = (a_1 + b_1) + c_1 \\ & a_2 + (b_2 + c_2) = (a_2 + b_2) + c_2 \end{aligned}

    Which we know is true from the field properties of R\mathbb{R}.

    Likewise, for multiplication, we get:

    a(bc)=(ab)c \vec{a} \cdot (\vec{b} \cdot \vec{c}) = ( \vec{a} \cdot \vec{b}) \cdot \vec{c}

    Again, by expanding this using vector multiplication, we get:

    [a1(b1c1)a2(b2c2)]=[(a1b1)c1(a2b2)c2] \begin{bmatrix} a_1 \cdot (b_1 \cdot c_1) \\ a_2 \cdot (b_2 \cdot c_2) \end{bmatrix} = \begin{bmatrix} (a_1 \cdot b_1) \cdot c_1 \\ (a_2 \cdot b_2) \cdot c_2 \end{bmatrix}

    Which we also know to be true from the field properties of R\mathbb{R}.

We can show the same thing for commutativity and distributivity of multiplication over addition using the exact same method.

Furthermore, we can find additive and multiplicative identities of R2\mathbb{R}^2; Namely, [0,0][0,0] and [1,1][1,1].

However, we cannot find any element satisfying the conditions of the multiplicative inverses, and since all the requirements must be met in order to have a field, we now know that R2\mathbb{R}^2 is not a field using our definitions for vector addition and multiplication.

Complex numbers are a solution to this problem.

Instead of using scalar multiplication as your multiplicative operator, we instead use the at first somewhat weird looking definition:

ab=[a1b1a2b2a1b2+a2b1] \vec{a} \cdot \vec{b} = \begin{bmatrix} a_1 b_1 - a_2 b_2 \\ a_1 b_2 + a_2 b_1 \end{bmatrix}

The addition operator remains the same.

Using these definitions, we recover the field properties in R2\mathbb{R}^2. Hurray!

One interesting property is that when the complex part, that is a2a_2 in a\vec{a}, is zero, the complex numbers behave just like the real numbers:

For multiplication that is:

ab=[a1b100a10+0b1]=[a1b10]=a1b1 \vec{a} \cdot \vec{b} = \begin{bmatrix} a_1 b_1 - 0 0 \\ a_1 0 + 0 b_1 \end{bmatrix}= \begin{bmatrix} a_1 b_1 \\ 0 \end{bmatrix} = a_1 b_1

And for addition:

a+b=[a1+b10+0]=a1+b1\vec{a} + \vec{b}= \begin{bmatrix} a_1 + b_1 \\ 0 + 0 \end{bmatrix}= a_1 + b_1

However, if the real number part is zero multiplying two complex numbers can result in a real number. In other words, multiplying complex numbers results in a rotation of a two dimensional coordinate system.

For example:

[0,1][0,1]=[0011,01+10]=[1,0] [0,1] \cdot [0,1] = [0\cdot 0 - 1\cdot 1, 0\cdot 1 + 1 \cdot 0] = [-1,0]

If we define [0,1]=i[0,1]=i, we see that:

ii=i2=1 i\cdot i = i^2 = -1

Which means:

1=i\sqrt{-1}=i

Notice that this is a consequence of defining multiplication in a way that preserves the field properties in R2\mathbb{R}^2, and not something mystical crazy mathematicians have pulls out of thin air.

This property also gives us a neat way of writing complex numbers. Since i=[0,1]i = [0,1], we can write the a2a_2 part as a real number multiple of ii. Furthermore, we can write the a1a_1 part as just a real number which gives us the familiar:

a+bi=[a,b]a + bi = [a,b]

This representation also gives us a simple way of finding the multiplicative inverse:

(a+bi)1=1a+bi=aa2+b2iba2+b2 (a+bi)^{-1} = \frac{1}{a+bi} = \frac{a}{a^2+b^2} - i\frac{b}{a^2+b^2}

Complex-valued neural networks

Since the complex numbers are a field, we can use everything we know from linear algebra with complex numbers.

This means that we can create neural networks which work with complex numbers instead of the usual real numbers.

Recent theoretical research has shown that complex neural networks have can encode more rich representations, and are more robust against noise. However, the potential of complex networks remain largely untapped. [2] [3] [4] [5]

In an upcoming essay, we will discuss how to create complex valued neural networks.

Conclusion

We have seen how we can introduce complex numbers as a natural generalization of the real numbers, and how 1=i\sqrt{-1}=i is consequence which arises from defining a multiplication operation which preserves the field properties in R2\mathbb{R}^2.


  1. You can think of a set analogous to a bag in programming; only it can be infinitely large. That is a set is just a collection of numbers. A set doesn't need to be ordered. ↩︎

  2. Deep Complex Networks ↩︎

  3. Learning Representations Using Complex-Valued Nets ↩︎

  4. Complex and Holographic Embeddings of Knowledge Graphs: A Comparison ↩︎

  5. Orthogonality of decision boundaries in complex-valued neural networks. ↩︎