Posts tagged with determinant

## How do I Create the Identity Matrix in R? Also a bit of group theory.

I googled for this once upon a time and nothing came up. Hopefully this saves someone ten minutes of digging about in the documentation.

You make identity matrices with the keyword diag, and the number of dimensions in parentheses.

> diag(3)
[,1] [,2] [,3]
[1,]    1 0 0
[2,]    0 1 0
[3,]    0 0 1 

That’s it.

> diag(11)
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11]
[1,]    1 0 0 0 0 0 0 0 0 0 0
[2,]    0 1 0 0 0 0 0 0 0 0 0
[3,]    0 0 1 0 0 0 0 0 0 0 0
[4,]    0 0 0 1 0 0 0 0 0 0 0
[5,]    0 0 0 0 1 0 0 0 0 0 0
[6,]    0 0 0 0 0 1 0 0 0 0 0
[7,]    0 0 0 0 0 0 1 0 0 0 0
[8,]    0 0 0 0 0 0 0 1 0 0 0
[9,]    0 0 0 0 0 0 0 0 1 0 0
[10,]    0 0 0 0 0 0 0 0 0 1 0
[11,]    0 0 0 0 0 0 0 0 0 0 1 

But while I have your attention, let’s do a couple mathematically interesting things with identity matrices.

First of all you may have heard of Tikhonov regularisation, or ridge regression. That’s a form of penalty to rule out overly complex statistical models. @benoithamelin explains on @johndcook’s blog that

• Tikhonov regularisation is also a way of puffing air on a singular matrix det|M|=0 so as to make the matrix invertible without altering the eigenvalues too much.

Now how about a connection to group theory?

First take a 7-dimensional identity matrix, then rotate one of the rows off the top to the bottom row.

> diag(7)[ c(2:7,1), ]
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,]    0 1 0 0 0 0 0
[2,]    0 0 1 0 0 0 0
[3,]    0 0 0 1 0 0 0
[4,]    0 0 0 0 1 0 0
[5,]    0 0 0 0 0 1 0
[6,]    0 0 0 0 0 0 1
[7,]    1 0 0 0 0 0 0 

Inside the brackets it’s [row,column]. So the concatenated c(2,3,4,5,6,7,1) become the new row numbers.

Let’s call this matrix M.7 (a valid name in R) and look at the multiples of it. Matrix multiplication in R is the %*% symbol, not the * symbol. (* does entry-by-entry multiplication, which is good for convolution but not for this.)

Look what happens when you multiply M.7 by itself: it starts to cascade.

> M.7   %*%   M.7
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,]    0 0 1 0 0 0 0
[2,]    0 0 0 1 0 0 0
[3,]    0 0 0 0 1 0 0
[4,]    0 0 0 0 0 1 0
[5,]    0 0 0 0 0 0 1
[6,]    1 0 0 0 0 0 0
[7,]    0 1 0 0 0 0 0
> M.7   %*%   M.7   %*%   M.7
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,]    0 0 0 1 0 0 0
[2,]    0 0 0 0 1 0 0
[3,]    0 0 0 0 0 1 0
[4,]    0 0 0 0 0 0 1
[5,]    1 0 0 0 0 0 0
[6,]    0 1 0 0 0 0 0
[7,]    0 0 1 0 0 0 0 

If I wanted to do straight-up matrix powers rather than typing M %*% M %*% M %*% M %*% ... %*% M 131 times, I would need to require(expm) package and then the %^% operator for the power.

Here are some more powers of M.7:

> M.7   %^%   4
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,]    0 0 0 0 1 0 0
[2,]    0 0 0 0 0 1 0
[3,]    0 0 0 0 0 0 1
[4,]    1    0    0    0    0    0    0
[5,]    0    1    0    0    0    0    0
[6,]    0    0    1    0    0    0    0
[7,]    0    0    0    1    0    0    0
> M.7   %^%   5
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,]    0 0 0 0 0 1 0
[2,]    0 0 0 0 0 0 1
[3,]    1    0    0    0    0    0    0
[4,]    0    1    0    0    0    0    0
[5,]    0    0    1    0    0    0    0
[6,]    0    0    0    1    0    0    0
[7,]    0    0    0    0    1    0    0
> M.7   %^%   6
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,]    0 0 0 0 0 0 1
[2,]    1    0    0    0    0    0    0
[3,]    0    1    0    0    0    0    0
[4,]    0    0    1    0    0    0    0
[5,]    0    0    0    1    0    0    0
[6,]    0    0    0    0    1    0    0
[7,]    0    0    0    0    0    1    0
> M.7   %^%   7
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,]    1    0    0    0    0    0    0
[2,]    0    1    0    0    0    0    0
[3,]    0    0    1    0    0    0    0
[4,]    0    0    0    1    0    0    0
[5,]    0    0    0    0    1    0    0
[6,]    0    0    0    0    0    1    0
[7,]    0    0    0    0    0    0    1


Look at the last one! It’s the identity matrix! Back to square one!

Or should I say square zero. If you multiplied again you would go through the cycle again. Likewise if you multiplied intermediate matrices from midway through, you would still travel around within the cycle. It would be exponent rules thing^x × thing^y = thing^[x+y] modulo 7.

What you’ve just discovered is the cyclic group P₇ (also sometimes called Z₇). The pair M.7, %*% is one way of presenting the only consistent multiplication table for 7 things. Another way of presenting the group is with the pair {0,1,2,3,4,5,6}, + mod 7 (that’s where it gets the name Z₇, because ℤ=the integers. A third way of presenting the cyclic 7-group, which we can also do in R:

> w <- complex( modulus=1, argument=2*pi/7 )
> w
[1] 0.6234898+0.7818315i
> w^2
[1] −0.2225209+0.9749279i
> w^3
[1] −0.9009689+0.4338837i
> w^4
[1] −0.9009689−0.4338837i
> w^5
[1] −0.2225209−0.9749279i
> w^6
[1] 0.6234898−0.7818315i
> w^7
[1] 1−0i

Whoa! All of a sudden at the 7th step we’re back to “1" again. (A different one, but "the unit element" nonetheless.)

So three different number systems

• counting numbers;
• matrix-blocks; and
• a ring of imaginary numbers

— are all demonstrating the same underlying logic.

Although each is merely an idea with only a spiritual existence, these are the kinds of “logical atoms” that build up the theories we use to describe the actual world scientifically. (Counting = money, or demography, or forestry; matrix = classical mechanics, or video game visuals; imaginary numbers = electrical engineering, or quantum mechanics.)

Three different number systems but they’re all essentially the same thing, which is this idea of a “cycle-of-7”. The cycle-of-7, when combined with other simple groups (also in matrix format), might model a biological system like a metabolic pathway.

Philosophically, P₇ is interesting because numbers—these existential things that seem to be around whether we think about them or not—have naturally formed into this “circular” shape. When a concept comes out of mathematics it feels more authoritative, a deep fact about the logical structure of the universe, perhaps closer to the root of all the mysteries.

In the real world I’d expect various other processes to hook into P₇—like a noise matrix, or some other groups. Other fundamental units should combine with it; I’d expect to see P₇ instantiated by itself rarely.

Mathematically, P₇ is interesting because three totally different number systems (imaginary, counting, square-matrix) are shown to have one “root cause” which is the group concept.

John Rhodes got famous for arguing that everything, but EVERYTHING, is built up from a logical structure made from SNAGs, of which P₇=C₇=Z₇ is one. viz, algebraic engineering

Or, in the words of Olaf Sporns:

[S]imple elements organize into dynamic patterns … Very different systems can generate strikingly similar patterns—for example, the motions of particles in a fluid or gas and the coordinated movements of bacterial colonies, swarms of fish, flocks of birds, or crowds of commuters returning home from work. … While looking for ways to compute voltage and current flow in electrical networks, the physicist Gustav Kirchhoff represented these networks as graphs…. [His] contemporary, Arthur Cayley, applied graph theoretical concepts to … enumerating chemical isomers….

Graphs, then, can be converted into adjacency matrices by putting a 0 where there is no connection between a and b in the [row=a, column=b], or putting a (±)1 where there is a (directed) link between the two nodes. The sparse [0's, 1's] matrix M.7 above is a transition matrix of the cyclical C₇ picture: 1 → 2 → 3 → 4 → 5 …. A noun (C₇) converted into a verb (%*% M.7).

In short, groups are one of those things that make people think: Hey, man, maybe EVERYTHING is a matrix. I’m going to go meditate on that.

## Angle = Volume

This is trippy, and profound.

The determinant — which tells you the change in size after a matrix transformation 𝓜 — is just an Instance of the Alternating Multilinear Map.

(Alternating meaning it goes + − + − + − + − ……. Multilinear meaning linear in every term, ceteris paribus:

$\large \dpi{200} \bg_white \begin{matrix} a \; f(\cdots \blacksquare \cdots) + b \; f( \cdots \blacksquare \cdots) \\ = \\ f( \cdots a \ \blacksquare + b \ \blacksquare \cdots) \end{matrix} \\ \\ \qquad \footnotesize{\bullet f \text{ is the multilinear mapping}} \\ \qquad \bullet a, b \in \text{the underlying number corpus } \mathbb{K} \\ \qquad \bullet \text{above holds for any term } \blacksquare \text{ (if done one-at-a-time)}$)



Now we tripThe inner product — which tells you the “angle” between 2 things, in a super abstract sense — is also an instantiation of the Alternating Multilinear Map.

In conclusion, mathematics proves that Size is the same kind of thing as Angle

Say whaaaaaat? I’m going to go get high now and watch Koyaanaasqatsi.

## Determinant

A matrix ℳ represents a sequence of + and × operations. At the end you’ve linearly transformed a space (sheared it, expanded it, rotated it — but kept the origin where it is.)

Did the amount of stuff in the picture change when you did that? If you kept everything in proportion then det |ℳ| = 1. If not, then det |ℳ| ≠ 1.

If the amount of stuff increased by 10% then det |ℳ|=1.1. If you effectively shrank the picture in half, then det |ℳ|=.5. And so on.

The determinant |ℳ| is the change in volume after the linear transformation.

This metaphor extends to 3-D and beyond.

• If water is flowing linearly in a stream, then |ℳ| needs to be 1, or else water (matter) would be being created.
• If money is flowing linearly in a billion-dimensional economic system, then |ℳ| is hopefully just a little bit above 1, if value is being created. (Central banks need to print |ℳ| times more money to prevent deflation.)
• And a hundred-dimensional linear dynamical system's phase space grows by |ℳ| at every step.