Posts tagged with mathematics

The rank-nullity theorem in linear algebra says that dimensions either get

  • thrown in the trash
  • or show up

after the mapping.


By “the trash” I mean the origin—that black hole of linear algebra, the /dev/null, the ultimate crisscross paper shredder, the ashpile, the wormhole to void and cancelled oblivion; that country from whose bourn no traveller ever returns.

The way I think about rank-nullity is this. I start out with all my dimensions lined up—separated, independent, not touching each other, not mixing with each other. ||||||||||||| like columns in an Excel table. I can think of the dimensions as separable, countable entities like this whenever it’s possible to rejigger the basis to make the dimensions linearly independent.


I prefer to always think about the linear stuff in its preferably jiggered state and treat how to do that as a separate issue.

So you’ve got your 172 row × 81 column matrix mapping 172→ separate dimensions into →81 dimensions. I’ll also forget about the fact that some of the resultant →81 dimensions might end up as linear combinations of the input dimensions. Just pretend that each input dimension is getting its own linear λ stretch. Now linear just means multiplication.

linear maps as multiplication
linear mappings -- notice they're ALL straight lines through the origin!

Linear stretches λ affect the entire dimension the same. They turn a list like [1 2 3 4 5] into [3 6 9 12 15] (λ=3). It couldn’t be into [10 20 30 − 42856712 50] (λ=10 except not everywhere the same stretch=multiplication).


Also remember – everything has to stay centred on 0. (That’s why you always know there will be a zero subspace.) This is linear, not affine. Things stay in place and basically just stretch (or rotate).

So if my entire 18th input dimension [… −2 −1 0 1 2 3 4 5 …] has to get transformed the same, to [… −2λ −λ 0 λ 2λ 3λ 4λ 5λ …], then linearity has simplified this large thing full of possibility and data, into something so simple I can basically treat it as a stick |.

If that’s the case—if I can’t put dimensions together but just have to λ stretch them or nothing, and if what happens to an element of the dimension happens to everybody in that dimension exactly equal—then of course I can’t stick all the 172→ input dimensions into the →81 dimension output space. 172−81 of them have to go in the trash. (effectively, λ=0 on those inputs)

So then the rank-nullity theorem, at least in the linear context, has turned the huge concept of dimension (try to picture 11-D space again would you mind?) into something as simple as counting to 11 |||||||||||.

Define the derivative to be the thing that makes the fundamental theorem of calculus work.

roots of x²⁶•y + x•z + y¹³•z + x•y¹³ + z²⁶     =   0


(Source: imaginary.org)

So, you never went to university…or you assiduously avoided all maths whilst at university…or you started but were frightened away by the epsilons and deltas…. But you know the calculus is one of the pinnacles of human thought, and it would be nice to know just a bit of what they’re talking about……

Both thorough and brief intro-to-calculus lectures can be found online. I think I can explain differentiation and integration—the two famous operations of calculus—even more briefly.


Let’s talk about sequences of numbers. Sequences that make sense next to each other, like your child’s height at different ages


not just an unrelated assemblage of numbers which happen to be beside each other. If you have handy a sequence of numbers that’s relevant to you, that’s great.


Differentiation and integration are two ways of transforming the sequence to see it differently-but-more-or-less-equivalently.

Consider the sequence 1, 2, 3, 4, 5. If I look at the differences I could rewrite this sequence as [starting point of 1], +1, +1, +1, +1. All I did was look at the difference between each number in the sequence and its neighbour. If I did the same thing to the sequence 1, 4, 9, 16, 25, the differences would be [starting point of 1], +3, +5, +7, +9.


That’s the derivative operation. It’s basically first-differencing, except in real calculus you would have an infinite, continuous thickness of data—as many numbers between 1, 4, and 9 as you want. In R you can use the diff operation on a sequence of related data to automate what I did above. For example do

  • seq <- 1:5
  • diff(seq)
  • seq2 <- seq*seq
    successor function and square function
  • diff(seq2)

A couple of things you may notice:

  • I could have started at a different starting point and talked about a sequence with the same changes, changing from a different initial value. For example 5, 6, 7, 8, 9 does the same +1, +1, +1, +1 but starts at 5.
  • I could second-difference the numbers, differencing the first-differences: +3, +5, +7, +9 (the differences in the sequence of square numbers) gets me ++2, ++2, ++2.
  • I could third-difference the numbers, differencing the second-differences: +++0, +++0.
  • Every time I diff I lose one of the observations. This isn’t a problem in the infinitary version although sometimes even infinitely-thick sequences can only be differentiated a few times, for other reasons.

The other famous tool for looking differently at a sequence is to look at cumulative sums: cumsum in R. This is integration. Looking at “total so far” in the sequence.

Consider again the sequence 1, 2, 3, 4, 5. If I added up the “total so far” at each point I would get 1, 3, 6, 10, 15. This is telling me the same information – just in a different way. The fundamental theorem of calculus says that if I diff( cumsum( 1:5 )) I will get back to +1, +2, +3, +4, +5. You can verify this without a calculator by subtracting neighbours—looking at differences—amongst 1, 3, 6, 10, 15. (Go ahead, try it; I’ll wait.)

Let’s look back at the square sequence 1, 4, 9, 25, 36. If I cumulatively sum I’d have 1, 5, 15, 40, 76. Pick any sequence of numbers that’s relevant to you and do cumsum and diff on it as many times as you like.


Those are the basics.

Why are people so interested in this stuff?

Why is it useful? Why did it make such a splash and why is it considered to be in the canon of human progress? Here are a few reasons:

  • If the difference in a sequence goes from +, +, +, +, … to −, −, −, −, …, then the numbers climbed a hill and started going back down. In other words the sequence reached a maximum. We like to maximize things, like efficiency, profit, 
  • A corresponding statement could be made for valley-bottoms. We like to minimise things like cost, waste, usage of valuable materials, etc.
  • The diff verb takes you from position → velocity → acceleration, so this mathematics relates fundamental stuff in physics.
  • The cumsum verb takes you from acceleration → velocity → position, which allows you to calculate stuff like work. Therefore you can pre-plan for example what would be the energy cost to do something in a large scale that’s too costly to just try it.
  • What’s the difference between income and wealth? Well if you define net income to be what you earn less what you spend,
    then wealth = cumsum(net income) and net income = diff(wealth). Another everyday relationship made absolutely crystal clear.
  • In higher-dimensional or more-abstract versions of the fundamental theorem of calculus, you find out that, sometimes, complicated questions like the sum of forces a paramecium experiences all along a sequential curved path, can be reduced to merely the start and finish (i.e., the complicatedness may be one dimension less than what you thought).
  • Further-abstracted versions also allow you to optimise surfaces (including “surfaces” in phase-space) and therefore build bridges or do rocket-science.
  • With the fluidity that comes with being able to diff and cumsum, you can do statistics on continuous variables like height or angle, rather than just on count variables like number of people satisfying condition X.
    kernel density plot of Oxford boys' heights.
  • At small enough scales, calculus (specifically Taylor’s theorem) tells you that "most" nonlinear functions can be linearised: i.e., approximated by repeated addition of a constant +const+const+const+const+const+.... That’s just about the simplest mathematical operation I can think of. It’s nice to be able to talk at least locally about a complicated phenomenon in such simple terms.
    linear maps as multiplication
  • In the infinitary version, symbolic formulae diff and cumsum to other symbolic formulae. For example diff( x² ) = 2x (look back at the square sequence above if you didn’t notice this the first time). This means instead of having to try (or make your computer try) a lot of stuff to see what’s going to work, you can just-plain-understand something.
  • Also because of the symbolic nicety: post-calculus, if you only know how, e.g., diff( diff( diff( x ))) relates to x – but don’t know a formula for x itself – you’re not totally up a creek. You can use calculus tools to make relationships between varying diff levels of a sequence, just as good as a normal formula – thus expanding the landscape of things you can mathematise and solve.
  • In fact diff( diff( x )) = − x is the source of this, this
    , this,
    , and therefore the physical properties of all materials (hardness, conductivity, density, why is the sky blue, etc) – which derive from chemistry which derives from Schrödinger’s Equation, which is solved by the “harmonic” diff( diff( x )) = − x.

Calculus isn’t “the end” of mathematics. It’s barely even before or after other mathematical stuff you may be familiar with. For example it doesn’t come “after” trigonometry, although the two do relate to each other if you’re familiar with both. You could apply the “differencing” idea to groups, topology, imaginary numbers, or other things. Calculus is just a tool for looking at the same thing in a different way.

The reactions against Grassmann make a humorous chapter in the history of mathematics. For example, Professor Pringsheim, dean of German mathematicians and the author of over one hundred substantial papers on the theory of infinite series, both convergent and divergent, kept insisting that Grassmann should be doing something relevant instead of writing up his maniacal ravings. “Why doesn’t he do something useful, like discovering some new criterion for the convergence of infinite series!” Pringsheim asserted, with all the authority that his position conferred.

Gian-Carlo Rota, Indiscrete Thoughts

(if you don’t get why this insult is funny: Pringsheim’s articles must have been dry and small-minded, whereas Grassmann’s packs a universe
into a kernel)


In many showers there are two taps: Hot and Cold. You use them to control both the pressure and the temperature of the water, but you do so indirectly: the pressure is controlled by the sum of the (position of the) two taps, while the temperature is controlled by their difference. Thus, the basis you are given:

Hot = (1,0)
Cold = (0,1)

Isn’t the basis you want:

Pressure = (1,1)
Temperature = (1,−1)

Alon Amit

(image would be the basis isomorphism)


other bases:

(Source: qr.ae)