Quantcast

Posts tagged with restricted Boltzmann machines

"The anchovies were nowhere near the sardines and the tuna. That’s because they were near the pizza toppings.

But it was only a problem because this was a three-dimensional grocery store. If it had been a thirty-dimensional grocery store they could have been near the pizza and the sardines.”

Geoff Hinton talks here about a number of different spaces that are not the physical space on which maths intuition develops.

  • The bag of words model of a document takes each word in a text to be a dimension of the document, with like 80,000 possibilities each or however many words there are in English. (The 80,000 possibilities are the underlying corpus.)
  • Latent semantic analysis of the bag-of-words type is how Google now does its search rankings. (PageRank only constitutes something like 30% of SERP ranking anymore, because [a] it’s too easy to game and [b] it’s inspecific to what’s being searched on. Domain authority and LSI comprise the rest. <—separate article)
  • Seeing a pixel-by-pixel representation of a 2-D image as a list vector is problematic because the first pixel in a 200x300 Facebook profile image is next to the second pixel and also next to the 201st pixel. I.e. one needs a 2-array.
  • Hinton talks about abstract feature space and energies — equivalently evolutionary fitness or economic utility and ravines and mountains upon this manifold.
  • The number of dimensions here is like the number of parameters (same as free parameters or degrees of freedom or arbitrary parameters in stats class) and in a neural net each “synapse” or graph edge is a lever you can pull.
  • The same metaphor — and this is a metaphor in a grand sense which I hope to cover before the year is up — applies to the equalizer on your uncle’s home stereo, i.e. the number of terms in the Fourier decomposition.