Quantcast

Posts tagged with extrapolation

149 Plays • Download

One of the more consequential kinds of extrapolation happens in the law.

In the case of Islamic law شريعة  the Hadith and the Qu’ran contain some examples of what’s right and wrong, but obviously don’t cover every case.

This leaves it up to jurist philosophers to figure out what’s G-d’s underlying message, from a sparse sample of data. If this sounds to you like Nyquist-Shannon sampling, you and I are on the same wavelength! (ha, ha)

 image
File:CriticalFrequencyAliasing.svg

File:AliasedSpectrum.png
image

image

Of course the geometry of all moral quandaries is much more interesting than a regular lattice like the idealised sampling theorem.

image
image
Lattice of beverages
Revised lattice



image
imagering lattice
ring lattice

image

Escher's grid







image



image

image

Evenly-spaced samples mapping from a straight line to scalars could be figured out by these two famous geniuses, but the effort of interpreting the law has taken armies of (good to) great minds over centuries.

image
image
A_2 lattice Voronoi and Delaunay cells
PCA of British MPs in the space of rollcalls

 

The example from this episode of In Our Time is the prohibition on grape wine:

  • What about date wine?
  • What about other grape products?
  • What about other alcoholic beverages?
  • What about coffee?
  • What about intoxicants that are not in liquid form?

The jurists face the big p, small N problem—many features to explain, less data than desirable to draw on.

image

splines_2d_linear
splines_2d_cardinal
splines_2d_catmullrom
splines_1d_step

splines_1d_cosineimage

Clearly the reason why cases A, B, or D are argued to connect to the known parameter from the Hadith matters quite a lot. Just like in common-law legal figuring, and just like the basis matters in functional data analysis. (Fits nicely how the “basis for your reasoning” and “basis of a function space” coincide in the same word!) .

Think about just two famous functional bases:

  1. Polynomials (think Taylor series),
    image
    image

    image
    image
    image
    image
    image
    image
    image

    and
  2. Sinusoidals (think Fourier series).
    Fourier Series
    image

Even polynomials look like a ‿or   ͡ ; odd polynomials look at wide range like a \ or / (you know how looks: a small kink in the centre ՜𝀱 but in broad distances like /), and sinusoidal functions look like ∿〜〜〜〜〜〜∿∿∿∿〰〰〰〰〰〰〰〰〰〰〰𝀨𝀨𝀨𝀨𝀨.

image

So imagine I have observations for a few nearby points—say three near the origin. Maybe I could fit a /, or a 𝀱, a , or a ‿.

image

All three might fit locally—so we could agree that

  • if grape wine is prohibited
  • and date wine is prohibited
  • and half-grape-half-date-wine is prohibited,
  • then it follows that so should be two-thirds-grape-one-third-date-wine prohibited—
  • but, we mightn’t agree whether rice wine, or beer, or qat, or all grape products, or fermented grape products that aren’t intoxicating, or grape trees, or trees that look like grape trees, and so on.

The basis-function story also matches how a seemingly unrelated datum (or argument) far away in the connected space could impinge on something close to your own concerns.

If I newly interpret some far-away datum and thereby prove that the basis functions are not  but 𝀨𝀱/, then that changes the basis function (changes the method of extrapolation) near where you are as well. Just so a change in hermeneutic reasoning or justification strategy could sweep through changes throughout the connected space of legal or moral quandaries.

image

This has to be one of the oldest uses of logic and consistency—a bunch of people trying to puzzle out what a sacred text means, how its lessons should be applied to new questions, and applying lots of brainpower to “small data”. Of course disputes need to have rules of order and points of view need to be internally consistent, if the situation is a lot of fallible people trying to consensually interpret infallible source data. Yet hermeneutics predates Frege by millennia—so maybe Russell was wrong to say we presently owe our logical debt to him.

image

In the law I could replace the mathematician’s “Let” or “Suppose” or “Consider”, with various legalistic reasons for taking the law at face value. Either it is Scripture and therefore infallible, or it has been agreed by some other process such as parliamentary, and isn’t to be questioned during this phase of the discussion. To me this sounds exactly like the hypothetico-deductive method that’s usually attributed to scientific logic. According to Einstein, the hypothetico-deductive method was Euclid’s “killer app” that opened the door to eventual mathematical and technological progress. If jurisprudence shares this feature and the two are analogous like I am suggesting, that’s another blow against the popular science/religion divide, wherein the former earns all of the logic, technology, and progress, and the latter gets superstition and Dark Ages.

(Source: BBC)




Linear extrapolations are preferred to discontinuous ones, except when the discontinuous extrapolation is correct.
picture via David A Edwards

hi-res




I don’t think so, for the following reasons:

  • Companies are still run imperfectly.
  • Costs are still not as efficiently optimised as possible.
  • Workers don’t produce output at the highest conceivable rate in their current occupation.
  • Plenty of potential engaged in low-productivity work, unemployed, or otherwise not producing (e.g., in school).
  • People do wrong, imperfect things all the time. So there’s room for improvement.

I realise this argument is incomplete. Just because there’s room to grow doesn’t mean we’ll get there. However I think this line of reasoning may prove productive even if my version of it doesn’t get quite there. So let’s press on.

Half a century after Solow, many economists and rest-of-us still think of economic growth as an exogenous “magical” process driven by abstract words such as “technology” or “skill” or “trade” or “innovation”—rather than as the macro sum of correct micro decisions taken by individuals at the company two towns over.

Solow growth model

Some of that surely is to blame on things like Y=C+I+G+NX. Everything “the government” spends is G—regardless of whether it’s spent on a really good idea, implemented well, or on a pie-in-the-sky promise of a half-price incinerator with huge cost overrun. This is like "economists’ K working to constrain our thinking". Or like the Mpemba effect where one first assumes temperature is one-dimensional (false) and then infers that “you have to go through here to get to there”.

 

I prefer to think about a time-varying graph

image.

  • Most of the people (nodes) in the graph I do not trade with directly.
  • The stores I shop at are “popular” nodes. Everyone in my neighbourhood hops 1 to our grocery store. And the grocery store hops 1 or 2 to many many suppliers all over the country (and outside our country).
  • Salaried people have one very-very thick edge connecting to their sole employer.
  • And so on.

With this model it’s less tempting to use abstract words like “technology” without getting more specific. Clearly this new highway will reduce transit times for many of the transaction-edges. And therefore reduce costs broadly, ergo growth. But its effects are (a) localised to those who touch that edge (or an edge that touches that edge) (b) helping some more than others (even hurting a few on net) and [c] rather than being inevitable, come from the good work of individuals, who could have screwed it up. Now it’s tempting to, rather than wish for some magical entrepreneurs or inventors, look at ordinary decisions; not divide the world into “government vs private enterprise”; consider the individuals in particular places who are helped or hurt rather than an average … basically I find this much more grounded and less prone to theoretical histrionics.

Instead of focussing on growth as a 1-D number (Mpemba), trying to correlate it with size of government or regress it against “trust index” or “ethnofract index” or other abstract highfalutins, the GDP number can only be gotten by integrating all the micro elements—which is, I think, as it should be. Then instead of counting on the magical 2% number to stay around 2% year after year, it’s nicely surprising if the total value of transactions this year was again at its record historical high. And when people can squeeze even a higher total of value throughput through the year (out of what? out of natural resources? out of ingenuity? out of Saudi muppets?) you have to wonder where that growth came from—if it was from a sequence of very old “industrial revolutions” or from many companies doing things a little better this year than they did last year. (¬∀, but on net.)

 

Will smart machines make low-wage jobs redundant?

P Krugman asserts in his blog post about Robert Gordon’s paper

machines may soon be ready to perform many tasks that currently require large amounts of human labor. …[I]t’s all too easy to make the case that most Americans will be left behind, because smart machines will end up devaluing the contribution of workers, including highly skilled workers whose skills suddenly become redundant.

So: yes, armies of back-office pencil-pushers have been replaced in the IT revolution by computers. Thank goodness: what a boring, repetitive worklife. But besides record-keeping and verification and copying and automated checking of things, what is it that humans do that’s been replaced by machines? Travel agents? Brick-and-mortar stores with poor selection and high prices? A lot of things computers are good at, like spidering the web, is not something that we previously paid humans to do.

For example Indeed.com, a billion-dollar company created within the last decade, uses electronic computation, networks, and O(100) software engineers to usefully index job search results from disparate sources across the Web. Who lost that job? This is basically a billion-dollar free lunch vis-à-vis unemployment.

Another example that comes to mind is the auto-scan machines at the grocery store. These machines actually make a single clerk more productive. Not that her wages went up necessarily, but the efficiency of the economy did.

So why do peole think automation will replace low-wage jobs? If it’s based on evidence rather than me sitting on my couch and spitballing I’m willing to listen. But from my armchair I see people computing electronic tasks that no human used to do, Siri computing what the iPhone users said, Toyota and NewBalance augmenting humans with machines, machine learning/AI/statistical forecasting making digital things better but again not disemploying anyone.

Despite the appeal of using widespread unemployment as a basis for dystopian fiction, I don’t see anyone scrambling at the profit opportunity to make window-washing droids, janitorial droids, pizza-delivery droids, anything that requires operating in a broad ambiente. The machines seem to be great for repeating the same task in a well-defined scenario, same as factory robots at the Toyota since decades ago.




A Conversation with Peter Thiel and Niall Ferguson

  • 1990’s: The pace of technological change is getting faster! and the pace at which it’s getting faster is getting faster!! Soon we’ll all be cyborgs and live for 1000 years in the post-singularity!!!
  • 2010’s: Things haven’t really gotten much better. Other than the Internet. In fact for poor [Americans] they’ve kind of gotten worse. Well, at least all those people who worked in finance got super rich. And Silicon Valley seems pretty hot … for a few people.
  • Sounds like extrapolating from current market conditions, to me. Although Thiel does cita cifras to make it sound like non-progress has been not-progressing for decades.
 

At minute 12, Thiel says petroleum engineering and chemical engineering have not been good fields to go into. I’m not sure where he gets that idea because those are the two highest-paying college majors (even above computer science) one can currently choose.

Sure, electrical engineers got hired for Quant roles in the past. But I wouldn’t measure value as mean of the top 10% of earners, I would measure it as the trimean. (BTW, how are your quants doing, Peter? Edwin Chen left to be a data scientist at Twitter so…I would guess not that well)

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Minute 17, Ferguson highlights Thiel’s push/pull critique of rocket scientists going to Wall Street: that regulation of rocket science / pharma / nuclear / extraction pushed the engineers to Wall Street, as much as money (financial deregulation?) pulled them there. “Engineers pushed to less productive areas.”

But … how could the returns to the financial sector have been so high if finance weren’t earning supranormal profits? I still buy the deregulation → synthetic assets → people who can synthesise the assets (lawyers and Monte Carlo programmers).

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

A lot of focus on the choices of exceptional individuals here. My bias is to view history as driven by normal people, not geniuses.

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Minute 19, “root-canal Republicanism”. Hasn’t the American Republican party changed too much since the 1930’s to use this term? Compare Teddy Roosevelt (progressive) and Newt Gingrich (cheerful).

past GDP growth corridor

US GDP 1947-2012

Things in Peter Thiel’s history of progress:

  • green revolution (50’s)
  • emergent aerospace industry (30’s)
  • emergent Hollywood industry (30’s)
  • plastics
  • 2 years’ improvement to life expectancy per decade
  • faster transportation
  • cheaper energy

A debate over whether technological slowdown means the US can’t borrow money to finance a Keynesian fiscal expansion (“smoothing”, if the future is up).

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

Minute 25. The superior technological functioning of the NHS as compared to US healthcare (US has cheaper computers and higher computer literacy): is this a question of public vs private provision of care? About deregulation? Maybe it’s not about such a grand narrative from the newspaper op-eds, but rather about boring details of the management of many firms. Thiel’s off-the-cuff idea is to appoint a health czar with an engineering degree. (What?) Or who has run an engineering firm. (How about someone who has run hospitals?)


\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

  • A very nice extemporaneous answer to the luck-vs-skill question.
  • Europe’s lost generation. What about the US’ lost generation or Japan’s lost generation?

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Minute 33. ”The US has had a lot of bubbles. But not a lot of growth. People have this cornucopian view that growth automatically happens.

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

Higher ed bubble

  • It’s impolite to point at the poor college graduates and ask “Where’s the return on human-capital investment?”
  • The high school → Harvard → Wall Street Tournament
  • Kids skipping grades (gee, lots of focus on geniuses)
  • Brilliant teachers teaching the masses. Again, focus on geniuses … and hasn’t this already been tried, like in the 70’s to have master teachers record videos and teach thru broadcast? Anyway, MIT OCW has been around for a decade now and nobody uses it as a substitute for school — at least nobody’s talking about it.
  • Peter, how can you base national education policy on what’s best for the most academically talented .00001% of students?

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

On chess: “Chess is dangerous. I, um, I still play. Probably too much. Well, not enough and too much, at the same time.”

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

Government investments of the "spray ‘n’ pray" variety. The cornucopia of given growth (S&P 500 average = 7%, therefore this company “is expected” to make 7% — at least if we spread our money out we’ll find some J curves!”

Apropos of his answer to the luck-vs-skill question. Thinking about why something is going to work. Thinking hard about what might happen and what might go wrong. (viz., Charlie Munger’s use of decision trees)

////////////////////////////////////////////////////////////////////////////////////////////////////////////

Minute 55. Someone from the audience also claiming lone geniuses push society forward. Einstein as an example. To my knowledge, Einstein didn’t build or design any power plants, didn’t even hold patents. As far as the 20th century’s technological progress, we can thank IBM and the chemists who worked on semiconductors and transistors, as well as all of the “little” engineers who made incremental improvements to battery life, turbine design, shaping cooling towers as a hyperboloid, building oil rigs that don’t break, and — oh yeah, the fatcats and morons who finance and actually make stuff.

Neal Stephenson, writing on the American West Coast in the 90’s, saidthat the wealth of a nation was in its people. (Postwar Japan being his example: from rubble to corporate samurai.)

But George Buckley, CEO of 3M, says that manufacturing, mining, and agriculture are the cornerstones of wealth.

Hmmm.

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

"Most people, at most times and places, have lived in societies that were essentially static." e.g., Thomas Malthus

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Minute 61: "The low-hanging fruit is making the government more efficient." Well, there are certainly government employees wasting their time on emails and talking by the water cooler in my city’s government. But HR costs ($30mm) aren’t a big fraction of the city economy ($6B) — and every bureaucracy has such lazy people. Now maybe he just means making smarter decisions, like investing in the right thing instead of the wrong thing. Well, somehow I don’t think that will be solved by just putting an engineer in the room or throwing technology at the problem.

Then again, maybe he’s just talking about Sarbox.

"The conversation in government is never about how can we do more with less." Wow, what a clueless statement. Everyone wants to spend more money and tax less. That is exactly trying to do more with less. Or how about Harrisburg, PA’s bankruptcy case? They tried to do more with less. (They tried to power the city at a tiny fraction of the cost—and hired a Colorado entrepreneur—with an engineering degree!—to get it done.)

Not to mention: regarding the US Federal government, they most certainly do focus on using technology and reducing wastage. Witness data.gov (I think this was an Obama initiative, although I don’t follow US politics closely enough to be sure). And witness, um, the GAO = Government Accountability Office. Just a brief look at gao.gov shows reports on:

  • streamlining government
  • measuring wastage
  • improving physical and technological infrastructure of government
  • convincing government to actually adopt their recommendations
  • how’s the TARP going? “exposure to AIG lessens”
  • Did departments follow our advice? (“5% of our 176 recommended actions were followed, 74% were partially followed, 21% were not followed”)
  • Because the US Congress members individually have the incentive to waste as little as possible on other constituencies, they’ve given the GAO the power to look into what the executive branch (probably minus the military) does with its money and even go so far as to protest specific bids.
  • http://gao.gov/products/GAO-12-342SP#mt=e-report&st=2 There’s their report on this supposed “low-hanging fruit”. I’ll leave it up to you how low it’s actually hanging.

Not surprising that a libertarian holds naive views on governance. I would be interested to see how his views would change if he served in government for a term.

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Minute 65: On the transition from innovative original founder to mechanistic bureaucracy. Back to a field where Thiel knows what he’s talking about.

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

Related: Out of Blake Masters’ notes from Peter Thiel’s CS183 class at Stanford, Peter Thiel’s story of the 1990’s tech bubble is my favourite. (They’re all really good, but his story of the 90’s tech bubble is brilliant.)




What’s the difference between leaving carbon progeny behind you and silicon progeny behind you? … [W]hat makes you feel that a planet teeming with sexually created successors would constitute a more valid extension of ‘we’-ness than a planet teeming with our intellectually created successors? [robots / cyborgs / conscious machines / strong AI computers]

The question comes down to how we human beings feel comfortable using and extrapolating the term pronoun “we”. Were “we” once languageless squirrel-sized mammals? Did “we” then become primates? Did “we” discover that “we” could use tools? Did “we” begin speaking some 50,000 years ago? Were “we” at that time an entirely agrarian society? Did “we” start living in cities a few thousand years ago? Did “we” discover geometry, algebra, and calculus? Did “we” try out communism for a few decades? Will “we” someday cure cancer? Will “we” someday fly to Mars? … Will “we” migrate into immortal software?

Doug Hofstadter, in Perspectives on Natural and Artificial Evolution

The whole essay (ok, most of it):

grâce à Virgil

 

The story of the primates reminds me of my favourite short story from Cosmicomics. Italo Calvino shrinks the generations of evolution into manageable bites, so that qfwfq, a lizard in this story, has a great-uncle n’ba n’ga who’s still a fish.

Well, you can read it yourself:




extrapolation and interpolation
The most important lesson I learned from this book:  regression is reliable for interpolation, but not for extrapolation.  Even further, your observations really need to cover the whole gamut of causal variables, intersections included, to justify faith in your regressions.
Imagine you have two causal variables, A and B, that are causing X.  Maybe your data cover a wide range of observations of A — some high, some low, some in-between.  And you have, too, the whole gamut of observations of B — high, low, and medium.  It might still be the case that you haven’t observed A and B together (not seen ).  Or that you’ve only observed them together (not seen ).  In either case, your regression is effectively extrapolating to the other causal region and you should not trust it.
Let’s keep the math sexy.  Say you meet an attractive member of your favourite sex.  This person A) likes to hunt, and B) is otherwise vegetarian.  Your prejudices  are that you don’t like hunters () and you do like vegetarians ().  By comparing the magnitudes of these preferences, you deduce that you should not get along with this attractive person, because the bad A part outweighs the good B part.

However, since you haven’t observed both A and B positive at once, your preconceptions are not to be trusted.  Despite your instincts , you go out on a date with Mr or Ms (A>0, B>0) and have a fantastic time.  Turns out there was a positive interaction term in the  range, it also correlates positively with the noise (it wasn’t noise, just unknown knowledge), and you’ve found your soul mate.

extrapolation and interpolation

The most important lesson I learned from this book:  regression is reliable for interpolation, but not for extrapolation.  Even further, your observations really need to cover the whole gamut of causal variables, intersections included, to justify faith in your regressions.

Imagine you have two causal variables, A and B, that are causing X.  Maybe your data cover a wide range of observations of A — some high, some low, some in-between.  And you have, too, the whole gamut of observations of B — high, low, and medium.  It might still be the case that you haven’t observed A and B together (not seen ).  Or that you’ve only observed them together (not seen ).  In either case, your regression is effectively extrapolating to the other causal region and you should not trust it.

Let’s keep the math sexy.  Say you meet an attractive member of your favourite sex.  This person A) likes to hunt, and B) is otherwise vegetarian.  Your prejudices  are that you don’t like hunters () and you do like vegetarians ().  By comparing the magnitudes of these preferences, you deduce that you should not get along with this attractive person, because the bad A part outweighs the good B part.

However, since you haven’t observed both A and B positive at once, your preconceptions are not to be trusted.  Despite your instincts , you go out on a date with Mr or Ms (A>0, B>0) and have a fantastic time.  Turns out there was a positive interaction term in the  range, it also correlates positively with the noise (it wasn’t noise, just unknown knowledge), and you’ve found your soul mate.