see things differently

Posts tagged with programming

Why is it that the standard of proof in software development theory is “Some famous person said it’s cool and I can think of an analogy to a Malcolm Gladwell book so therefore i’m right”?

Standards of proof in medicine—scurvy, limeys
Control groups and test groups.
Yes, all studies are flawed.
My Comment: The difference between nitpicking and valid criticism of a study is this. Any study is going to miss some data points (patchy, piecemeal coverage of the space we’re interested in) and the measurements are going to be wrong ε. ε may even be piped through a function (automorphism) such that the whole study is ruined. (for example you measured the wrong people or you measured the wrong way or you introduced bias or you measured a special proper subset but wrongly generalised the special property to the general case)

We know this going in. The challenge is to distinguish ε’s that are deadly to the conclusion from epsilon;’s that are inconvenient and imperfect but don’t phase-change the conclusion. (think bounded perturbation versus sign change)

This is why it’s nice to be able to make arguments using order-of-magnitude or sign. If you observe multiple orders of magnitude difference in beta; between the control group and the test group, then as long as your sample wasn’t horrifically terrible, the conclusion that A>B is still going to hold up.
Rank speculation versus data.
http://cochrane.org. Look at the data yourself. Do you see a connexion between vaccination and autism?
“It’s almost impossible to get a paper published in a top journal without having actually tried it in the real world.”
“All the work that’s been done to date on [estimating software development time] is pretty much worthless. … The engineers are just going to tell us what they think we want to hear.” (or their answer will be driven by anchoring effects)
Garbage in, garbage out. (He gives an example where some garbage estimates of how long a software project would take are used as the base data, before being thrown through a gleaming shimmering algorithm that makes them look meaningful.)
“If [developers are more accurate estimating how long a project will take on an hours-scale than a months-scale,] then that’s a powerful argument for using agile methods. I’ve heard many people make that argument, but we don’t have any data to back it up.”
You’ve all heard “Great programmers are 28 times more productive than OK programmers. Or 40, 50, 100. I just pick a number that’s big enough to be impressive but not so large that you’re going to doubt me.”
My question: I would love to see a #BigData person scrape the web for all such claims.
“All of the claims you’re reading about the relative productivity of programmers can be traced back to: 12 people in 1968 [batch programming] for an afternoon, or 54 people [in the 1980’s] for an hour. How confident are you in those claims now?”
“The best programmer I ever met — his only higher education was two years at a rabbinical college.”
Claims he could take $150M in research money and make 5% of $1,000B. (A classic appeal to the “No number can be less than 1%” theorem. Textbook VC-pitch logic.)
Klaus _____; “Regardless of language” (scheme, assembly, java, python, …) “programmers produce the same number of lines-of-code per hour.”
Boehm 1975: “Most bugs are introduced in the design & analysis phase.”
“The sooner a bug is caught in the development process, the less it costs to fix” (he seems to indicate the cost growth is exponential with time)
“Adding a feature doubles production time. (Conversely, if the developers can say “no” to a few features, development time goes down convexly.)”
“If you have to rewrite more than 25% of the code, you’re better off starting from scratch.”
Minute 33: “Hour for hour, the fastest way to fix code is to read it. Not to run it. Not to write unit tests. Sitting down and having someone else go through your code. 60%-90% of bugs can be removed before the code is even run.”
“But it turns out that most of the value comes from the first reader in the first hour of looking at the code.” That’s a couple hundred lines.
Conway’s Law is true.
A big-data statistical analysis of how Windows Vista was built, trying to find regressors that predict the fault rate.

Physical co-location of programmers did not matter.
Distance in organisational chart does matter.

Stereotypical anti-religious view of scientific progress. (“The difference between science and religion is…”). Blah. Not very evidence-based in an evidence-based lecture.
“Ask a successful start-up what’s the reason for their success and they’re going to get it wrong.” Up to academics to figure out what makes for success across various cases. (I agree; this is the point of management science. And he mentions the Harvard Business Review as a touchstone.)
Rob Pike: “I don’t think I’ve ever seen beautiful code.”
Typical self-perception of a rich programmer / white collar / professor. Whatev.
Some words about becoming an adult. Someday there will be no higher authority you can appeal to other than the people who were 18 when you were.
Making decisions when nobody knows the necessary information. (Seems out of place in a talk where he acts like his views are all backed-up by data. Whatever, though. I think we can all agree that being right is better than being wrong and that one should change one’s opinion whenever “objective evidence says you’re wrong” (whatever that means). And, obviously, as a leader you usually don’t get as much information as you would like—but you have to decide something anyway.)
The difference between the Bolsheviks and the Trotskyists is that whilst Bolsheiks believe the masses have to take to the streets to effect change, Trotskyists believe a handful of focussed people who get on the right committees can change the world. Examples: the teaching of evolution; abortion. Don’t write a blog (oops) or start a Facebook group to effect political change, go to the pressure point. Put on a suit, try to sound like an adult, and make your case calmly and rationally to the people in charge.

Hat tip to @gnycl.

Gerald Jay Sussman on biology & computation.

The human genome is 1 GB. So is Windows OS.
We have no idea how to program a 10¹²-unit thing like a human, or a cow.
A salamander regrows three elbows if you break off its arm and reattach it — responding to local errors.
We have no idea how to write computer vision in a few steps like neurons do.
Program efficiency doesn’t matter.
Memory is free right now.
Computation is free right now.
For a million bucks you can get a seriously computer—but what to do with it?
Most of the cost of a computer program is paying programmers. —Huw Evans
Yet we spend so much time modifying existing code. —Huw Evans
Correctness doesn’t matter. (Getting a reasonable answer is fine.)
Security doesn’t matter. (Humans are attacked by parasites all the time.)
Biological systems are written to solve problems that the designer didn’t foresee.
This LISPian stuff of writing programs that write programs looks very awesome; what I’ve pretty much always wanted to do with computers. (I tried and failed to use PHP’s multiple dollar signs to helpful effect.)
Some stiff-matrix stuff that you can understand if you watch some Gilbert Strang videos.
The point being: with bindings and such you can write a program that’s somewhat robust: performing operator overloading or similar things so that you can just tell the computer basically what you mean. (Rather than having to be so specific.)
“Mathematical symbols are impressionistic” — just think about how the fnof; symbol is used — yeah, some function, whatever, ya know what I mean.
Systems that accept a wide variety of inputs and only give a small range of outputs.
Your cells have about 1 GB of ROM and a few megabytes of RAM.
“Galileo discovered the value of a lie — to figure it out without the friction and then put it back in.”
Propagators: Independent Stateless machines connecting stateful cells
Satisficing / monotonically increasing local information about a referent.
Synchronising problems in parallel computation go away.
EE, not CS, point of view. (and I daydreamed off into economic theories using this circuit-diagram thinking)
“I hate modern languages, even the ones I invent! … There are no names for intermediate parts [of expressions].”
Min 44. Maybe this is an insight into explaining statistics to programmers. Errorful processes go in; this would be like multiple inputs (maybe a distribution assumption, maybe a bound on the error, maybe an independence assumption or equal variances) — and an errorful result comes out. Just like division-of-integers has two outputs—quotient and remainder—so do statistical processes spit out a ton of outputs as well:—answer(s) like beta;’s, error estimates (p’s and t’s), and tons of different ways of looking at what might be wrong with the assumptions (Durbin-Watson, structural F statistic, White estimators). Of course you could turn these outputs into Boolean by saying like “alpha; <.05 goes to TRUE” — but really the continuous alpha score is better.
Expert reasoning in circuit diagrams as local reasoning between neighbouring pieces of the circuit diagram.
Improving measurements by making independent measurements of the same thing by different methods.
Tracking of provenance & shadow premises—….daydreaming: relationship to religious faith/scepticism.
Giggling when logic subsystems conflict.
Truth Maintenance Systems — the ability to back out
Finding lies — medical statistical results
Globally inconsistent worldviews that are locally consistent.
Aaaand, find me the consistent sub-worldviews that are consistent.

(Source: infoq.com)

Category Theory for Programmers

redraw pictures of ƒ(A)=B so that the morphisms look like • and the objects look like →. Hey, why not?
string diagrams (process networks) are just as good as algebraic symbology
associativity
tensor products
if switch( switch( • )) ≠ • , then^* you’ve got a braided category, which is described by the algebra of knots. “I’m not sure why you would want your programming language to include the algebra of knots at a basic level, but … you could.” I think braided code totally makes sense … in a visualising-a-big-system-of-code kind of way.

^*Technically speaking: in Haskell it’s always true for every • input that switch( switch( • )) ≠ • . If we were talking about a language where switch( switch( • )) ≠ • is not always true — sometimes or possibly not true — then that language would be a braided category. ∃x | ¬ p(x) = ¬∀x p(x) — all symbols spoken in the subjunctive.

(Source: nuclearphynance.com)

Hard Boundaries, Soft Boundaries

Computer scientists use filters, ≥ signs, intersections (sql), and other forms of what I would call “hard boundaries”.

Grep either finds what you’re looking for, or it doesn’t.
The condition inside your while(){ loop either trips true and the interior code runs, or it trips false and it’s skipped.
You either follow someone on twitter, or you don’t.
You either crawled a webpage, or you didn’t.
In exploring a code tree or other graph, you either look at the node, or you don’t.
Two people either are Facebook friends, or they aren’t.
The tweet either included a word from this list, or it didn’t.

But, one needn’t be so conceptually constrained. Thinking in a fuzzy logic sense, it’s possible to create a “soft” boundary.

To use a classic example from Bart Kosko’s book, although the legal system imposes a “hard boundary” on adulthood (OK, a series of hard boundaries—16, 18, 21, 25), one really passes into adulthood gradually over time. (Unless you have your first kid at 16, in which case you grow up real quick. But talking about the upper-middle-class college-enrolled set here: most of them grow up slowly.)

That’s nice in a philosophical, contemplative way. But can we use the soft-boundary concept for anything useful? I think so.

For example, in this neo4j video (minute 5) Marko Rodriguez gives us the following line of Gremlin code:

g.v(1).outE.filter{it.label=='knows' & since > 2006}.count()

We could either be naïve about this and treat 2006 as a hard boundary, or make it a variable and perform sensitivity analysis. In fact, any time we see a number we could turn it into a parameter — ending with a hull of list. We could poke about in that parameter space and by doing so get a better idea of the shape of things than setting a naïve tripwire.

Is there a design pattern for this?

Notice also his gremlins can “be” on multiple nodes at once. That’s certainly not a binary data structure to the codomain. Other non-binary aspects to his graphs:

different words (“coloured edges” in graph parlance) like “speaks”, “has worked with”, “had a child with” — all of the richness and drama of Quine’s ontology of language wrought in the connectome of the graph
the network structure itself
and of course edge weights

Here’s an example from Unix for Poets:

cat bible | grep Abel | uniq -c

So-called “bright lines” appear also in the law (married vs not), statistical regression (dummy/indicator variables), and tax brackets (under $15,000.00 or ≥ $15,000.01).

They’re frustrating because they’re discontinuous. (Actually tax brackets are not but the first derivative is discontinuous.)

Imagine the following (non-existent, stupid) tax system:

If you make under £30,000/year you pay no tax.
If you make ≥ £30,000.01/year you pay 50% tax on every dollar you made (all the way down to £0.01).

It’s frustrating because it’s discontinuous. I might not go as far as to say that continuity, smoothness, holomorphicity, analyticity and so on are “natural to the human mind” — if in fact we can just take a monolithic view on “the” mind — but continuity and smoothness certainly seem—to me and to other mathematical writers I’m thinking of—like they’re more fair, just, or sensible.

Imagine you’re trying to catch an email spammer, and you’ve determined that the character ! is a good trigger for spams. You could either

set a hard boundary: more than 3 !’s, flagged for spam; le; 3 !’s, not flagged
or you could count the number of !’s in the text

The latter approach is more flexible:

you can change the parameter 3 to something else
you can pass the count through a function (like a sigmoid, monotone convex or monotone concave function, or the cumulative-prospect-theory function)
As in minute 14 of this d3.js video you can add (something like a) “blending” parameter
you can set a known algorithm (like logistic regression) to find the optimal parameter value for you
you can combine the ! count with other variables (like counts of the word herbal or counts of the forenames of people in the mail user’s address book)
you can combine the ! count with other variables and use a known algorithm (like a backprop net) to set all the optimal values for you
maybe you can find a way to half-instantiate your desired response when the count is “at half mast” or “in a middling range”.

Back to catching spammers, I drew up an idea for tumblr to catch its spammers a while ago. I noticed a few telltale markers of spam accounts:

quick liking in succession
squatting on a hashtag
high number of likes
no / low content in the title
at first the spammrs were not reblogging stuff (now they not only reblog but post fakey “original” looking text posts … that’s counter-evolution for ya) so they usually had no posts on their blog page
exist ads on the sidebar

They opted for social proof (let people “block” spammy likers from their dashboard and flag them as suspected spammers), which seems to have worked out very well. So I’m not saying “soft boundaries are always better” or something — just that if a “hard boundary” is preventing you from thinking about a problem like you want to, you can get around it pretty easily!

I think computer scientists do use soft boundaries, although they might not draw the same analogy to the “crisp” > sign as I am.

tag clouds don’t just count words — they increase the display size of the word depending how large the count is (maybe the sqrt of the count?). That tag clouds count different words rather could also be construed as a “coloured” codomain.
you don’t just return a webpage or not return a webpage in your crawler. You might get a 404, or you might get a 302. Or you might get a 200, 500, 303, 504, and so on. Additionally the page might be in HTML, JSON, or might simply flip a switch (“turn on my remote TV recording device”).

Business people (I’ve found) think naturally in terms of soft boundaries as well. If your client / boss is using the word “score” you can mimic that directly with what I’m calling a “soft boundary”.

All you’ve got to do is make up a functional that “measures stuff” any way you want, and slide your > sign along the resulting smooth scale.

Bare Bones Introduction to Git

This is the shortest introduction to git I could come up with.

Sometimes a really, really short introduction can get you over the hump to where you can spend a few hours going through a real introduction.

Here is how to use git:

sudo apt-get install git
sudo apt-get install tig


cd ~/school/spring2012/ayurveda/thesis/


git init
ls -R .git
tig


git add thesis_v1.doc thesis_v2.doc
git commit thesis_v2.doc -m "about 68% done and now I'm finally saving it properly"
git push
tig


git add *
git commit -a -m "adding everything else in, I'm not going to comment it all b/c I'm lazy"
git push
tig

NB: I don’t know where to execute this in Windows, but on Mac / Linux type the above into a terminal.

You can do this right now, in any directory where you want to back something up and keep working on it.

Even if you’re not a programmer, you can still benefit from git. Git saves all the files in your directory (well, if you git add * — otherwise it just saves the files you tell it to add). It does other things as well.

But just having an easily-exported backup of directories you work in—whether work is writing essays, spreadsheets, your thesis—is enough reason, in my opinion, to use a version-control system like git. It’s especially nice to have every successive version of your thesis (or project for a client) saved incrementally.

Note this does not get your files onto github. However you can follow their instructions and, if you don’t mind these files being public, you get a free remote backup on the github.

If you try the code above and it does not work, please leave a comment describing your system (e.g., Mac Snow Leopard) and the error message you got.

Javascript: The Good Parts by Douglas Crockford

I’m looking for a quote I saw years ago. It went something like this:

Here are two different stories of Creation.

The first is that G-d sculpted each and every animal, flower, fungus, gymnosperm, archaeobacterium, and primate individually, like the most colossal micromanager ever known to the Cosmos.

The second story is that G-d was smart enough to write a “computer” program which, from a few simple rules, would evolve not only the stars, galaxies, and planets, but also all of the life-forms mentioned above (as well as whatever’s yet to come).

Which Creator would you find more impressive?

I’m pretty sure the writer was an Israeli game theorist. Can anybody help me source this please?

icardbig>

averaging averages of averages to find a fixed point of a function
linear combinations of electrical signals, polynomials, waveforms, or vectors
in LISP/code > linear combination is (* scalar (+ vector vector))
computer ~~science~~ = magic, not done on computers
~~computer~~ science is about process, new ways to do things, and in particular making computers do things you wouldn’t expect them to do

The theory of universal algebras was well-developed in the twentieth century. [It] provides a basis for model theory, and [provides] an abstract understanding of familiar principles of induction, recursion, and freeness.

The theory of coalgebras is considerably [less] developed. Coalgebras arise naturally, as Kripke models for modal logic, as automata and objects for object oriented programming languages in computer science, and more.

Jesse Hughes, A Study of Categories of Algebras and Coalgebras

$Fuzzy Logic Not everything is so simple as true or false. Even declarative statements may evaluate outside {0,1}. So let’s introduce the kind-of: truth ∈ [0,1]. Examples of non-binary declarative statements: Shooting trap, my bullet nicked the clay pigeon but didn’t smash it. I 30%-hit the mark. I’m not exactly a vegetarian. I purposely eat ⅔ of my meals without meat, but — like yogini Sadie Nardini — I feel weak if I go 100% vegetarian. So I’m ⅔ contributing to the social cause of non-animal-eating, and I’m a ⅔ vegetarian. I’m sixteen years old. Am I a child, or an adult? Well, I don’t have a career or a mortgage, but I do have a serious boyfriend. This one is going to be hard to assign a single number as a percentage. So that’s the motivation for Fuzzy Logic. It sounds compelling. But the academic field of fuzzy logic seems to have achieved not-very-much, although there are practical applications. Hopefully it’s just not-very-much-yet (Steven Vickers and Ulrich Höhle have two interesting-looking papers I want to read). I see three problems which a Sensible Fuzzy Logic must overcome: Implication. Classical logic (“the propositional calculus”) uses a screwed up version of “If A, then B”. It equates “if” to “Either not A, or else B is true, or else both.”Fuzzy logic inherits this problem — but also lacks one clear, convincing “t-norm”, which is the fuzzy logic word for fuzzy implication. Can you come up with a sensible rule for how this should work?: A implies B, and A is 70% true. How true is B? Furthermore, should there be different numbers attached to “implies” ? Should we have “strongly implies” and “weakly implies” or “strongly implies if Antecedent is above 70% and does not imply at all otherwise” ? You can see where I’m going here. There is an ℵ2 of choices for the number of possible curves / distributions which could be used to define “A implies B”. Too specific. Fuzzy logic uses real numbers, which include transcendental numbers, which are crazy. Bart Kosko’s book explains FL with familiar two-digit percentages, which are for the most part intuitive. So I can accept that something might be 79% true — but what does it mean for something to be π/4 % true? Or e^e^π^e / 22222222222 % true?We’re encumbering the theory with all of these unneeded, unintuitive numbers. One-dimensional. For all of the space, breadth, depth, and spaceship adventures contained in the interval [0,1], it’s still quite limited in terms of the directions it can go. That is [0,1] comprises a total order with an implied norm. Again, why assume distance exists and why assume unidimensionality, if you don’t actually mean to. There are alternatives.Unidimensionality excludes survey answers like N/A I don’t know Sort of Yes and no It’s hard to say I’m in a delicate superposition , — or rather it maps effectively different answers onto the same number. Sometimes things are both good and bad; sometimes they are neither good nor bad; sometimes things are not up for evaluation; sometimes a generalised function (distribution) expresses the membership better than a single number; sometimes the ideas are topologically related or order related but not necessarily distance related; sometimes an incomplete lattice might be best. So those are my gripes with fuzzy logic. At the same time, Kosko’s book was my introduction to an interesting, new way of thinking. It definitely set my mind spinning. For the logical mind that wants a rigorous framework for understanding ambiguity, vagueness, and gray areas, fuzzy logic is a good start.$

Fuzzy Logic

Not everything is so simple as true or false. Even declarative statements may evaluate outside {0,1}. So let’s introduce the kind-of: truth ∈ [0,1].

Examples of non-binary declarative statements:

Shooting trap, my bullet nicked the clay pigeon but didn’t smash it. I 30%-hit the mark.
I’m not exactly a vegetarian. I purposely eat ⅔ of my meals without meat, but — like yogini Sadie Nardini — I feel weak if I go 100% vegetarian. So I’m ⅔ contributing to the social cause of non-animal-eating, and I’m a ⅔ vegetarian.
I’m sixteen years old. Am I a child, or an adult? Well, I don’t have a career or a mortgage, but I do have a serious boyfriend. This one is going to be hard to assign a single number as a percentage.

So that’s the motivation for Fuzzy Logic. It sounds compelling. But the academic field of fuzzy logic seems to have achieved not-very-much, although there are practical applications. Hopefully it’s just not-very-much-yet (Steven Vickers and Ulrich Höhle have two interesting-looking papers I want to read).

I see three problems which a Sensible Fuzzy Logic must overcome:

Implication. Classical logic (“the propositional calculus”) uses a screwed up version of “If A, then B”. It equates “if” to “Either not A, or else B is true, or else both.”

Fuzzy logic inherits this problem — but also lacks one clear, convincing “t-norm”, which is the fuzzy logic word for fuzzy implication. Can you come up with a sensible rule for how this should work?:
- A implies B, and A is 70% true. How true is B?
- Furthermore, should there be different numbers attached to “implies” ? Should we have “strongly implies” and “weakly implies” or “strongly implies if Antecedent is above 70% and does not imply at all otherwise” ?
You can see where I’m going here. There is an ℵ₂ of choices for the number of possible curves / distributions which could be used to define “A implies B”.
Too specific. Fuzzy logic uses real numbers, which include transcendental numbers, which are crazy. Bart Kosko’s book explains FL with familiar two-digit percentages, which are for the most part intuitive. So I can accept that something might be 79% true — but what does it mean for something to be π/4 % true? Or e^e^π^e / 22222222222 % true?

We’re encumbering the theory with all of these unneeded, unintuitive numbers.
One-dimensional. For all of the space, breadth, depth, and spaceship adventures contained in the interval [0,1], it’s still quite limited in terms of the directions it can go. That is [0,1] comprises a total order with an implied norm. Again, why assume distance exists and why assume unidimensionality, if you don’t actually mean to. There are alternatives.

Unidimensionality excludes survey answers like
- N/A
- I don’t know
- Sort of
- Yes and no
- It’s hard to say
- I’m in a delicate superposition
, — or rather it maps effectively different answers onto the same number.
- Sometimes things are both good and bad;
- sometimes they are neither good nor bad;
- sometimes things are not up for evaluation;
- sometimes a generalised function (distribution) expresses the membership better than a single number;
- sometimes the ideas are topologically related or order related but not necessarily distance related;
- sometimes an incomplete lattice might be best.

So those are my gripes with fuzzy logic. At the same time, Kosko’s book was my introduction to an interesting, new way of thinking. It definitely set my mind spinning. For the logical mind that wants a rigorous framework for understanding ambiguity, vagueness, and gray areas, fuzzy logic is a good start.

Seriously … why don’t math classes use computers? Excel, simple Python scripts, Mathematica / Sage, everything beyond the TI-83. Kids could be creating totally sweet visuals instead of cribbing formulae. And thinking instead of copying.

I can say from my experience teaching that giving kids some real data and having them muck around with it for an hour, was better than an hour of my lectures. (And guess what, they understood lecture better after they’d looked at real data.)

Why, why, why, why, why is maths education so senselessly, wretchedly bad?

The algorithmic art of Reza Ali
(via wowgreat & rudosvstecnicos)

hi-res

From nature to hardware to software in less time than it takes to drink a Starbucks Trenta. I love this.

In about seven minutes (5:00 to 12:00) he goes from

Maxwell’s Laws, to
lumped circuit elements, to
simple circuits, to
processors, to
operating systems, to
programming languages, to
video games.

Also to toasters and spaceship control modules.

(Source: ocw.mit.edu)

In the esoteric programming language Piet, the above picture is actually a program. “Get through your head that a program is just words” — or pixels, apparently.

It prints Hello World.

I love this. In about seven minutes (5:00 to 12:00) he goes from Maxwell’s Laws, to lumped circuit elements, to simple circuits, to processors, to operating systems, to programming languages, to video games—also to toasters and spaceship control modules.

(Source: ocw.mit.edu)

Hard Boundaries, Soft Boundaries

Bare Bones Introduction to Git

Fuzzy Logic

BROWSE