Quantcast

Posts tagged with recommendation engines

"The Internet makes everything available, but mere availability is meaningless if the products remain unknown to potential buyers."

(possibly attributable to Gavin Potter, a Netflix Prize contestant; possibly to Jordan Ellenberg, who writes for Slate and Wired)




Another unmeasurable distance is ★★★ movie ratings.

Movie ratings are drawn from the set {★,★★,★★★,★★★★,★★★★★} and related by the total ordering >:

  • ★★★★★ > ★★★★
  • ★★★★ > ★★★
  • ★★★ > ★★
  • ★★ > ★

and the transitivity of > gives the rest of the relations.

TWO PLUS TWO ≠ FOUR

However ★★★★ is not the same thing as 4, because 4 comes with all the baggage of being an integer. Baggage like the usual metric whereunder

  • |42| = 2,

whereas |★★★★−★★| ≠ ★★.

If one naïvely assumed {★,★★,★★★,★★★★,★★★★★} ≅ {1,2,3,4,5}, that would mean

  • |★★★−★|  =  |3−1|  =  |★★★★★−★★★|  =  |2|.

Which would be wrong.

HOW I THINK PEOPLE USE ★★★ RATINGS

There’s no reason to believe that the distance between ★★★★★ and ★★★ is 2 or that it’s the same distance as between ★★★ and ★. I believe there is a wider gulf between ★★★ and ★ for most people.

It depends on the person but creo q’ a lot of people basically only use three, four, and five stars. Mostly they just use four and five stars because they only watch movies they like.

Then when faced with two choices (★★★★ versus ★★★★★) they may think back to other movies they’ve rated, and wish they had a finer scale of gradation, or just something else to say about them — like in an orthogonal direction.

People use ★★ and ★ but not no creo very judiciously. It’s kind of like the hotness scale … but that’s another topic.



WHAT’S NEXT

I actually have a long, in-depth critique of the ★★★★ system—which also suggests better ways to do surveys in general. But I’ve gone on too long already so let me just preview that critique by saying:

Bad data in, bad recommendations out. Don’t blame yourself, Netflix Prize contestants.

PS: Really wanted a subjunctive mood while writing this. Thanks a lot, English Language. Not.




The Netflix Prize was awarded to the team with the algorithm that most accurately guessed people’s movie tastes. Accurate, according to some measure: root-mean-squared error, or the L₂ norm.

In my opinion, that’s the wrong measure of success. Netflix selected for algorithms that predicted well across all data, penalizing large misses extra. But that’s not what makes a recommendation algorithm good.

The best algorithm, I think, should observe my tastes and recommend just one product that I’ve never heard of (or at least never tried), that I absolutely love. It’s OK if I like a movie and you show me another one by the same director — but I could have done that myself. The best algorithm would say:

You like Cowboy Bebop + Out Of Africa + Winged Migration so you will like = Seven Samurai.

Cowboy Bebop indicates that I like Asian sh*t; Out Of Africa is an old classic; Winged Migration doesn’t have a lot of talking. Put them together and you get an Asian classic without a lot of talking.

That’s just an example of a recommendation that would fit my criteria of goodness.

In other words,

  1. only the "most recommended" movie matters
  2. it should blow me away
  3. it should be surprising.

RMSE fails #1 because accuracy in the highest recommendation matters just as much as accuracy in every other recommendation.

As a result, today’s recommendation engines are conservative in the wrong ways and basically hack together machine learning fads.




Facebook EdgeRank
Another quasimetric! Actually a pseudo-quasi-metric. Facebook’s news feed decides what status updates to show you based on three factors:
affinity u — how much you like somebody (FB’s guess)
type of post w — videos and photos are more interesting than “Chris liked Lili’s status update”
time since post d — show newer stuff

The affinity score is a pseudoquasimetric — it’s a one-way measure. If I’m always looking at your profile because you’re a fascinating girl that I met at a party and have been stalking ever since, your updates will show up more in my feed.
But, say you’re the girl and you didn’t think I was interesting at all and never looked at my profile after you Accepted my offer of internet-ual befriendship, my stories won’t be interesting to you and are unlikely to be displayed.
In other words, stalkers don’t show up in your news feed.
The properties of a quasimetric, again, are:
never below zero ✓
the only “distance of zero” is the distance of something to itself ✗
dist( ①→③ )  ≤   dist( ①→② )   +   dist( ②→③ ) ✓
Had to drop the second one because there might be lots of FB friends Ⓑ,Ⓒ,Ⓓ,Ⓔ who are not important to me Ⓐ and thus dist( Ⓐ→Ⓑ ) = 0 | Ⓐ≠Ⓑ. (But maybe not?)
Anyway, I threw a "pseudo" on the "quasimetric" just in case lots of different people are of zero interest.
Math news out.

Facebook EdgeRank

Another quasimetric! Actually a pseudo-quasi-metric. Facebook’s news feed decides what status updates to show you based on three factors:

  1. affinity u — how much you like somebody (FB’s guess)
  2. type of post w — videos and photos are more interesting than “Chris liked Lili’s status update”
  3. time since post d — show newer stuff

The affinity score is a pseudoquasimetric — it’s a one-way measure. If I’m always looking at your profile because you’re a fascinating girl that I met at a party and have been stalking ever since, your updates will show up more in my feed.

But, say you’re the girl and you didn’t think I was interesting at all and never looked at my profile after you Accepted my offer of internet-ual befriendship, my stories won’t be interesting to you and are unlikely to be displayed.

In other words, stalkers don’t show up in your news feed.

The properties of a quasimetric, again, are:

  1. never below zero
  2. the only “distance of zero” is the distance of something to itself
  3. dist( ①→③ )  ≤   dist( ①→② )   +   dist( ②→③ )

Had to drop the second one because there might be lots of FB friends Ⓑ,Ⓒ,Ⓓ,Ⓔ who are not important to me Ⓐ and thus dist( Ⓐ→Ⓑ ) = 0 | Ⓐ≠Ⓑ. (But maybe not?)

Anyway, I threw a "pseudo" on the "quasimetric" just in case lots of different people are of zero interest.

Math news out.