When Jeremy and I came up with the concept for
Flickchart, one of the biggest catalysts was that we felt like ratings were never quite good enough to truly describe how much we liked a film using some sort of numerical value. With only 4 or 5 stars, there just wasn’t enough nuance to differentiate from movie to movie. But what about 1-100 scales? Or 10 stars? It was just too arbitrary to pick a value in the middle somewhere, and say concretely that we liked a film
exactly 7.7 out of 10, or 64 out of 100. What could we actually point to within ourselves to say that one movie was a 65 out of 100 instead of a 64 out of 100. It just seems too hard to quantify when judging a single movie’s merits on their own, and coming up with a value that represents one’s honest opinion. The rest, as they say, is history…
That being said, we know there are a lot of people that are justifiably more used to the idea of rating movies instead of ranking them. It’s no wonder, as we’ve been doing it
reportedly since July 31, 1928 when
New York Daily News critic, Irene Thirer (PDF link) , awarded “
The Port of Missing Girls” one star. (Read more about the origins of star ratings at
The Critical Numbers and
Let’s Rate the Ranking Systems of Film Reviews – both from
Carl Bialik, aka “The Numbers Guy”, of
The Wall Street Journal.)
So if you’ve come from a long lineage of rating movies with stars (and let’s face it – we all have), here’s a little exercise to explain how you might reorient your Flickchart rankings and extrapolate them back to just simple ratings – using a few films from my list as examples.
To start things off, let’s take one of my favorite sequels:
Back To The Future Part II.
IMDb uses stars – ten stars in fact – to allow its users to rate movies.
Robert Zemeckis’s sequel currently has
a moderately strong showing at a global rating of 7.5 stars on IMDb. So that means it’s a 75. A C – if that was your grade on a school exam. Or on
Roger Ebert’s scale of 1 to 4 stars: a solid 3 stars. I have this film pretty high on my list – currently my #24. I also have
1015 movies that I’ve seen and ranked on Flickchart.
So to find out my rating for the movie, I can simply do a little math: 24 / 1015 gives me 0.0236453201970443. This rounds out to 0.02, or 2% – the top 2% of my favorite films. This means I would give this film a 98. An A. 4 stars.
Now here’s where the rub is: The number of people who would actually pick Back To The Future Part II over the original Back To The Future are slim (you know you are!), so how does this calculate?
Back To The Future is currently my #5. So 5/1015 equals 0.0049261083743842. Essentially a 100%. A+. 4 stars. Oh wait… how can both of these films be 4 stars? Exactly…
Let’s take another time-travel example:
Primer. Primer is a nice, hard-to-follow, low-budget, indie, psychological sci-fi yarn. It comes in for me currently at my #350. Over at
Rotten Tomatoes, this film has
a 72% Fresh rating on the Tomatometer. This was derived from 113 different critical reviews; 81 positive and 32 negative.
If I do the numbers for this one, I get 350/1015. This equals 0.3448275862068966, or 0.34. A 66%. A D, if I were getting the grade… Now, when you look at the film independently on my list and see that it’s my #350 of all-time, would you expect to say that I would give it a D? A 66% sounds bad, right? But – I like the movie. I don’t know that if I were to rate it individually that I could give it a score of 66, but I trust that it’s fairly accurate in its placement that there are 349 movies that exceed its virtues, thus its position as my #350 is sound. It’s found its more honest rating by the nature of falling into the echelon of films that’s come against it. By comparing Primer to other films, it allows me the benefit of not being clouded by judgement in the moment of saying, “Yeah, I really liked that movie… It’s an A. 4 stars. A 90%.” Because it’s not. Not really. My enjoyment of the film is such that I do in fact think it’s good, but not as good as 349 other movies I
also think are good.
Hopefully these examples prove that ratings are sometimes superfluous, and subject to frivolous, emotional uncertainty. It’s difficult to point to star ratings, or percentage ratings, and say that they can ever be truly accurate because of these exact circumstances that arise when having to make judgments upon any given film. This is why film criticism when written out in words will always outweigh the value of a number tied to a movie. Our hope is that by using comparisons between films (which you do subconsciously when thinking about any movie’s qualities) that you can achieve a similar outcome for how you
really feel about particular movies. Looking at two films’ accolades next to one another brings to mind so many things about each film; many of the same things you might express if you were to write your feelings about them. In a world of stars, we hope to bring you back down to Earth.
@One Small Popcorn –
There certainly can be merits to each system, in addition to the weaknesses. I agree that defining what the number of stars means is important if you are going to use that system. But from there how would you create an ordered list of the movies you have rated? Obviously all of the five-starred movies would be above the four-starred movies, but then what? That is part of what Flickchart attempts to address.
You bring up a good point, though. Knowing the order of a Flickchart user’s list would not necessarily tell you how “good” they think a movie is. If that user only ever saw terrible movies, knowing which movie is in their #1 spot is not very helpful. (Unless you’re into terrible movies.) Likewise, if a user only ever has seen great movies, there’s no sense in mocking whichever movie is in their dead last spot.
The good news in this scenario is that most people have seen movies that cover the spectrum, so looking at another user’s Top 20 movies actually is meaningful. But you’re right that the user would have to define where the cutoff is, for them, on their flickchart to differentiate between the good, the bad, and the ugly. That currently does not exist on the site. We also do not yet have a way for users to indicate what “weight” they would give a particular movie. For example, just how much more does Nathan like Back to the Future over Back to the Future 2? You can’t absolutely tell that yet, even though we know they are respectively #5 and #24 out of 1015 movies on his list. That is something we are working toward to further refine users’ lists and the combined, global list on the site.
Thanks for your post. We really appreciate hearing others’ thoughts on the whole thing, and feedback like that helps us hone our own opinions of the process.
Your conversion of ranking to rating is severely flawed. The percentage of films above a movie does not translate to a numerical evaluation of that movie’s merits. If someone scores the lowest in his class on a test, that does not mean his score is 0. If he’s in the middle of the curve, it does not mean his score is 50. You are assuming that all the movies you’ve seen are perfectly aligned from “bad” to “good” in even, incremental gradations.
If you’re talking about distance between films, an interface where you drag movies away from each other to show how far apart in your estimations they are would be cool, and could result in some kind of rankings list incorporating spaces between individual or groups of films. The films would still be in ranked order, but the intervals/gaps between them on the visibly reproduced scale would not be regular.
If you wanted to represent Star Wars as far and away the greatest movie of all time, then you’d drag it higher up the top of your scale. The 2nd movie would still be 2nd, but there’d be a huge visible gap between it and Star Wars.
So you would do your movie vs movie ranking, then tweak your final list by adding space between movies or groups of movies.
This could allow the user to communicate weighting, without having to go the rating route.
Also, to indicate your recommended cutoff point, you could be given the opportunity to place a seperator line or something. Could have as many of these as you saw fit.
Add these to the thousands of unsolicited requests you guys get every day!
@FitFortDanga – Yes. That’s exactly what I’m inferring. The top of my list represents the best, while the bottom of my list represents the worst. Everything else is in the gradient of most liked to least liked. That’s why I posit that the position of a film on my list most accurately reflects my opinion of said film.
It’s all so true. I mean, there is probably a good solid chunk of my Top 100 that I’ve given 10 stars on IMDB. (What can I say? I’m generous, like Ebert.)
I’ve often thought of comparing my IMDB rankings to my Flickchart, and adjusting one or the other accordingly, but I usually just opt to let my Flickchart work itself out. My IMDB rankings are often based on my feelings after just having seen a particular film; I often wind up changing them after another viewing.
But, of course, that’s the same with Flickchart, too. Very recently, I rewatched Groundhog Day. Subsequently, it jumped over 100 spots on my Flickchart. Think I gave it an 8 on IMDB, but it’s now in my Top 200. Such a subjective thing….
Even now, I’m looking at my Top 50 and seeing a couple problems. Most of those films I genuinely consider to deserve a “10” ranking (including The Fugitive, currently at #50). But I’m noticing a couple of films on there (Die Hard with a Vengeance being a really good example) that I would only consider to be an 8 or 9. Odd….
Both rankings and ratings serve a purpose. I tend to use ratings as an immediate response to a film. They’re a good short-hand both for reviewing and seeing how good someone thinks a film is. It takes all of five-seconds to come up with a rating plus if you want to, you can go back and change it later.
Rankings take time and effort to produce. For some (including me) there is endless fun to be had in the tinkering to be done. They can reflect your changing opinion on a film more subtly than a rating system plus they give rise to fun debates with friends over beer and a pizza.
Both have their place, but as HungryTyger says, just because you have a nice list of ranked films, it doesn’t imply a linear relationship with your ratings. There’s a correlation between my rankings and ratings which is especially close at the top and bottom ends, but more mixed up in the middle.
I think this point has been made in these comments, more or less, but I think the real idea is to give your lowest ranked film an F, and call F a grade of 50, rather than a grade of 0. The problem with considering all films from 0-60 to be an F is that F gets a waaaaay bigger share of the available numbers than D, C, B or A. It make the math a little more difficult, but maybe that’s the best way to make Primer the B that it is, rather than the D that this system indicates it is. (By the way, I’m more inclined to agree with the D grade for Primer — I was not only on the “hard to follow” bandwagon, but was on the “this film is obtuse and pointless” bandwagon. But that’s just me. I guess I’d say it was a good effort that failed more noticeably than most noble failures fail.)
Right, I mainly used the letter grades to show another form of merit. I agree that it’s probably unfair to give anything past my 60% mark an F, but it does mean something if those films are that far below so many others…