Skip to content
February 2, 2015

Jeopardy’s wild-card system is unfair. Here’s a better way

2015 Teachers Tournament
By the Numbers: The difficulty of predicting wild cards
Quarterfinals Semifinals Finals
1 2 3 4 5 1 2 3 1 2

With the 2015 Teachers Tournament kicking off Monday, Jeopardy! will once again become a less-cutthroat version of itself. Each night, Alex will remind us that “the four high-scorers among non-winners” will join the five victors as semifinalists next week.

To tournament contestants, the wild cards are a blessing: they guarantee that 60% of the field will play a second game (as opposed to around 27% in the past decade of regular play).

They are also a huge source of consternation. The main reason: the required score is nearly impossible to predict.

Wild-card cut-offs regression

Above is a chart of the wild-card cut-offs since Season 21.

We can use what’s known as the R-squared value to see how easy it is to predict the next wild-card cut-off. The R-squared value can range from 0 (there’s no way to predict) to 1 (you can predict with certainty).

A simple linear regression of these data yields an R-squared value of 0.036.

I’ve done analyses of wild-card cut-offs before, and my attempts at predicting have been abysmal. Now, imagine trying to play the game with this extra bit of uncertainty looming over your head.

For those of you interested, here’s more on R-squared.

There must be a better way

I must first acknowledge one fact of life: the powers that be have no reason to change the current format. It gives them a slate of nine semifinalists, and is easy to understand.

That said, there’s no reason to accept it as the best – or even the most exciting.

The most common complaint about wild cards is the inconsistency of Final Jeopardy! clues across games.

Consider, for example, the Hansel & Gretel clue in the concluding quarterfinal of the last Tournament of Champions. It far and away polled as the easiest of the five on JBoard (84% reported a correct response), and I am unaware of a TOC competitor who wouldn’t have gotten it.

Had all fourteen quarterfinalists with money received that clue and responded correctly, the standings would have been as follows:

Winners Wild cards Eliminated
Ben Ingram John Pearson (16,800) Mark Japinga (15,003)
Arthur Chu Sarah McNitt (16,700) Rebecca Rider (11,600)
Julia Collins Joshua Brakhage (16,206) Andrew Moore (10,399)
Terry O’Shea Sandie Baker (15,500) Rani Peffer (7,599)
Jared Hall Jim Coury (5,600)
Drew Horwood (-800)

Mark and Rebecca would have failed to qualify, with John and Sarah taking their places; Julia and Josh would have switched spots.

5 of the 9 players trailing heading into Final felt their scores were insufficient for a wild card, because they wagered nearly everything. (Terry O’Shea, who led by a slim margin on Thursday’s match, did as well.)

But with no way to predict what the cut-off is, should we leave players to the whims of a lucky Final Jeopardy! clue?

A modest proposal

In a single game of Jeopardy!, all three players are subject to the same microcosm. If one player gets a head-scratcher of a Final, the other two players do, as well.

When you compare across games, however, and you want to reward the best players, it makes sense to reduce as many variables as possible while changing neither the rules nor the experience for viewers.

Giving every game the same Final (or even more absurdly, the same set of 61 clues, as some have suggested) would make for terrible television. Short of that, what can be done?

I propose we turn the wild cards into a lottery based on each non-winner’s score.

It’s been done before (sort of)

Here’s how the lottery would work:

Each non-winner gets “entries” proportional to her total at the end of the game. For example, someone finishing with $10,000 receives around twice as many “entries” as someone with $5,000.

The drawing is a matter of optics. The staff could plug the numbers into a computer and let a random-number generator sort things out.

Or – and this would be my preference – they could give each player a card for every $1,000 (or some other large number), put all of those cards into a bowl, and draw four unique names.

There’s a precedent for this method: the selection of the participants in the Tenth Anniversary Tournament.

Plus, this would make for great TV (and great drama in the studio).

Speaking of wild cards…

It’s counterintuitive, but this change might make many players even more aggressive in Final. To justify this, I draw on another entity that recently changed its wild-card system: Major League Baseball.

Four teams in each League used to make the postseason: the three division champions, and the non-champion with the best record. All would start on even footing.

In 2012, however, MLB changed the format so that the top two non-champions would meet in a one-game playoff for the right to advance.

This increased the value of winning the division. No longer could a team guaranteed a playoff berth just coast to a wild card.

The same would be true here. You want to avoid the lottery? You’d best win the game.

Do I take the risk?

This system gives each player more of an opportunity to put his fate into his own hands – but with an increased risk, as doubling one’s score does not double one’s chances of advancing.*

Faced with the category ART SUBJECTS in my TOC, I wagered just over half of my $7,200 total. I got a nearly-impossible Final, and missed the final wild card by $500 after Chris Miller doubled up from $2,000 on a slam-dunk.

(To be fair, Chris ended up making the semifinals of the Ultimate Tournament of Champions, so he was certainly no slouch.)

Under my proposed system, I would have been happy to stand pat, as art was probably my weakest category, and first place already had the game locked up. I could have stood on my performance over the first 60 clues, rather than roll the dice on a single question.

*Let’s say I have 10,000 heading into Final, and the total “pool” of wild-card points is 50,000. I have 1/5 of the entries, or 20%. But if I wager everything, getting it right will give me 20,000 out of 60,000 total points: 33%.

A quick simulation

From the 2014 TOC, I gave each of the eight non-winners one draw for every $1,000 (rounded). I then simulated 10 drawings. Here were the results:

Name Score Entries Picked
Sandie Baker 15,500 16 90%
Mark Japinga 15,003 15 80%
Rebecca Rider 11,600 12 80%
Julia Collins 9,100 9 90%
Rani Peffer 7,599 8 50%
Jim Coury 5,600 6 10%
Sarah McNitt 500 1 0%
Andrew Moore 1 0 0%

The sample size was small enough to see some unexpected variations; see my Guide to Game Theory on expected value for more on this.

Often we drew a name that had already been picked, so we had to draw more than 4 cards; the average number of draws was 6.9 (range 5–12, standard deviation 2.4).

Of course, there is still an element of luck, but it’s greatly reduced. If there’s one area we can change in the tournament structure to ensure a better player has a better chance of advancing, this is it.

2015 Teachers Tournament
By the Numbers: The difficulty of predicting wild cards
Quarterfinals Semifinals Finals
1 2 3 4 5 1 2 3 1 2
  1. How appropriate is a linear regression on cutoff vs. time given that there is no demonstrable relationship between the two? Typically with a linear regression you have some basis for believing that the dependent variable is dependent on the independent variable, but that’s not the case with wildcard cutoffs. You could shuffle the tournaments in time without any loss of the meaning of the independent events and get a different regression model. All that a low coefficient of determination is telling you here is that the wildcard cutoff is not a function of time, but we already knew that. Perhaps it would be more useful to look at wildcard cutoffs as a function of the scores produced by the tournament players in regular play (works only for ToCs, though).

    The fact that the wildcard cutoff is unpredictable doesn’t make the system less fair, it makes it more fair, because every player is chancing it against the same uncertainty. I’m not sure how a lottery solution based on score differs substantively from the present system from the perspective of the player (since in either case the goal is to maximize score at the risk of falling into a zone of lesser chance of advancing), and this post doesn’t convince me that it’s more fair, although I’m open to being persuaded. The biggest problems with a lottery, though, would be the airtime it would take, and the requirement of ensuring its integrity, both of which would be unnecessarily costly to the production…

    • I did have some qualms about using a linear regression against time, for the reasons you cite. Since most of these tournaments feature a slate of fifteen totally unknown quantities, however, it seemed like the most reasonable variable for comparison.

      Players who are preparing for a tournament are likely using previous numbers as a guide; if enough players are doing this, these numbers should at the very least converge to a smaller range. (I know Ben Ingram, for one, memorized the quartiles for wild-card probabilities in the TOC.)

      Even within tournament types the R-squared value is very low. I might look at the additional information available for the TOC during a later quarterfinals.

      The idea of fairness relates to the inconsistency of FJ clues across games. If we assume that the ~38% of non-leaders who wagered big would have done the same no matter the category, you are filling a significant portion of the semifinalist field with people who got the “right” FJ.

      Of course, a huge part of Jeopardy! does come down to luck – but at least when you’re playing in the same game, you can mitigate others’ “luck” to some extent through good strategy.

  2. Michael Falk permalink

    I think r-squared would be a lot more appropriate if you didn’t throw teens, college, teachers, and ToC all on the same graph. For instance, a ToC competitor who has done his homework on his opponents is going to wager far differently than, say, a teen who does the arithmetic wrong but lucks into a win anyway. Using both of those to predict a trend in wild-card cutoff doesn’t seem sound.

  3. Roger permalink

    keith, if the cutoff was 15k for every tournament, what would the r^2 be? Would it be predictable?

What do you think?