Sunday, May 8, 2016

Random Thought - How to UnPuppy a Hugo Ballot

Alternate history is part and parcel of the science fiction genre. Imagining what might have happened if the world had taken a slightly different course has a long and proud tradition in genre fiction. Imagine, for example, what might have happened if the members of Worldcon had not chosen to insult Larry Correia by voting him as one of the five most promising writers in genre fiction. Imagine what might have happened if those same voters had not issued the same insult to Brad Torgersen, and then further insulting Torgersen by choosing his story Ray of Light as one of the five best novelettes of the year. If Worldcon voters had simply ignored these two men, rather than insulting them with recognition and honors, the pair might not have decided to launch and participate in an ideologically driven bloc-voting campaign that dominated the Hugo nominating process and wedged a collection of mediocre to miserable work onto the list of Hugo finalists in 2015.

The question is, how does one go about calculating the exact effect that this hate-motivated campaign had upon the 2015 Hugo award finalists. Or, in other words, how many Sad Puppy nominators were there? Of course, given that there was a related, but somewhat distinct campaign headed by Theodore Beale called the Rabid Puppies, one can't figure out how many Sad Puppy nominators there were without figuring out how many Rabid Puppy nominators there were as well. Both of the campaigns promoted very similar slates, but they diverged in a handful of small areas, which, where they existed, make it possible to figure out roughly how many members of each group nominated in the various categories.

The Hugo nominating process is relatively simple (although some changes are in the process of being passed), and has been for much of the sixty plus year history of the award. Members of the World Science Fiction Convention can make up to five nominations in each category of the award. These nominations are then tallied and the top five selections in each category become the shortlist of finalists. This "first past the post" style system served the Hugo awards well for decades, as most people participated as individuals voting their own personal preferences each year. The resulting ballot was the product of a chaotic electorate that used something akin to the "wisdom of the crowds" to generate a result. Organizing into a bloc to vote for a particular nominee or set of nominees has always been viewed as being against custom and a practice to which only the unethical would resort. In 1987, a group of Scientologists banded together and bloc-voted the L. Ron Hubbard novel Black Genesis onto the list of Hugo finalists. In the subsequent round of voting, Black Genesis finished behind "No Award". The message that Hugo voters have sent has been consistent through the history of the award: Bloc voting is considered out-of-bounds, and will be treated accordingly. Despite the various Pups whining about how they were responding to alleged "secret slates" and "hidden cabals" that had been manipulating the awards, they have never proffered any evidence that would substantiate such claims, and, to be honest, most of their assertions have been so far divorced from anything resembling reality that calling them laughable is giving them too much credit.

After each Hugo award ceremony, the Hugo administrators release the voting and nominating data for the awards, including the top fifteen potential finalists in each category as well as the total number of nominations they received. Using this list, we can tell who might have made it onto the list of Hugo finalists had the two Puppy slates not existed. The easiest way to do this would be to simply edit out all of the finalists who were from either or both of the two slates and replace them as appropriate with the next one, two, three, four, or five nominees from the longlist. This sort of brute force methodology would be too simplistic - there is the possibility that some of the slated works would have become Hugo finalists even without the push given to them by the bloc-voting campaign. For example, Guardians of the Galaxy was enormously popular, and would have certainly become a Hugo finalist even if the Puppies had completely ignored its existence.

So how does one figure out how many nominators there were taking their cue from the two slates. In the Best Novel category the two slates overlapped in four spaces, with each suggesting a different fifth book. Both slates pushed Skin Game by Jim Butcher, Monster Hunter Nemesis by Larry Correia, Lines of Departure by Marko Kloos, and the Dark Between the Stars by Kevin J. Anderson. The Sad Puppy slate put Trial by Fire by Chuck Gannon in its fifth slot, while the Rabid Puppy slate threw its support behind The Chaplain's War by Brad Torgersen. Neither slate was able to put the novels they didn't have in common with one another onto the list of Hugo finalsts: Trial by Fire came in sixth place in the nomination round, while The Chaplain's War came in seventh. The totals for the Puppy supported works in the category looked like this:

Skin Game387
Monster Hunter Nemesis372
Lines of Departure270
The Dark Between the Stars263
Trial by Fire199
The Chaplain's War196

This gives a maximum figure for combined Puppy support of 387, which is actually quite close to the support the Sad Pups gave to Trial by Fire combined with the support the Rabid Pups gave to The Chaplain's War (which, when combined, yield a total of 395). Using these as proxies for Puppy support would assume that every vote in favor of all of these novels was a vote made by someone participating in the bloc-voting engaged in by the two Puppy slates, which would probably be an exaggeration. Chuck Gannon's novel, for example, was able to muster a Nebula Award nomination for his novel in 2015 without help from the two slates, and Jim Butcher is popular enough that assuming that every person putting his novel on their Hugo nominating ballot was a Puppy of some stripe is probably an unwarranted stretch. On the other hand, in 2014, without the benefit of a slate, Gannon's book Fire with Fire did not place in the top fifteen nominated novels for the Hugo despite also being nominated for a Nebula. In 2013, the last time Jim Butcher had a novel that was eligible for the Hugo (specifically, Cold Days), his book also did not place in the top fifteen most nominated works in the category. This doesn't mean that neither book got any nominations, but it does mean that Trial by Fire received fewer than 66 nominations in 2014, and Cold Days received fewer than 55 in 2013.

If one were to treat the nominations for Trial by Fire and The Chaplain's War in conjunction with the nominations for Skin Game as proxies for the level of support for then one could assess the Sad Puppy and Rabid Puppy strength in the category as being about 195 nominators each. Given that The Dark Between the Stars was only able to garner 263 votes, one could argue that the total combined strength of the Sad and Rabid Puppies is somewhat lower than that - perhaps as low as 130 for each camp - which actually maps reasonably close to the difference between the number of nominations Trial by Fire received in 2014 and the maximum number of nominations Fire With Fire could have received in 2013. On the other hand, one could also argue that the lower nomination total for The Dark Between the Stars is simply the result of slate decay: Those participating as bloc-voters nominated the top choices on the slate and as they worked their way down the ballot simply got lazy and didn't fill in all of the blanks. The point here is that this is, at best, an inexact estimate that requires a lot of assumptions and a lot of educated guesses. In the case of the Best Novel category, it doesn't really matter, since whether you estimate the total Sad and Rabid Puppy support as adding up to roughly 390 or roughly 260, none of the novels that appeared on either slate would have made it onto the list of finalists once one removes those nominations from their totals. Skin Game, the top Puppy-slated work, would have ended up with 127 nominations, placing it seventh in the revised totals, just behind Andy Weir's novel The Martian and just ahead of Brandon Sanderson's novel Words of Radiance.

Does this mean that we can just say that Sad Puppy support was somewhere between 195 nominators, and Rabid Puppy support was roughly the same? Well, no. The problem with that kind of mechanical analysis is that it ignores the well-observed fact that fewer people nominate in the the "down ballot" categories such as Fan Writer and Fanzine, meaning it takes fewer nominations to get onto the Hugo ballot in those categories than it does in the popular categories such as Best Novel and Best Long Form Dramatic Presentation, a phenomenon that is known as "ballot decay". Many people vote in the most popular categories, many of those same people are either not interested in the other categories, or don't feel themselves to be knowledgeable enough to nominate in them. Although one might think those simply voting for a slate of candidates for the ballot would not experience this tendency, a review of the voting patterns in the nominating stage of the 2015 Hugo process reveals that they do. Therefore, to determine what the Hugo ballot would have looked like absent the two Puppy campaigns, one needs to come up with an estimated figure of Sad and Rabid Puppy nominators for each of the categories in which either or both slate had candidates. This is, as with noted before with the Best Novel analysis, an extremely inexact science that involves taking the published stats and making a collection of reasonably educated guesses to come up with an approximate figure.

The interesting thing that emerges from such an endeavor is how generally anemic the Sad Puppy support seems to have been in several categories. Even in the year that they were the prime story of the Hugo season, the Sad Puppies appear to have been, in fact, junior partners to the Rabid Puppies. Although these numbers are estimates, in almost every category, the stats show that the Sad Puppies at best matched the Rabid Puppies in terms of numbers, and in many categories, were clearly outnumbered by the hydrophobic counterparts.

CategoryEstimated Sad PuppiesEstimated Rabid Puppies
Novel
130-195
130-195
Novella
130-165
145-170
Novelette
80-100
165
Short Story
25-80
100-150
Dramatic Presentation, Long
70
100
Dramatic Presentation, Short
40
85-100
Editor, Short
60-110
160
Editor, Long
75-130
165
Professional Artist
40-60
120
Semiprozine
100-110
100
Fanzine
60-80
120
Fanwriter
130
65
Campbell
75-85
140

There were three categories where the Sad and Rabid Puppy slates overlapped completely, and one in which the slates offered no candidates. The slate had no selections for the Best Fan Artist category. In the other three categories, while one can make an estimate of the two groups' combined strength, one can't feasibly subdivide them. In Best Related Work, I estimate the combined Sad and Rabid Puppy voting bloc to be roughly 205-270 strong. In Best Graphic Story, with both slates supporting the same single work, they mustered 201 voters. Finally, in the Best Fancast category, I estimate the combined Sad and Rabid Puppy strength to have been roughly 160.

Using these two sets of estimates, one can come up with a revised Hugo ballot with the works that probably would have been on the list of finalists absent the existence of the two Puppy campaigns. I'll tackle that next.

Random Thoughts     Home

No comments:

Post a Comment