Eric Kripke, elite showrunner
From episodes to seasons, and what a showrunner is worth.
Parts 1 and 2 detail how writers and directors receive credit for the over- or underperformance of episodes relative to the performance of the rest of that season. That creates an obvious question: who is responsible for the performance of a season of television? Within this current framework, showrunners and the recurring cast receive the credit or the blame for the ratings spread out across the whole season. So we predict the ratings of the full season, given the body of work of the showrunners and cast, on top of a baseline expectation given information such as network, genre, and year. We'll walk through a showrunner example with Eric Kripke.
Supernatural: Certified Hit
Kripke created Supernatural, and he's credited as its showrunner, solo or with partners, for its first seven seasons. Every one of those seasons outperformed its baseline expectation. By the time he left Supernatural, his showrunner projection was an astounding +0.86 points of ratings.
Revolution: A Shocking Whiff
In 2012 Kripke launched Revolution on NBC. We predicted the first season to garner a very strong 8.07, buoyed by Kripke's body of work. The model was very wrong, as it came in at just 6.46. Below you can see how that prediction came together.
So who takes the blame? Kripke came in with a strong body of work, while the cast at the aggregate level was roughly average. As discussed in Part 2, in these scenarios we are more willing to give the previously strong contributor a partial pass. So Kripke took a small slice of the underperformance (−0.38), with the cast, led by Billy Burke, wearing the remaining −1.22. One bad season didn't erase seven good ones, but it did dent his record. Going into his next pilot, his body of work was down from +0.86 to +0.73.
The Boys: Sweet Milky Redemption
In 2019 Kripke launched The Boys on Amazon. Before getting to its first season prediction, now is a great time to explain two things: how the model adjusts the target variable (seasonal ratings), and how it creates a baseline expectation.
A season's rating is the average of its episodes, with one adjustment. Some slots in a season are rated differently no matter who makes them: premieres tend to come in a little low, and the last two episodes, the finale especially, come in high. So before averaging, we adjust for these expectation deviations. For The Boys' first season, the premiere slot was expected to come in 0.24 below the season, the penultimate episode 0.15 above it, and the finale 0.36 above it. Additionally, the season number affects the ratings. On average, second seasons rate differently than eighth seasons.
Baseline Expectation
Before we look at anyone involved in the show, we predict the rating from the demographic information detailed below. As you can see, each of these inputs has been carefully crafted to be causal in nature. What does that mean? Simply put, it means that changing one value shouldn't necessitate changing another value. This allows you to draw conclusions about the importance of each input in the model. There are very few things in life that annoy me more than when someone doesn't decorrelate their model inputs and then speaks definitively about what drives the prediction. One of them is when you arrive at Chipotle and see your order is ready, but instead of sitting on the takeout shelf it's being held hostage behind the counter in a village of other brown bags. But for now, let's focus on the model.
We start by taking the average rating of all shows from the past year. This sets a level. Every other input is then measured as over expected: not how a network's shows rate, but how much better or worse they rate than the level, after accounting for every other input at the same time. Prime Video's +0.44 below isn't the average rating of Prime Video shows; it's how much Prime Video shows beat what their genres, season lengths and formats alone would have predicted. The same goes for genres, season length (short seasons rate lower: −0.25 for 8 episodes), half-hour vs hour, and limited series. All of them are estimated together in one model, so none of them can take credit for another: an hour-long Prime Video show doesn't get paid for being hour-long twice. That's the decorrelation. And each one is built only from other shows, so The Boys can't feed its own baseline.
If a show belongs to a franchise, we also bring in how that franchise's other shows have rated, so a spinoff like Better Call Saul starts from Breaking Bad's level, and its cast doesn't get credit for people already loving the universe. A second step then decides how much each input actually counts (the weight), and we adjust for the season number. Finally, first seasons are pulled a little toward the middle, because history shows brand-new shows land closer to average than their profile suggests.
Just as important as what goes in is when. Every one of these numbers is an out-of-sample prediction made forward in time. The baseline model is refit every year using only seasons that aired in earlier years, with each of those seasons' inputs exactly as they stood when it premiered. So The Boys' 2019 baseline came from a model that had never seen a single 2019 season, The Boys included. The same rule runs through the whole forecast: bodies of work only count seasons that finished before this one started, and the weights on them are refit each year on earlier years only. Nothing in a forecast knows anything the world didn't know the day before the season premiered.
| Input | The Boys | Over expected | Weight | Adds |
|---|---|---|---|---|
| Level | Every show's average from the year before (2018) | 7.42 | ||
| Network | Prime Video | +0.44 | 0.72 | +0.31 |
| Genres | Action, Comedy, Crime | −0.02 | 0.73 | −0.02 |
| Season length | 8 episodes | −0.25 | 0.85 | −0.21 |
| Format | Hour-long | +0.09 | 0.57 | +0.05 |
| Limited series | No | +0.00 | 0.64 | +0.00 |
| Franchise | None (Gen V came later) | 0 | ||
| Season number | 1 | +0.00 | ||
| Model constant | +0.07 | |||
| A show like this | 7.62 | |||
| First-season adjustment | new shows are pulled toward the middle: 74% of the gap is kept | −0.34 | ||
| Baseline expectation | 7.29 |
Showrunner & Cast
With the baseline expectation established, we include the bodies of work for the showrunner and cast (below).
This time the surprise agreed with Kripke's record, so he took a bigger share of it, +0.46 of the +1.03, and the cast split the rest by billing (more on that in Part 4). His credit for the season, +1.09, is the best of his career.
How season 2 gets its prediction
Once a show has a season under its belt, the baseline expectation steps back and the show's own history takes over. Season 1's rating of 9.04 gets split into two pieces: everyone's credit (+1.76, which belongs to the people) and what's left over (7.29, the show's baseline, which belongs to no one). Season 2's prediction starts from that baseline, adds a small adjustment for a show moving into its second season, and then adds back what the returning people carry. When someone leaves, the show keeps part of what they built: about 60% of their last-season credit stays in the prediction (nobody left here). Anyone new comes in with their own body of work. Season 2 added two co-showrunners alongside Kripke, which nudged the prediction slightly.
Season 2 came in at 9.01, within 0.02 of the prediction.
Ratings are IMDb user ratings from IMDb's public data files. Numbers here are from the model that built this page, as it credited each season; the forecasts on show pages were frozen on the December 31 before each season and can differ slightly (The Boys: 8.03 for season 1, 9.03 for season 2).
Next in the series: Grey's Anatomy and the gift or curse of accumulation (Part 4).