The better you are, the more credit you get
How a surprise gets split among the people who made it.
In Part 1, I made a very hand-wavy statement that Megan Ganz received 74% of the credit for the surprising success of her debut episode, "Cooperative Calligraphy." Today, we'll discuss exactly how that split is decided.
Here's a thought experiment. Let's say a rookie pitcher, who you know nothing about, is making his major league debut. He promptly gives up a home run to Barry Bonds, the best hitter in the modern era. Another rookie pitcher makes his debut and gives up a home run to the backup catcher, who is the worst hitter in the league. If you know nothing else about either rookie pitcher, you're going to assume the second pitcher is worse, because the first outcome (a home run to Barry Bonds) is more expected given the quality of the hitter. That philosophy powers our models.
On TV, the writer and the director are the pitcher and the hitter. Every year, the model measures how much of a typical episode's over- or underperformance is driven by the writer vs the director. In most years, it falls at roughly 70% writer and 30% director. This is derived empirically from the model fit. Then it tilts that split for each episode, depending on whether the over- or underperformance (the "surprise") agrees with each person's body of work, or record. If an episode beats its prediction, whoever the model already thought was good takes a bigger slice of the credit. If it falls short, whoever the model thought was good takes a smaller slice of the blame. It's a zero-sum game, so the entire pie has to be divvied up.
There are four ways that can go. Here's one real example of each.
weak writer
strong writer
A strong director with a weak writer
By 2009, Peter Tolan had directed 11 episodes of Rescue Me, most of which outperformed fellow episodes within each season. His average impact on a Rescue Me episode at that point was +0.30. Evan Reilly's 13 episodes as a writer had run a bit behind (−0.08). In season 5 they made two episodes together.
"Iceman" was a hit: it beat its prediction by +0.33. That's what you'd expect from Tolan's record, so Tolan gets 41% of the surprise, as opposed to the typical ~30%.
"Jump" missed by −0.39. Because the model entered the episode thinking favorably of Tolan and somewhat skeptical of Reilly, Tolan takes just 14% of the blame.
A weak director with a strong writer
Flip the roles. Howard Gordon was one of 24's key writers, and his episodes ran well ahead of their seasons (+0.19). Jon Cassar directed dozens of hours of 24, but on the show his episodes ran a little behind the rest (−0.09). That's not to say Cassar was a bad director. Just that, measured against the rest of 24, his hours came in lower.
"Day 5: 7:00 a.m.-8:00 a.m.," a classic episode that doesn't end great for David Palmer, rated 0.62 higher than predicted. Gordon's record agrees with it, so Cassar gets less of the credit than a director normally would: 25%.
"Day 4: 8:00 a.m.-9:00 a.m." underperforms by −0.31. The same logic runs the other way and Cassar takes 43% of the blame.
How far can it tilt?
Not very far, most of the time. Across every episode with one writer, one director and a large residual in either direction, the director's share lands between 25% and 35% of the surprise 94% of the time. The tilt only gets extremely large when a truly elite track record is paired with a truly dismal one.
Ratings are IMDb user ratings from IMDb's public data files. Credits are from the model that built this page.
Next in the series: Eric Kripke, elite showrunner (Part 3).