Knuckleballs

Unpredictable, rare, and occasionally effective…but always entertaining.

The Value of a Bullpen

with 2 comments


In the Joe Nathan post, commenter Mac asked how much should a closer actually make.  I did a poor job answering his question (read: I ignored it), so today, I’m going to explore not only the value of a closer, but the value of each member of the bullpen.

To fully explain this, I need to present two ideas.  This is a good investment, because I’ll talk about these ideas a lot.  The first is Pythagorean expectation for baseball teams.  Based on a study by Bill James, you can get a general idea about how well a baseball team did by how many runs they score and how many runs they allow.  The term “Pythagorean” is used because the formula looks a lot like the Pythagorean Theorem.  The formula for a team’s winning percentage is:

Win % = (Runs Scored)^2 / (Runs Scored^2 + Runs Allowed^2)

There have been a lot of studies to see if this formula can be further refined, and based on its relative simplicity, it can.  For the purposes of estimating a team’s wins by these two numbers, this basic equation is fairly good.  Let’s take an average team.  Team A scores 800 runs in a season and allows 800.  Based on the formula they should expect to win about 50 percent of their games, or 81.  This passes the sniff test; if you score as much as you allow, you probably will win as much as you lose.  But it would also work for any exponent, so let’s try a few more.

Let’s try it for team’s at the extremes.  Last year’s Boston Red Sox scored 872 runs and allowed 736.  By the formula, they should have expected a 0.584 winning percentage, or 94.6 wins.  They finished the year with a 95-67 record, good for a 0.586 winning percentage.  On the other end of the spectrum, Baltimore scored 741 runs and allowed 876 in 2009.  These numbers give a winning percentage of 0.417, good for 67.6 wins.  The Orioles actually won 64 games, for a 0.395 winning percentage.

The formula does a pretty good job estimating a team’s winning percentage.  Deviations are usually caused by luck; teams winning many 1-run games are fairly lucky, and teams losing lots of 1-run games are usually unlucky.  A  team’s record in 1-run games one year has almost no bearing on it the next year.  Good (or bad) managers and strong (or weak) bullpens can make a difference, but show little difference in this regard.

Back to the average team.  Let’s say instead they score 810 runs and allow 800.  This equates to a 0.506 winning percentage and almost exactly 82 wins.  This means that the extra ten runs equaled almost exactly one win.  At 900 runs scored and 800 allowed, the winning percentage is 0.559 and 90.5 wins.  At 1,000 runs scored and 800 allowed, the winning percentage is 0.610 and 98.8 wins.  At 800 runs scored and 600 runs allowed, the winning percentage is 0.640, for 103.7 wins.  So as we get further away from average, ten runs can mean more or less  depending on the nature of the team.  Either way, it is still a fair approximation for how many wins the runs are worth.  Now that we’ve established how many wins every run is worth, I’m going to attempt to determine how many runs a bullpen is worth, and to do that, I need to explain something called the leverage index.

Leverage index (LI) is a measure of how important each situation is in a baseball game depending on the game state (inning, score, outs, and number of players on base).  An LI of 1 signifies a neutral situation – of completely average importance to winning and losing.  A higher LI shows more important situations, and a lower LI shows the situation as having less of an impact on the result of the game.  60 percent of game situations fall into this lower situation.  The entire table can be seen here. It acts as a multiplier for runs allowed; therefore, in situation with an LI of 2, every run allowed really counts as “2 normal runs.”  Therefore, better pitchers should pitch in high-leverage situations because they are less likely to give up runs.  Nothing I’ve said so far should be too surprising, I’m just putting numbers to it so we can measure it.

An average team throws about 1400 innings per season; I’m going to allot about 900 of them to starting pitchers (5.5 innings per game) and the rest to the bullpen.  A replacement level pitcher has an ERA of about 5.00.  The following table shows some rough numbers for a typical bullpen; I’ve allotted innings, ERAs, and LI’s based on my best guesses.   The final column, RAR, stands for runs above replacement.  It evaluates how much better each pitcher is than a replacement-level pitcher.  The equation is:

RAR = ( 5.00 – ERA ) * ( IP / 9 ) * LI

The tables below are for demonstration purposes only, and I’ll link to my spreadsheet so you can play with it if you’d like.  Here’s the table for a healthy, pretty good bullpen:

Reliever ERA IP LI RAR
Closer 2.50 70 1.7 33
Setup 1 3.00 65 1.4 20
Setup 2 3.75 65 1.2 11
Middle Relief 1 4.00 65 1.0 7
Middle Relief 2 4.25 65 0.9 5
Middle Relief 3 4.50 65 0.7 3
Long Man 1 4.75 55 0.5 1
Long Man 2 4.75 50 0.5 1
Replacement 5.00 0 0.0 0
Total: 3.88 500 1.02 80

As seen, the closer is worth about three wins in a vacuum.  However, baseball is not played in vacuums; it’s played on fields.  Here’s what happens with the replacement level pitcher taking the closer’s role.

Reliever ERA IP LI RAR
Closer 2.50 0 0.0 0
Setup 1 3.00 65 1.4 20
Setup 2 3.75 65 1.2 11
Middle Relief 1 4.00 65 1.0 7
Middle Relief 2 4.25 65 0.9 5
Middle Relief 3 4.50 65 0.7 3
Long Man 1 4.75 55 0.5 1
Long Man 2 4.75 50 0.5 1
Replacement 5.00 70 1.7 0
Total: 4.23 500 1.02 47

Exactly 33 runs difference.  So Joe Nathan, for example, measured against a replacement level closer, is worth 33 runs, or roughly 3 wins.  But due to bullpen chaining, the team could do better by replacing the closer with their second best pitcher, and so on:

Reliever ERA IP LI RAR
Closer 2.50 0 0.0 0
Setup 1 3.00 65 1.7 25
Setup 2 3.75 65 1.4 13
Middle Relief 1 4.00 65 1.2 9
Middle Relief 2 4.25 65 1.0 5
Middle Relief 3 4.50 65 0.9 3
Long Man 1 4.75 60 0.7 1
Long Man 2 4.75 60 0.5 1
Replacement 5.00 55 0.5 0
Total: 4.23 500 1.01 57

So the closer might be worth 3 wins, but the team would only lose 2 by re-structuring their bullpen.  However, the effect is probably even less significant.  More than likely, they’ll try to get someone from outside the organization to take one of the setup roles.  Let’s say they find another “Setup 2.”  Here’s what it looks like:

Reliever ERA IP LI RAR
Closer 2.50 0 0.0 0
Setup 1 3.00 65 1.7 25
Setup 2 3.75 65 1.4 13
Middle Relief 1 4.00 65 1.0 7
Middle Relief 2 4.25 65 0.9 5
Middle Relief 3 4.50 60 0.7 2
Long Man 1 4.75 60 0.5 1
Long Man 2 4.75 55 0.5 1
Setup 2.2 3.75 65 1.2 11
Total: 4.07 500 1.01 64

And all of a sudden we’re down to roughly 1.5 wins.  Based on win values evaluated during free agency, teams have paid between 4 and 4.5 million dollars per win over the last several years.  So you could pay your closer 12 or 13 million dollars (as K-Rod, Kerry Wood, and others have gotten in recent years), as his RAR would suggest, or you could pay a bullpen full of 3.50 ERA guys and fare better.

Reliever ERA IP LI RAR
Closer 3.50 70 1.7 20
Setup 1 3.50 65 1.4 15
Setup 2 3.50 65 1.2 13
Middle Relief 1 3.50 65 1.0 11
Middle Relief 2 3.50 65 0.9 10
Middle Relief 3 3.50 65 0.7 8
Long Man 1 3.50 55 0.5 5
Long Man 2 3.50 50 0.5 4
Replacement 5.00 0 0.0 0
Total: 3.50 500 1.02 85

A replacement-level team would win something like 40 games.  Since a team needs to win nearly 90 to get to the playoffs, 50 wins above replacement are needed from the rest of the roster.  Even with a good bullpen, I’ve shown that a team is looking at 8 or 9 at the most.  Bad bullpens would show 3 or 4 at the worst.  I think allocating anymore than 20 to 25 percent of the payroll to the bullpen is a bad idea, particularly if one guy is making 60 to 80 percent of that.  Obviously paying anything close to market value for wins when you aren’t a playoff-caliber team is foolish, and that money would be better allocated for the draft or international signings.  But for a team that needs each win, trying to scrape together every extra win they can find, paying the marginal value for a win might be  better allocated in the rotation or the lineup.

Advertisements

Written by Dan Hennessey

March 18, 2010 at 10:07 PM

Posted in Uncategorized

2 Responses

Subscribe to comments with RSS.

  1. Have you not considered the effect a closer like me has on a team’s fanbase? I single-handedly caused 17 heart attacks in Indians fans in ’05. And I was GOOD. You gotta imagine what a Replacement Level Pitcher would do with an entire season. Killing off that much of the fanbase could affect the team affording unlimited post-game spreads in the clubhouse!!

    Bob Wickman

    March 23, 2010 at 11:39 PM

  2. […] On Monday, I wrote an article on ESPN about the depth of the A’s bullpen.  Last spring, after Joe Nathan’s season-ending injury, I wrote a post about how much that would cost the Twins, and more generally, how much bullpens really matter to a team*. […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: