Wednesday, December 31, 2008

Nationals 2008 Stolen Bases Win Probability Added

A while ago I wrote a post taking Manny Acta to task for the Nationals' ineffectiveness in the running game. The Nationals ranked second-to-last in MLB with a SB success rate of just 65.3% (81 SB, 43 CS).

In that post, I wrote the following:
To be helping the team, the stolen base success rate needs to be around at least 73%. That's because the run value (the average runs added as a result of a particular event) of a stolen base is 0.175, while the run value of a caught stealing in -0.467. (These run value rates come from The Book: Playing the Percentages in Baseball by Tom Tango, Mitchel Lichtman, and Andy Dolphin and calculated by tallying up the results of a massive amount of play-by-play data from the history of MLB.)
To be precise, the "break-even point" for stolen bases using this context neutral run value metric is 72.7%, and the bottom line for the 2008 Nationals using this baseline is that the the Nationals' poor running game took a net 5.9 runs off the board, which translates into a bit more than half a win.

However in that post I included the following important caveat:
Like sac bunts, the value of a stolen base is highly context dependent, so you can't just assume that you're hurting the team if your cumulative run value is below zero. For instance, if you have 3 SB and 2 CS, you'll be in the negative, but if those stolen bases were all in tie games in the ninth inning and the caught stealings were in blowout losses, then you're ahead in terms of win value. But setting that aside, if you're anywhere below 70% you're clearly taking runs off the board.
I decided I would actually crunch these numbers for the Nationals. The question is, factoring in the context of when we ran, did the Nationals actually cost themselves wins in the running game in 2008, and if so how many?

What I did was to look at all 124 stolen base attempts in 2008, calculate the win expectancy (WE) of the base-out-score situation (i.e., the likelihood that the team would win given the score, number of outs, and men on base) at the start of the play as well as the WE that would be created by both the successful steal and caught stealing. Once I had those numbers, I was able to calculate the break-even point for stealing in that situation based on win value rather than run value. Again, all this is based on work by Tango-Lichtman-Dolphin.

You can review all these numbers on Google Spreadsheets by clicking here, or if you want an editable version of the spreadsheet email me and I'll send it along. If you're interested in repeating this analysis for your own team, the formula for calculating break-even point is (WE loss from CS / WE gain from SB) / ((WE loss from CS / WE gain from SB) - 1). All the other numbers come from the win expectancy tables in chapter 1 of The Book.

Here's what these numbers show us. The revised break-even point for the Nationals based on in-context win expectancy in 2008 was 70.6%. That compares to the aforementioned 72.7% break-even point based on context-independent run values. What that means is that Manny seems to be doing a pretty good job picking his opportunities to steal, situations when the WE bang for the buck is in the Nationals' favor. I can't tell you whether other managers do an even better job of "picking spots," but it seems to me intuitively that for something like this that a 2.1% difference is pretty big.

You might think that since the Nationals' 65.3% success rate is closer to the revised in-context break-even point of 70.6%, that we cost ourselves somewhat less than half a win. You'd be wrong. In fact, the net win probability added (WPA) of all our steal attempts actually cost us .798 wins. That's because we did better when it was less important and got caught more often when it was particularly harmful to be giving away outs on the bases. So much for optimizing our production.

Now that I had all this info together, I thought it would be fun to see which players were smartest about "picking their spots." In other words, which players were smartest about running when the "break-even point" is lowest. Of course, this assumes the player is the one deciding whether to run or not, which probably isn't the case except with maybe one or two guys if anyone, but nonetheless here are the break-even points for our most frequent stealers:
Dukes: 74.9%
Milledge: 71.4%
Bonifacio: 70.2%
Harris: 70.3%
Kearns: 68.6%
Guzman: 68.6%
Lopez: 65.9%
Bernadina: 64.3%
Pena: 63.4%
And finally, our base-stealers by cumulative WPA:
Dukes: +.07
Harris: +.042
Bernadina: -0.019
Milledge: -0.027
Bonifacio: -0.035
Pena: -0.052
Guzman: -0.164
Kearns: -0.171
Lopez: -0.255
(I should note that because the WE chart I'm working from in The Book only includes base-out-score situations up to four-run leads a few of these calculations are off a bit. Specifically, in five of these situations, the Nationals were trailing or leading by more than four, and in these cases I just calculated the win probability added as if the difference was four. If anyone has the math to correct this, lemme know. Also, some of the successful steals may have been better scored as defensive indifference. Finally, it's also possible that some of these stolen base attempts would have been better recorded as hit-and-run attempts, but MLB's play-by-play data doesn't distinguish between hit-and-run plays and stolen base attempts. The impact of these errors is pretty minimal, but worth noting.)


Thomas said...

Looking at your last ranking, it would pretty closely track the way most fans would list the Nats with the best to least baseball "smarts": Dukes on top, Lopez on the bottom.


Steven said...

You know I thought about that too, but actually, I would probably say that looking at average "break even point" is a better way to gauge baseball smarts. Again with the caveat that Manny is probably the one making the decision when to steal and when not to steal 90% of the time or more, those average break even points (the next-to-last ranking) tell you who's doing the best job picking spots to run when the bang-for-buck ratio is most in the team's favor. In other words it's a lot smarter to run with two outs, tie score, bottom of the ninth and a man on first than it is to run down 2 in the same situation.

This list tells you that Dukes is actually the least smart, running with a cumulative break-even point of 74.9%. Lopez is on the smart end at 65.9%. Meaning Lopez was more often the guy running with two outs in the bottom of the ninth of a tie game and Dukes was more often the guy running while down 2.

Lopez is at the bottom on total WPA because he got caught more than Dukes did. And that's definitely also somewhat a measure of baseball smarts--even if the steal sign is on you have to decide whether you have a good enough jump to go. But I would say that his SB% is more a measure of just how good he is on the basepaths--his speed, the quality of his jumps, etc. In other words it's more physical than intellectual.

So where does that leave us? Well, Dukes was clearly the superior baserunner overall, and in fact the best on the team. And if that makes him seem smarter, then fans aren't wrong. But smarts and physical skill both play big roles, and in Dukes's case it looks like it's more physical skill than smarts at least in this part of his game.

Dave Nichols said...

excellent analytical work Steven. thank you.

Steven said...

Thanks. You know, I was just re-looking at those numbers, and I gotta say it's kind of amazing that Felipe Lopez managed to cost us a quarter of a win just from caught stealings. Wow. Now there's a guy going out of his way to hurt his team.

Jeff said...

The 2008 run matrix has the SB the break even number at 71.4%:

I like the research and am looking into a manager grading system and will probably implement a system similar to what you did here.

Steven said...

Wow thanks. I didn't know BP maintained and yearly matrix. Thanks!

Steven said...

BTW--do you know if there's a place where Tango's WE table are updated? I've looked a bunch but haven't found it.

Jeff said...

No, I did a quick search with no luck, you could just ask him.