Wednesday, October 29, 2008

So How Much Is Dave Duncan Worth, Anyway?

The other day, after the Nationals announced a boatload of new coach hirings, I did a post contending that coaches in Major League Baseball don't matter very much. It seems to me that coaches are mostly motivators, basically staff directors keeping their guys in line but not really affecting their performance on the field very much. By the time players reach this level of baseball, they've usually received a decade of coaching, and their talent determines the lion's share of their success from there. My guess was that the difference between a great hitting coach, for instance, and a terrible one, would amount to less than a single win over the course of a full season.

But I didn't have any real evidence to back this up. I've read studies showing that show that the cumulative effect of all a manager's decisions over the course of a season amount to only a handful of wins one way or the other, but I've never seen any such study on the effect of coaches.

There were a number of comments poo-pooing my post, which you can read if you're interested. Those comments inspired me to at least test my hypothesis by attempting to measure the impact of one given pitching coach. I chose Dave Duncan, the St. Louis Cardinals pitching coach who is almost universally recognized as one of the very best in the game.

My methodology is pretty simple: I took all the pitchers who have thrown at least 40 innings both with and without Duncan and added up their innings pitched and earned runs with and without Duncan. My gut tells me that of all the coaches on a team, the pitching coach probably has the biggest impact on his players' performances. And knowing what a great pitching coach Duncan is renowned to be, I figured that if coaches have any measurable impact at all, then that would have to show up in Duncan's quarter-century of coaching.

(Of course, this methodology is a little bogus. There are many different variables outside the pitching coach's control that can cause a pitcher to do better or worse in any given season. If I was any good at math, I'd do one of those fancy Nate Silver-style regression analyses to isolate the influence of the pitching coach from other factors like aging, park effects, league adjustment, etc. But alas, I'm not that good at math, so this is the best I could do. My assumption is that with this much data all these other factors will hopefully come out in the wash.)

Here's what I found: since 1982, when Duncan debuted as a pitching coach in Seattle, 140 pitchers met both of my "40 innings with Duncan" and "40 innings without Duncan" thresholds. All told, these 140 pitchers threw 34,145 1/3 innings with a 4.02 ERA with Duncan. Without him, they threw 163,053 innings with a 3.95 ERA.

Yep, you read that right. Pitchers have actually done worse with Duncan than without him. And the difference is pretty big--a 0.07 difference in ERA that projects out to about 11 additional runs allowed, or little more than one additional loss for a typical team. How can this be? It just doesn't pass the smell test that Duncan is actually worse than the average pitching coach.

Looking at the data closer, there are actually a few extreme cases that are skewing the data against him. Early in his career, Duncan had an odd habit of coaching over the hill Hall of Fame pitchers for one over-40 season. He had Gaylord Perry, Steve Carlton, Jerry Koosman, and Tom Seaver all for one or two seasons long past their primes. It's clearly not fair to hold it against Duncan that he couldn't get much out of a 41-year-old Steve Carlton.

So I took those guys out out of the equation and that tilts the balance back towards Duncan; the remaining 138 pitchers posted a cumulative 4.02 ERA with Duncan against a 4.09 ERA without him. Again, that 0.07 difference in ERA over the course of a full season projects to about 11 runs prevented. Measured this way, the consensus best pitching coach in all of baseball is worth about one additional win per year--not insignificant certainly, but not enormous.

However, looked at another way, this data seems to suggest that Duncan may be having an even bigger impact than that. If you simply count up the number of pitchers who posted a better ERA with Duncan than without him, you'll see that 79 did better with him, while just 61 did better without him--a pretty significant spread.

Still, this isn't conclusive evidence one way or another. This could all just be evidence that the A's, Cardinals, White Sox, and Mariners did a good job evaluating pitchers and dumping them or acquiring them at the right times. Oakland Colesium and both Busch Stadiums are all pitchers' parks. There's a lot of statistical noise here. But based on this little sample, I don't really see anything to disprove my initial feeling: coaches, even the very best of the best, don't matter all that much.

Here's the full list, if you're interested. (I wouldn't bet my life there are zero mistakes in here, so if you find one, let me know and I'll update the numbers.)


IP w/DD ER w/DD ERA w/DD IP w/o DD ER w/o DD ERA w/o DD
Gaylord Perry 216.7 106 4.40 5133.6 1740 3.05
Floyd Bannister 1058.3 471 4.01 1329.7 607 4.11
Mike Moore 1018.3 430 3.80 1813.3 951 4.72
Jim Beattie 172.3 64 3.34 976.4 468 4.31
Gene Nelson 122.7 63 4.62 957.3 433 4.07
Bob Stoddard 67.3 18 2.41 365.7 176 4.33
Edwin Nunez 35.3 18 4.59 617.0 286 4.17
Bill Caudill 95.7 25 2.35 571.3 248 3.91
Ed Vande Berg 76.0 20 2.37 443.0 206 4.19
Mike Stanton 83.0 45 4.88 301.3 152 4.54
Larry Andersen 79.7 53 5.98 915.6 295 2.90
Bryan Clark 122.7 39 2.86 393.6 199 4.55
La Marr Hoyt 496.3 223 4.04 815.0 359 3.96
Richard Dotson 735.0 330 4.04 1122.3 542 4.35
Britt Burns 517.7 234 4.07 576.7 211 3.29
Jerry Koosman 169.7 90 4.77 3669.7 1343 3.29
Dennis Lamp 173.0 80 4.16 1657.7 719 3.90
Salome Barojas 126.7 44 3.13 263.3 127 4.34
Dick Tidrow 91.7 43 4.22 1655.0 672 3.65
Juan Agosto 162.0 66 3.67 464.3 213 4.13
Tom Seaver 547.3 223 3.67 4235.3 1298 2.76
Ron Reed 73.0 25 3.08 2404.7 928 3.47
Dan Spillner 140.0 57 3.66 1352.7 642 4.27
Bert Roberge 40.7 17 3.76 178.7 80 4.03
Jerry Don Gleaton 52.7 33 5.64 394.6 178 4.06
Tim Lollar 83.0 43 4.66 823.0 387 4.23
Joel Davis 176.7 88 4.48 71.0 47 5.96
Bob James 168.3 60 3.21 238.7 112 4.22
Joe Cowley 162.3 70 3.88 307.0 149 4.37
Neil Allen 113.0 48 3.82 875.3 378 3.89
Jose DeLeon 79.0 26 2.96 1818.3 767 3.80
Steve Carlton 63.3 26 3.69 5154.0 1838 3.21
Dave Schmidt 92.3 34 3.31 809.7 355 3.95
Bill Dawley 106.7 40 3.38 364.0 139 3.44
Curt Young 875.3 398 4.09 231.7 132 5.13
Joaquin Andujar 216.0 107 4.46 1937.0 750 3.48
Jose Rijo 339.7 179 4.74 1540.4 497 2.90
Dave Stewart 1617.3 712 3.96 1012.4 442 3.93
Chris Codiroli 103.0 52 4.54 567.3 311 4.93
Eric Plunk 322.0 154 4.30 829.0 334 3.63
Moose Haas 113.0 48 3.82 1542.0 690 4.03
Rick Langford 55.0 45 7.36 1436.0 619 3.88
Jay Howell 195.7 80 3.68 649.0 233 3.23
Steve Ontiveros 523.0 230 3.96 138.7 40 2.60
Dave Lieper 125.3 52 3.73 152.7 71 4.18
Doug Bair 45.0 15 3.00 864.3 352 3.67
Bill Krueger 40.0 29 6.52 1157.3 548 4.26
Storm Davis 464.0 219 4.25 1316.7 577 3.94
Dennis Eckersley 750.0 239 2.87 2535.7 1039 3.69
Greg Cadaret 139.0 50 3.24 585.4 271 4.17
Rick Honeycutt 455.6 158 3.12 1704.4 735 3.88
Bob Welch 1270.6 556 3.94 1821.4 635 3.14
Todd Burns 291.0 91 2.81 198.7 98 4.44
Jim Corsi 148.7 35 2.12 332.7 139 3.76
Scott Sanderson 206.3 89 3.88 2355.3 1004 3.84
Reggie Harris 44.3 20 4.06 76.7 46 5.40
Joe Klink 101.7 39 3.45 63.0 39 5.57
Joe Slusarski 194.0 115 5.33 111.6 61 4.92
Andy Hawkins 77.0 41 4.79 1481.3 690 4.19
Ron Darling 723.3 372 4.63 1637.0 644 3.54
Eric Show 51.7 34 5.92 1603.3 639 3.59
Todd Van Poppel 406.7 260 5.75 500.3 302 5.43
Kelly Downs 201.7 105 4.69 762.0 308 3.64
Jeff Parrett 98.3 33 3.02 626.3 273 3.92
Shawn Hillegas 68.4 49 6.45 447.0 215 4.33
Bobby Witt 387.3 191 4.44 2077.7 1133 4.91
Rich Gossage 85.7 36 3.78 1723.6 569 2.97
Steve Karsay 209.0 115 4.95 394.3 154 3.51
Joe Boever 79.3 34 3.86 675.0 295 3.93
Mike Mohler 158.7 93 5.28 258.4 138 4.81
Billy Taylor 46.3 18 3.50 278.3 134 4.33
Mark Acre 86.3 46 4.80 40.7 27 5.97
Carlos Reyes 147.0 75 4.59 411.0 214 4.69
Todd Stottlemeyer 775.3 343 3.98 1417.3 699 4.44
Mike Harkey 66.0 46 6.27 590.0 281 4.29
Doug Johns 54.7 28 4.61 331.3 192 5.22
Ariel Prieto 58.0 32 4.97 294.3 158 4.83
Steve Wojciechowski 48.7 28 5.18 90.0 59 5.90
Andy Benes 778.7 371 4.29 1726.7 735 3.83
Alan Benes 415.3 197 4.27 78.7 55 6.29
Donovan Osborne 392.0 178 4.09 481.7 213 3.98
Mike Morgan 103.0 60 5.24 2669.3 1243 4.19
T.J. Mathews 144.3 44 2.74 291.0 141 4.36
Tony Fossas 135.3 42 2.79 280.3 138 4.43
Cory Bailey 57.0 19 3.00 150.0 72 4.32
Mike Petkovsek 184.7 89 4.34 525.3 284 4.87
Danny Jackson 55.0 34 5.56 2017.7 889 3.97
Matt Morris 1377.3 552 3.61 428.7 247 5.19
Manuel Aybar 246.3 153 5.59 144.7 69 4.29
John Frascatore 175.7 66 3.38 195.3 99 4.56
Rigo Beltran 54.3 21 3.48 52.0 31 5.37
Lance Painter 145.7 75 4.63 304.3 187 5.53
Kent Mercker 266.3 150 5.07 1059.0 462 3.93
Juan Acevedo 200.7 95 4.26 369.3 179 4.36
Jeff Brantley 50.7 25 4.44 808.7 299 3.33
Kent Bottenfield 324.0 150 4.17 587.7 310 4.75
Jose Jimenez 184.3 113 5.52 337.0 172 4.59
Braden Looper 450.6 224 4.47 530.7 205 3.48
Garrett Stephenson 505.0 256 4.56 146.3 73 4.49
Larry Leubbers 45.7 26 5.12 97.7 53 4.88
Mark Thompson 54.3 23 3.81 282.7 192 6.11
Ricky Bottalico 73.3 40 4.91 555.3 239 3.87
Heathcliff Slocumb 103.0 44 3.85 528.0 242 4.12
Darryl Kile 544.3 214 3.54 1621.0 778 4.32
Pat Hentgen 194.3 102 4.72 1881.0 894 4.28
Britt Reames 40.7 13 2.88 177.3 115 5.84
Dave Veres 224.0 83 3.34 470.0 182 3.48
Mike James 89.3 40 4.03 224.7 88 3.53
Mike Timlin 163.3 61 3.36 1041.0 425 3.67
Mike Matthews 140.0 62 3.99 103.7 61 5.30
Luther Hackman 119.3 57 4.30 92.7 63 6.12
Dustin Hermanson 222.0 113 4.58 1061.0 487 4.13
Woody Williams 588.7 231 3.53 1627.7 800 4.42
Steve Kline 247.3 74 2.69 435.0 192 3.97
Chuck Finley 85.3 36 3.80 3112.0 1330 3.85
Travis Smith 54.0 43 7.17 57.7 38 5.93
Jason Isringhausen 408.0 135 2.98 499.3 229 4.13
Jeff Fassero 95.7 55 5.17 1938.0 874 4.06
Brett Tomko 202.7 119 5.28 1538.3 787 4.60
Danny Haren 118.7 64 4.85 878.7 348 3.56
Cal Eldred 171.3 65 3.41 1196.7 607 4.57
Esteban Yan 43.3 29 6.02 652.0 368 5.08
Kiko Calero 83.7 26 2.80 159.0 70 3.96
Russ Springer 133.7 45 3.03 664.0 358 4.85
Jason Marquis 602.7 308 4.60 666.3 334 4.51
Jeff Suppan 572.0 251 3.95 1677.0 907 4.87
Chris Carpenter 666.7 230 3.11 870.7 467 4.83
Ray King 102.0 33 2.91 309.0 125 3.64
Julian Tavarez 130.0 42 2.91 1239.3 635 4.61
Jason Simontacchi 285.0 151 4.77 70.7 50 6.37
Al Reyes 74.7 16 1.93 354.0 166 4.22
Sidney Ponson 68.7 40 5.24 1633.0 896 4.94
Josh Hancock 89.7 40 4.02 88.0 43 4.40
Kip Wells 162.7 103 5.70 1065.7 535 4.52
Todd Wellemeyer 255.3 101 3.56 178.3 112 5.65
Joel Pineiro 212.3 113 4.79 1030.0 515 4.50
Ryan Franklin 158.7 58 3.29 888.7 430 4.35
Troy Percival 40.0 8 1.80 657.3 234 3.20
Kyle Lohse 200.0 84 3.78 1164.0 624 4.82
Ron Villone 50.0 26 4.68 1069.3 565 4.76

3 comments:

Steve Shoup said...

First Steven i got to say WOW thats a lot of work you did just to prove a point. While I agree with you on the premise that coaches don't always matter in the W/L column I think coaches matter more than you give them credit for. Their impact can't always be measured by a Team standard ( a team's batting average or team ERA) b/c its up to the idividual player to grasp and implement the new mechanics or pitches. For instance some of the pitchers that Duncan had when they were just coming up who had long successful careers maybe learned a new pitch or new pitch mechanics from Duncan and while it might not have showed up at the time helped that pitcher throughout his career. Maybe Duncan impacted Dan Haren and he is the pitcher he is today b/c of him or maybe not. I think Dave Duncan is a good pitching coach but I just don't think any math could really quantify it. For instance I feel St. Claire is a good pitching coach as well even though the Nats have had some pretty bad pitching numbers for the last 2 years. But given the fact that he has gotten the most out of Traber, Bacsik, Bowie, and Redding and accelerated the development of John Lannan I feel comfortable in saying he's a good coach.

Steven said...

I wasn't really trying to prove a point so much as actually look at data in a way I've never seen.

And you're right it's possible that Danny Haren learned something under Duncan that helped him later and who knows.

I still think though that if you believe that coaches can generate 10+ more wins as some folks have specualted, that this THAT level of impact would have to be more apparent in the numbers.

JayB said...

Steven,

Thanks for the efforts but it seems we are no where further than we were before your efforts.

No good study can be done due to the lack of controls. Your work has all the flaws you point out and many many others.....Pitches who left Duncan in STL and moved to AL where ERA's are higher? Pitchers who left Duncan in Oakland to go to NL where ERA's are lower......Just too many problems with the data you put out.......I think what we do know from our own experiences in our own careers is that leadership matters and so do personal relationships in focusing our efforts and motivation.

Like I said before, I say 10 wins, you say less than one win.......We will never know......oh and my 10 wins was a 2008 Nationals specific estimate......not the same for a vet team like the Yanks or Red Sox.