But I didn't have any real evidence to back this up. I've read studies showing that show that the cumulative effect of all a manager's decisions over the course of a season amount to only a handful of wins one way or the other, but I've never seen any such study on the effect of coaches.
There were a number of comments poo-pooing my post, which you can read if you're interested. Those comments inspired me to at least test my hypothesis by attempting to measure the impact of one given pitching coach. I chose Dave Duncan, the St. Louis Cardinals pitching coach who is almost universally recognized as one of the very best in the game.
My methodology is pretty simple: I took all the pitchers who have thrown at least 40 innings both with and without Duncan and added up their innings pitched and earned runs with and without Duncan. My gut tells me that of all the coaches on a team, the pitching coach probably has the biggest impact on his players' performances. And knowing what a great pitching coach Duncan is renowned to be, I figured that if coaches have any measurable impact at all, then that would have to show up in Duncan's quarter-century of coaching.
(Of course, this methodology is a little bogus. There are many different variables outside the pitching coach's control that can cause a pitcher to do better or worse in any given season. If I was any good at math, I'd do one of those fancy Nate Silver-style regression analyses to isolate the influence of the pitching coach from other factors like aging, park effects, league adjustment, etc. But alas, I'm not that good at math, so this is the best I could do. My assumption is that with this much data all these other factors will hopefully come out in the wash.)
Here's what I found: since 1982, when Duncan debuted as a pitching coach in Seattle, 140 pitchers met both of my "40 innings with Duncan" and "40 innings without Duncan" thresholds. All told, these 140 pitchers threw 34,145 1/3 innings with a 4.02 ERA with Duncan. Without him, they threw 163,053 innings with a 3.95 ERA.
Yep, you read that right. Pitchers have actually done worse with Duncan than without him. And the difference is pretty big--a 0.07 difference in ERA that projects out to about 11 additional runs allowed, or little more than one additional loss for a typical team. How can this be? It just doesn't pass the smell test that Duncan is actually worse than the average pitching coach.
Looking at the data closer, there are actually a few extreme cases that are skewing the data against him. Early in his career, Duncan had an odd habit of coaching over the hill Hall of Fame pitchers for one over-40 season. He had Gaylord Perry, Steve Carlton, Jerry Koosman, and Tom Seaver all for one or two seasons long past their primes. It's clearly not fair to hold it against Duncan that he couldn't get much out of a 41-year-old Steve Carlton.
So I took those guys out out of the equation and that tilts the balance back towards Duncan; the remaining 138 pitchers posted a cumulative 4.02 ERA with Duncan against a 4.09 ERA without him. Again, that 0.07 difference in ERA over the course of a full season projects to about 11 runs prevented. Measured this way, the consensus best pitching coach in all of baseball is worth about one additional win per year--not insignificant certainly, but not enormous.
However, looked at another way, this data seems to suggest that Duncan may be having an even bigger impact than that. If you simply count up the number of pitchers who posted a better ERA with Duncan than without him, you'll see that 79 did better with him, while just 61 did better without him--a pretty significant spread.
Still, this isn't conclusive evidence one way or another. This could all just be evidence that the A's, Cardinals, White Sox, and Mariners did a good job evaluating pitchers and dumping them or acquiring them at the right times. Oakland Colesium and both Busch Stadiums are all pitchers' parks. There's a lot of statistical noise here. But based on this little sample, I don't really see anything to disprove my initial feeling: coaches, even the very best of the best, don't matter all that much.
Here's the full list, if you're interested. (I wouldn't bet my life there are zero mistakes in here, so if you find one, let me know and I'll update the numbers.)
| IP w/DD | ER w/DD | ERA w/DD | IP w/o DD | ER w/o DD | ERA w/o DD | |
| Gaylord Perry | 216.7 | 106 | 4.40 | 5133.6 | 1740 | 3.05 |
| Floyd Bannister | 1058.3 | 471 | 4.01 | 1329.7 | 607 | 4.11 |
| Mike Moore | 1018.3 | 430 | 3.80 | 1813.3 | 951 | 4.72 |
| Jim Beattie | 172.3 | 64 | 3.34 | 976.4 | 468 | 4.31 |
| Gene Nelson | 122.7 | 63 | 4.62 | 957.3 | 433 | 4.07 |
| Bob Stoddard | 67.3 | 18 | 2.41 | 365.7 | 176 | 4.33 |
| Edwin Nunez | 35.3 | 18 | 4.59 | 617.0 | 286 | 4.17 |
| Bill Caudill | 95.7 | 25 | 2.35 | 571.3 | 248 | 3.91 |
| Ed Vande Berg | 76.0 | 20 | 2.37 | 443.0 | 206 | 4.19 |
| Mike Stanton | 83.0 | 45 | 4.88 | 301.3 | 152 | 4.54 |
| Larry Andersen | 79.7 | 53 | 5.98 | 915.6 | 295 | 2.90 |
| Bryan Clark | 122.7 | 39 | 2.86 | 393.6 | 199 | 4.55 |
| La Marr Hoyt | 496.3 | 223 | 4.04 | 815.0 | 359 | 3.96 |
| Richard Dotson | 735.0 | 330 | 4.04 | 1122.3 | 542 | 4.35 |
| Britt Burns | 517.7 | 234 | 4.07 | 576.7 | 211 | 3.29 |
| Jerry Koosman | 169.7 | 90 | 4.77 | 3669.7 | 1343 | 3.29 |
| Dennis Lamp | 173.0 | 80 | 4.16 | 1657.7 | 719 | 3.90 |
| Salome Barojas | 126.7 | 44 | 3.13 | 263.3 | 127 | 4.34 |
| Dick Tidrow | 91.7 | 43 | 4.22 | 1655.0 | 672 | 3.65 |
| Juan Agosto | 162.0 | 66 | 3.67 | 464.3 | 213 | 4.13 |
| Tom Seaver | 547.3 | 223 | 3.67 | 4235.3 | 1298 | 2.76 |
| Ron Reed | 73.0 | 25 | 3.08 | 2404.7 | 928 | 3.47 |
| Dan Spillner | 140.0 | 57 | 3.66 | 1352.7 | 642 | 4.27 |
| Bert Roberge | 40.7 | 17 | 3.76 | 178.7 | 80 | 4.03 |
| Jerry Don Gleaton | 52.7 | 33 | 5.64 | 394.6 | 178 | 4.06 |
| Tim Lollar | 83.0 | 43 | 4.66 | 823.0 | 387 | 4.23 |
| Joel Davis | 176.7 | 88 | 4.48 | 71.0 | 47 | 5.96 |
| Bob James | 168.3 | 60 | 3.21 | 238.7 | 112 | 4.22 |
| Joe Cowley | 162.3 | 70 | 3.88 | 307.0 | 149 | 4.37 |
| Neil Allen | 113.0 | 48 | 3.82 | 875.3 | 378 | 3.89 |
| Jose DeLeon | 79.0 | 26 | 2.96 | 1818.3 | 767 | 3.80 |
| Steve Carlton | 63.3 | 26 | 3.69 | 5154.0 | 1838 | 3.21 |
| Dave Schmidt | 92.3 | 34 | 3.31 | 809.7 | 355 | 3.95 |
| Bill Dawley | 106.7 | 40 | 3.38 | 364.0 | 139 | 3.44 |
| Curt Young | 875.3 | 398 | 4.09 | 231.7 | 132 | 5.13 |
| Joaquin Andujar | 216.0 | 107 | 4.46 | 1937.0 | 750 | 3.48 |
| Jose Rijo | 339.7 | 179 | 4.74 | 1540.4 | 497 | 2.90 |
| Dave Stewart | 1617.3 | 712 | 3.96 | 1012.4 | 442 | 3.93 |
| Chris Codiroli | 103.0 | 52 | 4.54 | 567.3 | 311 | 4.93 |
| Eric Plunk | 322.0 | 154 | 4.30 | 829.0 | 334 | 3.63 |
| Moose Haas | 113.0 | 48 | 3.82 | 1542.0 | 690 | 4.03 |
| Rick Langford | 55.0 | 45 | 7.36 | 1436.0 | 619 | 3.88 |
| Jay Howell | 195.7 | 80 | 3.68 | 649.0 | 233 | 3.23 |
| Steve Ontiveros | 523.0 | 230 | 3.96 | 138.7 | 40 | 2.60 |
| Dave Lieper | 125.3 | 52 | 3.73 | 152.7 | 71 | 4.18 |
| Doug Bair | 45.0 | 15 | 3.00 | 864.3 | 352 | 3.67 |
| Bill Krueger | 40.0 | 29 | 6.52 | 1157.3 | 548 | 4.26 |
| Storm Davis | 464.0 | 219 | 4.25 | 1316.7 | 577 | 3.94 |
| Dennis Eckersley | 750.0 | 239 | 2.87 | 2535.7 | 1039 | 3.69 |
| Greg Cadaret | 139.0 | 50 | 3.24 | 585.4 | 271 | 4.17 |
| Rick Honeycutt | 455.6 | 158 | 3.12 | 1704.4 | 735 | 3.88 |
| Bob Welch | 1270.6 | 556 | 3.94 | 1821.4 | 635 | 3.14 |
| Todd Burns | 291.0 | 91 | 2.81 | 198.7 | 98 | 4.44 |
| Jim Corsi | 148.7 | 35 | 2.12 | 332.7 | 139 | 3.76 |
| Scott Sanderson | 206.3 | 89 | 3.88 | 2355.3 | 1004 | 3.84 |
| Reggie Harris | 44.3 | 20 | 4.06 | 76.7 | 46 | 5.40 |
| Joe Klink | 101.7 | 39 | 3.45 | 63.0 | 39 | 5.57 |
| Joe Slusarski | 194.0 | 115 | 5.33 | 111.6 | 61 | 4.92 |
| Andy Hawkins | 77.0 | 41 | 4.79 | 1481.3 | 690 | 4.19 |
| Ron Darling | 723.3 | 372 | 4.63 | 1637.0 | 644 | 3.54 |
| Eric Show | 51.7 | 34 | 5.92 | 1603.3 | 639 | 3.59 |
| Todd Van Poppel | 406.7 | 260 | 5.75 | 500.3 | 302 | 5.43 |
| Kelly Downs | 201.7 | 105 | 4.69 | 762.0 | 308 | 3.64 |
| Jeff Parrett | 98.3 | 33 | 3.02 | 626.3 | 273 | 3.92 |
| Shawn Hillegas | 68.4 | 49 | 6.45 | 447.0 | 215 | 4.33 |
| Bobby Witt | 387.3 | 191 | 4.44 | 2077.7 | 1133 | 4.91 |
| Rich Gossage | 85.7 | 36 | 3.78 | 1723.6 | 569 | 2.97 |
| Steve Karsay | 209.0 | 115 | 4.95 | 394.3 | 154 | 3.51 |
| Joe Boever | 79.3 | 34 | 3.86 | 675.0 | 295 | 3.93 |
| Mike Mohler | 158.7 | 93 | 5.28 | 258.4 | 138 | 4.81 |
| Billy Taylor | 46.3 | 18 | 3.50 | 278.3 | 134 | 4.33 |
| Mark Acre | 86.3 | 46 | 4.80 | 40.7 | 27 | 5.97 |
| Carlos Reyes | 147.0 | 75 | 4.59 | 411.0 | 214 | 4.69 |
| Todd Stottlemeyer | 775.3 | 343 | 3.98 | 1417.3 | 699 | 4.44 |
| Mike Harkey | 66.0 | 46 | 6.27 | 590.0 | 281 | 4.29 |
| Doug Johns | 54.7 | 28 | 4.61 | 331.3 | 192 | 5.22 |
| Ariel Prieto | 58.0 | 32 | 4.97 | 294.3 | 158 | 4.83 |
| Steve Wojciechowski | 48.7 | 28 | 5.18 | 90.0 | 59 | 5.90 |
| Andy Benes | 778.7 | 371 | 4.29 | 1726.7 | 735 | 3.83 |
| Alan Benes | 415.3 | 197 | 4.27 | 78.7 | 55 | 6.29 |
| Donovan Osborne | 392.0 | 178 | 4.09 | 481.7 | 213 | 3.98 |
| Mike Morgan | 103.0 | 60 | 5.24 | 2669.3 | 1243 | 4.19 |
| T.J. Mathews | 144.3 | 44 | 2.74 | 291.0 | 141 | 4.36 |
| Tony Fossas | 135.3 | 42 | 2.79 | 280.3 | 138 | 4.43 |
| Cory Bailey | 57.0 | 19 | 3.00 | 150.0 | 72 | 4.32 |
| Mike Petkovsek | 184.7 | 89 | 4.34 | 525.3 | 284 | 4.87 |
| Danny Jackson | 55.0 | 34 | 5.56 | 2017.7 | 889 | 3.97 |
| Matt Morris | 1377.3 | 552 | 3.61 | 428.7 | 247 | 5.19 |
| Manuel Aybar | 246.3 | 153 | 5.59 | 144.7 | 69 | 4.29 |
| John Frascatore | 175.7 | 66 | 3.38 | 195.3 | 99 | 4.56 |
| Rigo Beltran | 54.3 | 21 | 3.48 | 52.0 | 31 | 5.37 |
| Lance Painter | 145.7 | 75 | 4.63 | 304.3 | 187 | 5.53 |
| Kent Mercker | 266.3 | 150 | 5.07 | 1059.0 | 462 | 3.93 |
| Juan Acevedo | 200.7 | 95 | 4.26 | 369.3 | 179 | 4.36 |
| Jeff Brantley | 50.7 | 25 | 4.44 | 808.7 | 299 | 3.33 |
| Kent Bottenfield | 324.0 | 150 | 4.17 | 587.7 | 310 | 4.75 |
| Jose Jimenez | 184.3 | 113 | 5.52 | 337.0 | 172 | 4.59 |
| Braden Looper | 450.6 | 224 | 4.47 | 530.7 | 205 | 3.48 |
| Garrett Stephenson | 505.0 | 256 | 4.56 | 146.3 | 73 | 4.49 |
| Larry Leubbers | 45.7 | 26 | 5.12 | 97.7 | 53 | 4.88 |
| Mark Thompson | 54.3 | 23 | 3.81 | 282.7 | 192 | 6.11 |
| Ricky Bottalico | 73.3 | 40 | 4.91 | 555.3 | 239 | 3.87 |
| Heathcliff Slocumb | 103.0 | 44 | 3.85 | 528.0 | 242 | 4.12 |
| Darryl Kile | 544.3 | 214 | 3.54 | 1621.0 | 778 | 4.32 |
| Pat Hentgen | 194.3 | 102 | 4.72 | 1881.0 | 894 | 4.28 |
| Britt Reames | 40.7 | 13 | 2.88 | 177.3 | 115 | 5.84 |
| Dave Veres | 224.0 | 83 | 3.34 | 470.0 | 182 | 3.48 |
| Mike James | 89.3 | 40 | 4.03 | 224.7 | 88 | 3.53 |
| Mike Timlin | 163.3 | 61 | 3.36 | 1041.0 | 425 | 3.67 |
| Mike Matthews | 140.0 | 62 | 3.99 | 103.7 | 61 | 5.30 |
| Luther Hackman | 119.3 | 57 | 4.30 | 92.7 | 63 | 6.12 |
| Dustin Hermanson | 222.0 | 113 | 4.58 | 1061.0 | 487 | 4.13 |
| Woody Williams | 588.7 | 231 | 3.53 | 1627.7 | 800 | 4.42 |
| Steve Kline | 247.3 | 74 | 2.69 | 435.0 | 192 | 3.97 |
| Chuck Finley | 85.3 | 36 | 3.80 | 3112.0 | 1330 | 3.85 |
| Travis Smith | 54.0 | 43 | 7.17 | 57.7 | 38 | 5.93 |
| Jason Isringhausen | 408.0 | 135 | 2.98 | 499.3 | 229 | 4.13 |
| Jeff Fassero | 95.7 | 55 | 5.17 | 1938.0 | 874 | 4.06 |
| Brett Tomko | 202.7 | 119 | 5.28 | 1538.3 | 787 | 4.60 |
| Danny Haren | 118.7 | 64 | 4.85 | 878.7 | 348 | 3.56 |
| Cal Eldred | 171.3 | 65 | 3.41 | 1196.7 | 607 | 4.57 |
| Esteban Yan | 43.3 | 29 | 6.02 | 652.0 | 368 | 5.08 |
| Kiko Calero | 83.7 | 26 | 2.80 | 159.0 | 70 | 3.96 |
| Russ Springer | 133.7 | 45 | 3.03 | 664.0 | 358 | 4.85 |
| Jason Marquis | 602.7 | 308 | 4.60 | 666.3 | 334 | 4.51 |
| Jeff Suppan | 572.0 | 251 | 3.95 | 1677.0 | 907 | 4.87 |
| Chris Carpenter | 666.7 | 230 | 3.11 | 870.7 | 467 | 4.83 |
| Ray King | 102.0 | 33 | 2.91 | 309.0 | 125 | 3.64 |
| Julian Tavarez | 130.0 | 42 | 2.91 | 1239.3 | 635 | 4.61 |
| Jason Simontacchi | 285.0 | 151 | 4.77 | 70.7 | 50 | 6.37 |
| Al Reyes | 74.7 | 16 | 1.93 | 354.0 | 166 | 4.22 |
| Sidney Ponson | 68.7 | 40 | 5.24 | 1633.0 | 896 | 4.94 |
| Josh Hancock | 89.7 | 40 | 4.02 | 88.0 | 43 | 4.40 |
| Kip Wells | 162.7 | 103 | 5.70 | 1065.7 | 535 | 4.52 |
| Todd Wellemeyer | 255.3 | 101 | 3.56 | 178.3 | 112 | 5.65 |
| Joel Pineiro | 212.3 | 113 | 4.79 | 1030.0 | 515 | 4.50 |
| Ryan Franklin | 158.7 | 58 | 3.29 | 888.7 | 430 | 4.35 |
| Troy Percival | 40.0 | 8 | 1.80 | 657.3 | 234 | 3.20 |
| Kyle Lohse | 200.0 | 84 | 3.78 | 1164.0 | 624 | 4.82 |
| Ron Villone | 50.0 | 26 | 4.68 | 1069.3 | 565 | 4.76 |
3 comments:
First Steven i got to say WOW thats a lot of work you did just to prove a point. While I agree with you on the premise that coaches don't always matter in the W/L column I think coaches matter more than you give them credit for. Their impact can't always be measured by a Team standard ( a team's batting average or team ERA) b/c its up to the idividual player to grasp and implement the new mechanics or pitches. For instance some of the pitchers that Duncan had when they were just coming up who had long successful careers maybe learned a new pitch or new pitch mechanics from Duncan and while it might not have showed up at the time helped that pitcher throughout his career. Maybe Duncan impacted Dan Haren and he is the pitcher he is today b/c of him or maybe not. I think Dave Duncan is a good pitching coach but I just don't think any math could really quantify it. For instance I feel St. Claire is a good pitching coach as well even though the Nats have had some pretty bad pitching numbers for the last 2 years. But given the fact that he has gotten the most out of Traber, Bacsik, Bowie, and Redding and accelerated the development of John Lannan I feel comfortable in saying he's a good coach.
I wasn't really trying to prove a point so much as actually look at data in a way I've never seen.
And you're right it's possible that Danny Haren learned something under Duncan that helped him later and who knows.
I still think though that if you believe that coaches can generate 10+ more wins as some folks have specualted, that this THAT level of impact would have to be more apparent in the numbers.
Steven,
Thanks for the efforts but it seems we are no where further than we were before your efforts.
No good study can be done due to the lack of controls. Your work has all the flaws you point out and many many others.....Pitches who left Duncan in STL and moved to AL where ERA's are higher? Pitchers who left Duncan in Oakland to go to NL where ERA's are lower......Just too many problems with the data you put out.......I think what we do know from our own experiences in our own careers is that leadership matters and so do personal relationships in focusing our efforts and motivation.
Like I said before, I say 10 wins, you say less than one win.......We will never know......oh and my 10 wins was a 2008 Nationals specific estimate......not the same for a vet team like the Yanks or Red Sox.
Post a Comment