Simulating the upcoming baseball season a lot of times to estimate each team's chances of winning the division or the Wild Card is hardly a new concept. Baseball Prospectus, Diamond Mind, and the Replacement Level Yankee Weblog have been doing this for years, and The Hardball Times just published their take on this year's probabilities. Why do we need another blogger joining the fray?
My short answer* is that most of the other simulations are flawed because they use Diamond Mind to simulate the seasons, and Diamond Mind simply doesn't cut it for this purpose.
The best way I can illustrate this is by looking at ReplacementLevel.com's 2008 projections. Focus on the StD W column. This represents the range of outcomes for that team between -1 standard deviation and +1 standard deviation from the mean. Scanning the numbers, you'll notice the spread is almost always 12 or 13 wins, e.g. 68-80.
This is way too precise. Way back in 2005, Keith Woolner studied just how precise our projections could be. He simulated 1000 seasons using the actual 2004 standings and came up with an average standard deviation of about 6.3 wins per team per season, which agrees with basic statistics. (If a team is projected to win x games, their standard deviation in a vacuum will equal the square root of [x*(162-x)/162], which will be between 6.1 and 6.36 wins for all reasonable estimates of x.)
In this utopian simulation, the spread between -1 standard deviation and +1 standard deviation should be 12.6 wins, which matches the Diamond Mind results. However, Woolner was using perfect information: if a team won 92 games in 2004, they were a 92-win club for every game of every simulation. Each of that team's pitchers was equally good, and players never got injured or took days off. We don't live in that world; here on Earth, four teams** missed their Vegas over/under by an average of 21 games apiece last year.
Hopefully we are not so arrogant that we pretend to know everything about the upcoming season. How many innings is Rich Harden going to pitch in 2009? I don't know, but I do know that the answer will greatly affect the Cubs' win total. Realistically, we aim for a standard deviation of 8.5 or 9 wins per team, not 6.3. We need a way to account for all the question marks, and the answer is a random variable in each simulation.
Baseball Prospectus gets this part right. Each team has a constant base rating derived from their projected talent, but their strength in each individual simulation will bounce up and down around this base. The Red Sox will not be a .600 team every time; Josh Beckett's arm might blow up in one sim, whereas he wins the Cy Young in a different one.
In both BP and Diamond Mind sims, a 97-win team will average 97 wins across a million simulated seasons; but when the random variables are added, an 86-win underdog has a much better chance of overtaking the juggernaut in any given sim. (One constant in sports is that big underdogs almost always prefer more variance; call it Kill Phil.) This is why you will see last-place teams make the playoffs far more often in BP simulations than in Diamond Mind simulations.
Onward! A few weeks ago, I came across an ingenious Monte Carlo simulator for the baseball season. Using Excel, you can input each team's projected runs scored and allowed, and more importantly, you can adjust the standard deviation for each team's projected winning percentage. That makes the playoff percentages much more realistic, and it's why I'll be using this spreadsheet for my simulations. (If all goes according to plan, I'll run the sims twice to show how profound this effect is, once with a normal standard deviation and once with none at all.)
Up next: 2009 season forecasts from CHONE, PECOTA, ZiPS, and The Hardball Times. Stay tuned.
* Other complaints: 100 or 1000 simulations are not really enough to generate accurate percentages, BP's odds usually "overreact" to early season results, and the BP depth charts are ineffective at forecasting bullpen usage, often to the point where a ninth reliever is projected for as many innings as a setup man.
** Rays, Tigers, Mariners, and Padres.