Tuesday, March 31, 2009

The 2009 Season, Simulated (CHONE)

(Read the intro to this series.)

This post comes a little late, but work is busy this time of year. On the plus side, while you were all waiting with baited breath, ReplacementLevel.com released this year's Diamond Mind projections. The standard deviation for each team? You guessed it, about six games.

Since the goal is to compare playoff odds against the ideal model, I'm going to brazenly steal the RS/RA outputs from the ReplacementLevel.com CHONE projections. Here's what we come up with in 10,000 simulations for each:

First run: With normal levels of uncertainty. This is as accurate as we can get in the real world:


Second run: "Ideal" projections:

Thursday, March 19, 2009

The 2009 Season, Simulated (Hardball Times)

(Read the intro to this series.)

Inputs: Runs scored and allowed taken from here, then adjusted for each team's strength of schedule.

Methods: xlsSports playoff simulator. The standard deviation for each team's winning percentage is set to .035 for the first simulation, and zero for the second. 10,000 seasons were simulated for each run.

First run: With normal levels of uncertainty. This is as accurate as we can get in the real world:

Second run: "Ideal" projections:


Of note: since we ran 10,000 projections instead of 100, the A's pull ahead of the Angels with their better run differential, the gap between the Yankees and Red Sox narrows considerably, and the Braves now win the division more often than the Phillies, as they should.

Wednesday, March 18, 2009

March 19 WBC Update

Team WBC%


Jap 24.3
Kor 13.2
USA 43.6
Ven 19.0

The 2009 Season, Simulated (PECOTA)

(Read the intro to this series.)

Inputs: Runs scored and allowed taken from here, then adjusted for each team's strength of schedule.

Methods: xlsSports playoff simulator. The standard deviation for each team's winning percentage is set to .035 for the first simulation, and zero for the second. 10,000 seasons were simulated for each run.

First run: With normal levels of uncertainty. This is as accurate as we can get in the real world:

Second run: "Ideal" projections:

The difference might not seem profound, but it is. The weaker teams are much, much more likely to make the playoffs in the real world than the theoretical one.

The 2009 Season, Simulated (Intro)

Simulating the upcoming baseball season a lot of times to estimate each team's chances of winning the division or the Wild Card is hardly a new concept. Baseball Prospectus, Diamond Mind, and the Replacement Level Yankee Weblog have been doing this for years, and The Hardball Times just published their take on this year's probabilities. Why do we need another blogger joining the fray?

My short answer* is that most of the other simulations are flawed because they use Diamond Mind to simulate the seasons, and Diamond Mind simply doesn't cut it for this purpose.

The best way I can illustrate this is by looking at ReplacementLevel.com's 2008 projections. Focus on the StD W column. This represents the range of outcomes for that team between -1 standard deviation and +1 standard deviation from the mean. Scanning the numbers, you'll notice the spread is almost always 12 or 13 wins, e.g. 68-80.

This is way too precise. Way back in 2005, Keith Woolner studied just how precise our projections could be. He simulated 1000 seasons using the actual 2004 standings and came up with an average standard deviation of about 6.3 wins per team per season, which agrees with basic statistics. (If a team is projected to win x games, their standard deviation in a vacuum will equal the square root of [x*(162-x)/162], which will be between 6.1 and 6.36 wins for all reasonable estimates of x.)

In this utopian simulation, the spread between -1 standard deviation and +1 standard deviation should be 12.6 wins, which matches the Diamond Mind results. However, Woolner was using perfect information: if a team won 92 games in 2004, they were a 92-win club for every game of every simulation. Each of that team's pitchers was equally good, and players never got injured or took days off. We don't live in that world; here on Earth, four teams** missed their Vegas over/under by an average of 21 games apiece last year.

Hopefully we are not so arrogant that we pretend to know everything about the upcoming season. How many innings is Rich Harden going to pitch in 2009? I don't know, but I do know that the answer will greatly affect the Cubs' win total. Realistically, we aim for a standard deviation of 8.5 or 9 wins per team, not 6.3. We need a way to account for all the question marks, and the answer is a random variable in each simulation.

Baseball Prospectus gets this part right. Each team has a constant base rating derived from their projected talent, but their strength in each individual simulation will bounce up and down around this base. The Red Sox will not be a .600 team every time; Josh Beckett's arm might blow up in one sim, whereas he wins the Cy Young in a different one.

In both BP and Diamond Mind sims, a 97-win team will average 97 wins across a million simulated seasons; but when the random variables are added, an 86-win underdog has a much better chance of overtaking the juggernaut in any given sim. (One constant in sports is that big underdogs almost always prefer more variance; call it Kill Phil.) This is why you will see last-place teams make the playoffs far more often in BP simulations than in Diamond Mind simulations.

Onward! A few weeks ago, I came across an ingenious Monte Carlo simulator for the baseball season. Using Excel, you can input each team's projected runs scored and allowed, and more importantly, you can adjust the standard deviation for each team's projected winning percentage. That makes the playoff percentages much more realistic, and it's why I'll be using this spreadsheet for my simulations. (If all goes according to plan, I'll run the sims twice to show how profound this effect is, once with a normal standard deviation and once with none at all.)

Up next: 2009 season forecasts from CHONE, PECOTA, ZiPS, and The Hardball Times. Stay tuned.

* Other complaints: 100 or 1000 simulations are not really enough to generate accurate percentages, BP's odds usually "overreact" to early season results, and the BP depth charts are ineffective at forecasting bullpen usage, often to the point where a ninth reliever is projected for as many innings as a setup man.

** Rays, Tigers, Mariners, and Padres.

March 18 WBC Update

Team Pool% WBC%
Cub 27.2 9.9
Jap 33.3 13.2
Kor 39.4 13.0
USA 67.3 45.0
Ven 32.7 18.9

Monday, March 16, 2009

March 16 WBC Update

Team Pool% WBC%
Cub 17.6 8.0
Jap 51.0 23.6
Kor 26.4 10.1
Mex 5.0 1.6
PR 24.0 11.5
USA 47.2 30.6
Ven 28.8 14.6

Saturday, March 14, 2009

March 15 WBC Update

Team Pool% WBC%
Cub 30.7 13.0
Jap 38.5 17.3
Kor 19.0 6.9
Mex 11.7 3.7
Ned 0.2 0.0
PR 23.8 11.7
USA 47.6 32.7
Ven 28.5 14.7

Wednesday, March 11, 2009

Updated WBC Probabilities

Now that the field is set (though Cuba and Mexico still need to have their seeding determined) here's how I see the odds shaking out:

Team Pool% WBC%
Cub 30.4 12.0
Jap 38.1 16.1
Kor 17.6 5.8
Mex 13.8 4.2
Ned 0.4 0.1
PR 14.7 7.4
USA 62.0 41.9
Ven 22.9 12.5

Amusing (to me, at least): Jayson Stark marvels at the 200-1 odds Bodog offered on the Netherlands to win the WBC at its outset. I still wouldn't bet them at that price now.

How bad a bet was this? According to Stark, you could have bet the Netherlands to advance from Pool A at 50-1, and CRIS offered 30-1 on them to win the WBC before today's game. That's already a payoff of 1580-1 on a parlay, and we haven't even shopped around for better odds.

Wednesday, March 4, 2009

World Baseball Classic Estimates

Team W%
Australia .0
Canada .8
China .0
Chi Taipei .1
Cuba 11.4
DR 20.6
Italy .0
Japan 14.2
Korea 5.8
Mexico 4.6
Netherlands .0
Panama .6
PR 5.4
S Africa .0
USA 27.3
Venezuela 9.0