openWAR on Baseball Analytics
One of Loyola's very own professors, Dr. Greg Matthews, created a competitive aggregate measure with baseball analytics called openWAR, that is based upon public data and methodology with greater rigor and transparency. WAR stands for Wins Above Replacement. This measure aggregates the contributions of a player in each facet of the game: hitting, pitching, base running, and fielding. Current versions of WAR depend upon proprietary data, ad hoc methodology, and opaque calculations to capture overall player performance.
As MLB award season arrives, no prize looms larger than Most Valuable Player. From a statistical perspective, the best guide to the MVP award is undoubtedly wins above replacement, and some voters develop their MVP ballots at least in part based on WAR. The problem is that we don’t know who truly has the most WAR in each league. WAR looks like a single easy-to-understand stat, but it’s the product of a complex model.
The true value of a player varies from what you find on the leaderboard — but we're not sure by how much. That’s because of sample size. Although a whole season of baseball seems like a lot, it still doesn’t provide enough data to allow us to be completely sure of each player’s value. With statistics, comes uncertainty. We measure that uncertainty with confidence intervals. The smaller intervals yield greater confidence and larger intervals yield a lesser confidence. A confidence interval is a range that, based on statistical analysis, is thought to contain the true value of a player a certain percentage of the time.
Dr. Matthews uses openWAR to generate fictional seasons with randomized plays. He found the confidence intervals are simply too large to surely determine who is the more valuable player. Much of the uncertainty in WAR comes from our imperfect measurements of defense. Because the vast majority of defensive plays are routine, a player’s true defensive skill can be seen only on the few plays that are between the impossible and the everyday. Without detailed data on how defenders are positioned before the play starts, most of our metrics are confounded by the front office’s ability to instruct its players on where to stand.
WAR itself is not complete. Although all versions of WAR available today cover the basics of player value (hitting, fielding, base running and pitching), no current version picks up on some of the more esoteric skills in baseball, such as a catcher’s pitch framing. Despite all of its flaws, WAR is still the best available tool for judging value, and certainly exceeds the older alternatives, such as RBIs and pitcher wins. At a minimum, WAR can tell us who the MVP isn’t.
Dr. Matthews earned a Ph.D. in statistics in 2011 from the University of Connecticut. Prior to this, he spent two years in the "real world" in a direct marketing department building predictive models after receiving his B.S. in actuarial science and M.S. in applied statistics from Worcester Polytechnic Institute (WPI) in 2004 and 2005, respectively. Most recently Dr. Matthews completed a 3 year appointment as a post-doctoral research fellow at the University of Massachusetts-Amherst. His research interests include statistical disclosure control, missing data methods, statistical genetics, and statistics in sports. He runs a blog about statistics and its applications. Check it out for more updates on the openWAR discussion and a direct download to his application.