A run expectancy matrix is essentially a table that shows the average number of runs that will score in an inning after a given combination of outs and baserunners, such as 1 out and a runner on second. The most popular, publicly-available matrix for major league play is the one put forth by Tango et al. in The Book. Below is how the 2013 Appalachian League run expectancy matrix looks based on my compilation of plate appearances from the MLBAM archived web data (accessible here).
So if a 2013 Appalachian Leaguer was at the plate with 1 out and a runner at second, on average the team would score 0.612 runs in the remainder of that inning inclusive of that plate appearance.
Quantifying the Effect on Run Expectancy of a Plate Appearance
Armed with that knowledge, we can now assign the average run expectancy to each plate appearance that occurred in the league based on the relevant permutation of outs and baserunners. At that stage it becomes possible to quantify the value of each plate appearance by subtracting the run expectancy before the plate appearance from the run expectancy after the plate appearance and adding however many runs scored during the plate appearance to the total. This "value" amounts to how the outcome of the plate appearance altered the run expectancy for the inning, and this will be termed the "effect on run expectancy" in what follows.
Quantifying the Effect on Run Expectancy of Plate Appearance General Outcomes
To quantify how a general outcome (walk, strikeout, groundball, etc.) affects run expectancy, we can simply filter out all of those particular events that occurred in the season and average the effect on run expectancy of the plate appearances. And via this approach we can determine for instance that a walk on average had a +0.35 effect on run expectancy (i.e., increased run expectancy by 0.35 runs), a strikeout had a -0.29 effect on run expectancy, and so on.
Included were all pitchers who faced at least 150 batters in the 2013 Appalachian League season and averaged at least 10 batters faced per contest. Data generated during any relief appearances were pooled with the starting numbers. Fifty-eight pitchers so qualified.
Plate Appearance Outcomes Studied
Based on the official MLBAM play outcome data and after excluding bunts and foulouts from the sample, the remaining plate appearance outcomes for each pitcher were categorized into one of twelve categories: 1) walk or hit-by-pitch, 2) strikeout, 3) infield flyball, 4) groundball to the batter’s pull-third of the field, 5) groundball to the batter’s center-third of the field, 6) groundball to the batter’s oppo-third of the field, 7) line drive to the batter’s pull-third of the field, 8) line drive to the batter’s center-third of the field, 9) line drive to the batter’s oppo-third of the field, 10) outfield flyball to the batter’s pull-third of the field, 11) outfield flyball to the batter’s center-third of the field, 12) outfield flyball to the batter’s oppo-third of the field. The average effect on run expectancy of each event type for the 2013 Appalachian League campaign is shown in the table below. Note that flyballs, line drives, and groundballs have a wider range of possible effects on run expectancy than do infield flyballs, strikeouts, and walks or hit-by-pitches.
Now, knowing how many events of each type a given pitcher allowed we can compute what the average effect on run expectancy of a plate appearance versus the pitcher should be by assuming the average effect specified in the table occurred in each instance of the corresponding outcome (for example, all line drives to the center-third were assigned an effect of +0.35 runs). Before proceeding in this fashion, one should check and correct for aberrant classifications of play events coming out of individual league parks; in this league the line drive rate in the Royals’ Burlington affiliate stadium was 5 standard deviations (SD) lower than the average of the other 9 venues while its outfield flyball rate was 3 SD higher as balls that reached the outfield in the air were seldom classified as line drives; I employed a park factors-based approach to control for these sorts of park-related play classification anomalies (event totals in a given park were scaled up or down based on how each club’s home stadium play classifications compared to their road game play classifications). Note that by working with the expected effect on run expectancy of a plate appearance rather than its true effect of run expectancy in this analysis, each pitcher's outcomes on batted balls are now virtually fielding- and ballpark-independent; similarly, the effects on run expectancy of each of their batted balls (esp. the home runs), walks/hit-by-pitches, and strikeouts are no longer context-dependent (for a given pitcher, one or more of the 12 event types may be biased towards occurring in situations that were more or less consequential than typical in terms of run expectancy for that event type).
Grading the Performance of Each Pitcher
Once the average expected effect on run expectancy of a plate appearance versus each pitcher is determined, we can compute the mean and standard deviation of the 58 values and use that to assign an overall Performance Score to each of the 58 pitchers. Here, a 50-point score will indicate league-average performance and each 10 points on the scale will amount to 1 SD with scores greater than 50 beating league-average. To gauge the pitcher’s relative strengths/weaknesses, 3 component subscores are also reported: 1) a Batted Ball Subscore is computed similarly based on the pitcher’s average effect on run expectancy for only plate appearances that involved one of the 10 types of batted ball outcomes, 2) a Control Subscore is computed likewise based on the pitcher’s walk and hit batsmen total per non-bunt, non-foulout plate appearances, and 3) a Strikeout Subscore is computed based on the pitcher’s strikeout total per the same denominator. To quantify the pitcher’s age relative to league peers, an Age Score is also reported based on how many standard deviations the pitcher was young versus the mean for the sample of study pitchers (>50 score connotes younger than league-average).
An asterisk in the results tables denotes a lefthanded thrower, while green or red text highlights a score that is at least 1 SD above or below league-average, respectively.
Two general types of pitchers can be found towards the top of the Performance Score list. The first are relatively old by league standards and include top perfomer Eddie Campbell, Blake McKnight, Andrew Waszak, and Jordan Mills; with these sorts we will probably need to see how they fare in full-season ball against more age-appropriate competition to get a better read on their respective talents. The second group is comprised of pitchers who are performing very well in spite of being rather young by league standards, though not exceptionally so; in this cohort we find Hunter Wood, Felix Jorge, Chase DeJong, Jacob Faria, Edwin Diaz, and Robert Whalen. These latter types would stand to be the truer prospects of the study sample, though few would consider any of them upper-echelon prospects at present.
Towards the bottom of the Top 29 (look up), we spy a few pitchers who are very young by league standards but managed to perform a bit better than league average, namely Jairo Labourt, Alexander Reyes, Chris Flexen, and Jake Newberry.
Among the Bottom 29 of the 58 study pitchers (look down) we find 3 of the 4 youngest arms of the study and they belong to Alberto Tirado, German Marquez, and Thaddius Lowry.
- Have you seen any of these arms throw in person (especially during 2013)?
- What are your thoughts on grading the performance of minor league pitchers via a run-expectancy-based method? Or even via this particular run-expectancy-based method?