Evaluating prospects using just stats
by Bososx13 on Dec 17, 2011 8:16 PM EST
Email
Print
As most of you know, I'm a big stats guy, I was reading some archives on BP and I found this http://www.baseballprospectus.com/article.php?articleid=6153 . It was ranking players based on PECOTA. PECOTA is a projection system, when Nate Silver was there it was really good, but it's pretty much crap after he left, but he did a prospect ranking based on current projected PECOTA WARP and the upside of a player, using PECOTA's long term projections.
PECOTA nailed Dustin Pedroia neither Kevin Goldstein nor BA had him on their top 100 list and PECOTA had him 6th. PECOTA also ranked Tim Lincecum and Evan Longoria higher than BA and Kevin Goldstein. It also nailed Joey Votto, they put him 15 spots higher. It also ranked Jeremy Hellickson and Justin Masterson who weren't ranked by Goldstein or BA. It did rate Justin Upton, Andrew McCutchen and Troy Tulowitzki a LOT lower than expected which is not good, also Jacoby Ellsbury was lower than expected. But it also rated Clayton Kershaw higher than expected and Cameron Maybin 25th instead of 6 or 7. Matt Garza was also lower than expected, but Delmon Young was also much lower than expected which is good. It rated Miguel Montero and Daric Barton more than expected and Colby Rasmus less than expected. It put Jeff Niemann in the 90s instead of the 20's like Goldstein and BA. It ranked Peter Borjous 84th when BA and Goldstein didn't rank him. I won't give them a success for rating Nick Adenhart much lower, even though his WAR is very low because who knows what could've happened.
I was inspired by this post because I remember someone saying that tools matter and stats don't. While PECOTA had some obvious misses, it had a couple players very high who Goldstein and BA didn't put on their list, who ended up being very good players. It goes to show that using stats AND scouts, not stats or scouts is the best way to go, and that's why I do that.
57 comments
|
0 recs |
Do you like this story?
Comments
"I remember someone saying that tools matter and stats don't."
If I inspired you to write this, then I take that as a compliment.
In fairness, the exact quote was; “Tools matter, stats don’t. At least in the minors.”
Carrying it one step further, stats matter less the lower level one is playing in.
Harper was at the lowest (full season) level one could play at.
Teams don’t care what players do on the field outside of getting hurt.
For many, especially for a high school age draftee such as Harper, this is their first time away from home for such a long period of time.
Players have to learn how to pay their rent, to do their laundry, to eat right, to pack for road trips, to fight the long stretches of down time, especially on road off days.
The life of a professional baseball player is less about baseball than anything else.
So given everything you just said about the adjustment they're making,
Doesn’t that make Harper’s 168 wRC+ (highest in the SAL for anyone with over 300 PA) that much more impressive?
No.
It changes nothing.
I don’t care what he did in A ball.
Even though it’s a much smaller sample size, what he did in AA carries much more weight.
And I certainly don’t need some off the wall sabermetric stat to tell me that, either.
Off the wall?
It really isn’t. Just gives a more accurate run value to everything offensive players do and then compares them to everyone else in the league.
And you really don’t care that he was the best hitter in A ball as an 18 year old? That isn’t at all relevant to how good he currently is or projects to be? Why?
And even if you think what he did in AA carries more weight, he was still above average their offensively as an 18 year old. That is incredibly impressive. Seriously, who was the last 18 year old to put up above average offensive numbers in AA in his first professional season?
No, I don't.
If he had hit .213 I wouldn’t care any more or less either way.
You’re making it seem like Harper ran over my dog or something, and that’s not only unfair but ignorant, because of your obvious bias.
If you were a fan of any other team, we wouldn’t be having this discussion.
“Seriously, who was the last 18 year old to put up above average offensive numbers in AA in his first professional season?”
The league average was .259. Harper hit .256.
The league SLG was .395, same as Harper.
The league OBP was .329, so was Harper’s.
Over thirty-seven games, he was barely league average.
Over a full 140 game season, he projected to eleven homers.
The same amount Beau Mills hit.
In 62 games.
Or that Matt den Dekker hit in 74 games.
Over 140 games, he drives in 46 runs.
The same amount Cody Overbeck drives in in 62 games.
You know, the legendary Cody Overbeck?
Should I keep going?
In fairness to him, though, he would have finished in the top 25 in strikeouts.
Umm...
You’re making it seem like Harper ran over my dog or something, and that’s not only unfair but ignorant, because of your obvious bias.
If you were a fan of any other team, we wouldn’t be having this discussion.
Nixa being a Braves fan makes him biased in favor of a Nats prospect?
http://bullpenbanter.com
RIP Randy "Macho Man" Savage
OK, fine, but he's still biased towards Harper.
Either way, the point stands.
I'm biased based on what?
If anyone is biased here its you. I’ve laid out my personal reasoning. Yours seems to be based on a combination of him not being good enough in AA as an 18 year old and your personal opinion (that I haven’t seen a single reputable source back up) that he’s already maxed out his potential.
"Over thirty-seven games, he was barely league average."
Key words: “Over thirty-seven games”
Kind of a small sample size to use for your argument Kels, especially after espousing the uselessness of stats vs. scouting, when scouts have universally praised Bryce’s abilities.
by cookiedabookie on Dec 17, 2011 11:52 PM EST up reply actions
I've mentioned sample size at least twice before,
but no one seems to grasp it.
are all stats below AA meaningless then?
and consequently, is the only way to evaluate players below AA through scouting?
I'm honestly thinking you're just a troll
Beau Mills and Cody Overbeck are 6 years older than Harper. Matt den Dekker is 5 years older. The fact that Harper was better than league average in AA as an 18 year old is a huge plus. Seriously, can anyone else since Rodriguez say they did better as an 18 year old in AA in their first professional season?
then how do you evaluate him compared to league average?
is he below league average because he hit 3 points lower?
"PECOTA nailed Dustin Pedroia neither Kevin Goldstein nor BA had him on their top 100 list "
BA had Pedroia 77th in 2006.
heh
I’m unaware of anybody who DOESN’T use all the information at their disposal.
I think
some people do not put ENOUGH emphasis on stats, because of the fact that Pedroia who is an MVP caliber type player did not get ranked by KG or BA because they said his tools weren’t good enough. Keith Law said that Pedroia would be nothing better than a utility infielder. I just think that people don’t put enough emphasis on stats. Weighing stats and tools, most of the major industries probably go 70-30 or 80-20 tools vs stats. I think it should be 50-50, seeing as how both lists have major misses that the other list had, and major hits the other list didn’t have.
"because of the fact that Pedroia who is an MVP caliber type player did not get ranked by KG or BA because they said his tools weren’t good enough"
They’re NOT good enough.
What makes Pedroia special isn’t a tool, it can’t be quantified by a scouting report or a stat sheet.
It’s his heart and his mind.
He’s one of the hardest workers out there and has made himself a star by busting his ass 24 hours a day.
Based strictly on athletic ability, Pedroia IS a utility player.
It’s not fair to knock BA or KG over his omission any more than it is to give BP credit for making a good guess.
I think Pedroia's tools are a little underrated
He has an amazing hit tool, plays very good defense, has 20 homer power and 20 steal speed. This year he was a .300 20/20 player with a +17 UZR, wouldn’t that usually be thought of as very toolsy. He also has very good plate discipline.
This quote is from the link provided by the author...
“Dustin Pedroia (PECOTA #6, BA N/R, KG N/R). I’ve watched quite a few of his plate appearances this year and can’t say I’m impressed. Then again, it isn’t easy to be impressed by someone who is struggling to hit .200. Still, Pedroia’s BABIP thus far this year is a meager .205, versus a career minor league rate of .317. He strikes out so rarely and walks so often that if he can get that number up to a modest .280, he should be good for a .360 OBP, which isn’t without value at a middle infield position. Yeah, I’m hedging my expectations a little.”
So, despite ranking him sixth, Silver admitted to not liking Pedroia much and considered him a “development project.”
So, in the long run, how he ranked him was absolutely a guess, and he lucked out.
Pedroia has FAR exceeded any expectations placed upon him, anyone who saw him in college and said he’d be a perennial MVP candidate is a liar.
It wasn't just a guess by Silver
the fact that Pedroia just kept on destroying the minors showed that he should have been ranked higher, and that’s why I think BA and KG place too much of their rankings on scouting reports. It’s a lot like mrkupe said. You’re more likely to get money off of scouts than stats.
The reason
Pedroia wasn’t on the Top 100 list in 2007 was because of his performance in Boston at the end of ’06.
If he had played better, or had spent the whole season in Pawtucket, I’m confident he would not only would have made their list, but would have been higher than 77th.
"It wasn't just a guess by Silver"
It absolutely was.
Anytime you take a player’s minor league numbers and run them through a computer program, your result is a guess.
Good or bad.
It was not a guess
Silver used statitsical evidence to make that ranking. He looked at Pedroia’s numbers, his age, and all comparable players to Pedroia and how they did. He said he wasn’t impressed with him because nobody would be impressed with a guy hiting .200
"It was not a guess"
It absolutely was a guess.
Statistics are a definitive record of something which has already happened.
They are a poor indicator of predicting the future.
You know of anyone who projected Jacoby Ellsbury to hit 32 homers this year?
How about Ian Kennedy winning 20 games?
Didn’t think so.
The same article you provided had Silver ranking Troy Tulowitzki 78th.
BA had him fifteenth and Goldstein 24th.
Dan O’Dowd picks up the phone tomorrow and calls Ben Cherington and offers him Tulowitzki for Pedroia even up, Dustin’s a Rockie.
Careful when cherry-picking, especially with sabermetrics.
Always bites you in the ass in the long run.
are you joking?
you picked two breakout seasons, and because no one predicted them, you concluded that stats are worthless?
Maybe you should be careful when cherry-picking, too.
I would definitely not take that trade for the Red Sox
Tulowitzki is so much more expensive. Pedroia has a better career WAR. Using a simple MARCEL calculator, Pedroia has 5.8 WAR, Tulowitzki has 5.4. Pedroia has a better career wRC+ and UZR/150. ZiPS projections are very good for projecting the future. CAIRO is very good too. When Silver was there, PECOTA was very good at projecting the future. They didn’t project Ellsbury to hit 30 homers because nobody did.
let's be fair and say you're both half right, half wrong
It was definitely a guess, but it was a highly educated and well-informed guess. Projecting the future IS a guess, there is no getting around that. Nate Silver is immensely talented, but he doesn’t have access to a time machine. There are ways to stack that deck so to speak, but you’re still going to have times (maybe a lot of times, depending on how strong your system is) when you’ve got 13 and the dealer flips a 10.
if thats the case
wouldn’t we expect that the scouts, the guys that watch him bust his ass day in and day out, study his work ethic, etc., to rate him more highly than something that can’t take this into account? I don’t see how the fact that he doesn’t have great prototypical tools is at all relevant to this argument.
Pedroia wasn't really disliked scouting-wise, though
He has certainly come in at the high end of expectations, but there are tons of guys with great numbers in the college game all the time who didn’t get drafted nearly as highly as he did. He was liked much more than a lot of heavy saber guys seem to want to admit.
really?
Peds was ranked on BA’s Top 100 list once: it was pre-2006.
just saying you like a guy doesn’t mean anything. people say they like a prospect all the time. it doesn’t mean they wouldn’t take a prospect that thye liked more.
if the scouts liked him, he would have ranked higher. i think it’s that simple. BA’s rank is as good as it gets when it comes to putting a scout’s projected EV on a prospect.
by blue bulldog on Dec 19, 2011 4:38 AM EST up reply actions
it doesn't mean anything?
Maybe it doesn’t to you, and perhaps groupthink minimizes the extent to which dissent is represented, but there is clearly room to “not like” a prospect. Even one who is not highly thought of.
Invoking “top 100” arguments is silly, because as I’m sure you recognize, cutting off a top prospects list at 100 is arbitrary in the first place. There are more than 100 good prospects in the minors at any given time.
Pedroia was in the minors for three seasons. The first was his draft season, and as a 2nd rounder who wasn’t a toolshed, he probably didn’t really have a shot at making it. His first full season he played at AA/AAA and, as you mentioned, he made it. He came in low in the rankings in his third season, but he was still good enough to make his league’s top 20. Obviously Pedroia was never an ideal prospect from a scouting perspective, but there has always been a healthy respect for what he can do. He wouldn’t have ended up going the places he has gone (starting player for one of the elite college programs in the country, 2nd round draft pick, making the majors in two years and change, MVP caliber performance) otherwise.
i'd imagine most 2nd rounders are liked by some scouts somewhere.
No scout has a perfect track record, so it seems a little odd that we hold analysts to such high standards.
You need a mix of scouting and stats to successfully evaluate prospects; how heavily you weight them will impact the types of players you favor. As kelsdad alluded to above, there’s a human element to any baseball player that eludes quantification, although that doesn’t necessarily mean it’s something you can pick up through scouting. That said, there’s really no reasonable way to deny that statistics, at any level, have some predictive power. I’d be wiling to bet that the top 10 first-year players in wRC+ in rookie ball have, on average, better careers than the bottom 10.
If we all agreed on one way to evaluate prospects, what fun would this be? It’s boring to just agree all the time.
you're missing the point
any look at an expected value chart shows that the value drops off exponentially the farther away you go from being the #1 prospect in baseball
not being on the top 100 list is an indication that the scouts thought his expected value was low. again, it’s really that straight forward. and they were wrong. it happens.
by blue bulldog on Dec 19, 2011 11:49 PM EST up reply actions
actually, I'm not
I’m very much aware of a relationship between expected value and prospect ranking, not that we should be holding onto said relationship too closely for practical purposes.
Obviously just about everybody “missed” on Pedroia to a certain extent (as he’s a perennial All Star performer). It doesn’t mean he wasn’t expected to have at least a decent major league future. He wouldn’t have sniffed the 2nd round of the MLB draft without scouts seeing him and vetting his abilities. Same for him making a BA list the year after he was drafted. You seem to be ignoring both of those things.
Again, there are more than 100 good prospects in MiLB at any given time. You try to make a list of the 100 best, even in the best case you’re going to be “wrong” all over the place. Not everybody deserving of recognition is going to get in. The relationship between expected value and ranking is useful to note from the macro level, but not so much from the micro level. A high ranking does not make an individual player more valuable in terms of expected performance.
er...
actually that’s exactly what a high ranking means
if you don’t expect a higher ranked prospect to have a higher expected value than a lower ranked prospect what’s the point of even ranking prospects?
that’s what a ranking fundamentally is. you’re saying that in a vacuum, you’d be willing to trade the #11 prospect for the #10 prospect, and so on and so forth, until you get the #1 prospect, who you would not be willing to trade for any other prospect in baseball
not being on a Top 100 list just means that there were 100 guys on that list that the scouts would have been willing to trade Pedroia for. who cares about the fact that the scouts say they “liked” him?
at the end of the day, it’s all relative, and the scouts demonstrated with their ranking that they liked other prospects a lot more than Pedroia
by blue bulldog on Dec 20, 2011 1:31 AM EST up reply actions
that is not what I said
My point wasn’t that there aren’t differences in perception, my point was that these differences in perception aren’t nearly as profound as you seem to be suggesting. For one, saying “the scouts” isn’t exactly true – it’s Baseball America’s thoughts on their compilation of what they gleam from scouts/other sources. Two, “the scouts” aren’t of one mind – I’m sure there were people who really liked Pedroia and those who thought he was a utility player. Seems a little harsh to kill a player’s expected value just because you happened to talk to one guy and not the other, or that you valued the information from one guy more than the other.
I find it hard to believe that just because Pedroia didn’t make a list one year, this meant that suddenly there were a bunch of players who were liked “a lot more”. This is understood even in lists compiled by members of the community here, as we recognize that a difference in the ranking of an individual player really isn’t that big a deal unless we’re talking about something on the order of 40-50 spots. Do you really think it’s any different for the likes of Baseball America?
have you ever taken it from a marketing perspective?
What is likely to sell more: sabermetric analysis taken from numbers that anybody can figure out, or scouting-oriented analysis with an insider perspective that one ostensibly “can’t get anywhere else”?
There are very good reasons beyond purely baseball reasons as to why “major industry” sources (and even many of those who aren’t) base their legitimacy off of scouting, not stats. It doesn’t necessarily mean that the sabermetric component is ignored by those parties, but it is certainly de-emphasized. Keep this in mind next time you fling advanced statistics around like pieces of candy: we all know about those already, and the people who you seem to presume don’t know, are the ones who probably know better than most anybody else.
Sorry if I sound annoyed, but it’s just a little silly to rehash the whole saber/scouting thing all over again for no reason other than that you got in a random argument with somebody else.
it's actually the exact opposite
if you have a good sabermetric formula, you will probably get bought out by a team, and they will use your proprietary formula.
in fact, you can think of it this way. quality sabermetric analysis is too valuable, and priced out of the average consumer’s range. that’s why it’s not that meaningful to have “major industry” sources use sabermetric analysis and sell it to the public. on the other hand, scouting of BA’s calibre is easily attainable and therefore cheap, at a price of $66 a year.
as a practical matter, that’s the major reason why you see scouting being used more heavily in these publicly available sources.
by blue bulldog on Dec 19, 2011 4:44 AM EST up reply actions
depends on the audience you're playing to
Major league organizations have different information needs than the mass public. For the organizations, scouting information is obviously not at a premium – if anything, they probably struggle a lot with information overload there. For “quality sabermetric analysis”, yes, I’ll agree, they would probably care much more about paying $$$ for it.
For consumer-level information needs (which is what I was looking at), I think the opposite is true. Sabermetrics are hard to market successfully to consumers – not only do you have to crunch the numbers in novel ways, but you have to be able to write about said numbers in a clear and interesting manner. IMO, nobody has really done both of those things very well since Nate Silver, although I’ll admit I have high standards for readability. Scouting-driven analysis, on the other hand, has the advantage of being written in an argot easily accessible to baseball fans.
I just don’t think the overall market interest in advanced sabermetrics is all that high, and there are limits to the interested public’s appetite for such things. Fangraphs is free and Baseball Prospectus is in the same price range as BA, those two sources probably cover what most people would want. If there was a market for more, seems pretty logical that there’d be someone ready and willing to cover that.
There's nothing in this OP that makes a compelling case for PECOTA or any other approach
Honestly, if there’s a system out there that’s head and shoulders above any other, it had better give me more than Gold Nugget Pedroia every 6 years. And it had better CONSISTENTLY perform at least slightly better at ranking everyone else, too.
Yusmerio Petit, guys.
Yusmerio Petit.
Scouting MUST be involved on some level.
RIP Greg Halman
Well using a well known stats failure
Without acknowledging the many scouting failures is a bit disingenuous. There are failures on both sides of the coin, and I agree both should be used. I myself lean more stats (probably 60-40 stats vs. scouting, with different values depending on where a player is age- and development-wise), but when I first got into prospects I focused almost solely on stats.
by cookiedabookie on Dec 18, 2011 3:09 AM EST up reply actions
An excellent example of why to use stats and scouts
I pointed out that both lists missed and picked major players that the other list didn’t have, or had much lower.
Comments For This Post Are Closed
Secondary Sidebar
User Tools
Related FanPosts on Minor League Ball
FanPosts
Community blog posts and discussion.
Recommended FanPosts
Recent FanPosts
