1 year later: my 2010 prospect list vs. BA's
by daveh33 on Apr 2, 2011 6:35 PM EDT
Email
Print
This is to open up some discussion more than anything. Over the past couple of seasons, I've begun to wonder whether the experts are really that great at formulating rankings. They are undoubtedly good at what they do, which is compiling tons of minor league baseball content and opinions and scouting reports. But regardless of their resources and knowledge, I am not always pleased with their final Prospect rankings. And its not only BA, its pretty much every expert opinion. I know they have to keep these interesting, and they have a lot of different talking heads involved. A lot of scouts' opinions to take into account, etc. But after a year, I think my list stands up quite well in comparison. Granted, it is only a year, but this is just for fun. I recognize that we won't be able to truly judge these lists for probably another 10 years or so. Last year was the first year that I spent a great deal of time on my own list though [40+ hours researching/compiling/watching vid/etc], so its the first time I've felt that I had a good shot at making a quality ranking that might stand the test of time.
I always publish my list 2 or 3 weeks ahead of BA's, just to be sure I won't be influenced by their rankings. I do of course use some of their opinions to judge a prospect, but mostly only Callis and Manuel. I also rely on Sickels, Piliere, Law, and Eisenberg. And even some of the members here.
here is the url to my fanpost from last year:
The rankings seen are my own, then I listed BA's, and then the result. a P=push, W= win, L=loss, lol= when they matched my ranking and N/A=injured player or really unsure.
for the results, I figured something like this:
for rankings 1-20 :: anything 3 spots or better is a difference, or a win/loss.
21-50:: 4 spots or more.
51-75:: 5 spots or more.
76-100:: 6 or more.
And this is kind of a loose system, since it isn't always clear whether a prospect should be higher/lower. A lot of it is personal opinion. So if you see some results that are a 'Push' even though the difference is great enough for a result, it's because I was unsure of which way to go or because the range/difference is right in-between where I'd rank the prospect.
I also didn't weigh the results, meaning a win is a win, a loss is a loss. Although I probably should have, as Wil Myers alone might have made my score very impressive.
At the end of the rankings, I have the final tally.
And feel free to provide some feedback... if you feel I was too generous/strict with any of the results. Its not always so clear/straightforward as to whether a prospect deserved to be ranked higher/lower based on his current prospect standing in 2011.
@font-face { font-family: "Times New Roman"; }p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: "Times New Roman"; }a:link, span.MsoHyperlink { color: blue; text-decoration: underline; }a:visited, span.MsoHyperlinkFollowed { color: purple; text-decoration: underline; }table.MsoNormalTable { font-size: 10pt; font-family: "Times New Roman"; }div.Section1 { page: Section1; }
1. Jason Heyward - OF ATL BA = 1. Result = P
2. Stephen Strasburg - P WAS BA=2. Result =P
3. Buster Posey - C SF BA=7. Result =W
4. Carlos Santana - C CLE BA=10. Result =W
5. Michael Stanton - OF FLA BA =3. Result =P
6. Desmond Jennings - CF TB BA =6. Result =lol.
7. Jesus Montero - C/DH NYY BA =4. Result =P
8. Brian Matusz - P BAL BA =5. Result =P
9. Neftali Feliz - P TEX BA =9. Result =lol.
10. Dustin Ackley - CF/2B SEA BA =11. Result =P
11. Martin Perez - P TEX BA =17. Result =L
12. Domonic Brown - OF PHI BA =15. Result =W
13. Justin Smoak - 1B TEX BA =13. Result =lol
14. Pedro Alvarez - 3B/1B PIT BA =8. Result =W
15. Christian Friedrich - P COL BA =33. Result =L
16. Aroldis Chapman - P CIN BA =22. Result =W
17. Logan Morrison - 1B/OF FLA BA =20. Result =W
18. Jeremy Hellickson - P TB BA =18. Result =lol
19. Jenrry Mejia - P NYM BA =56 Result =N/A
20. Michael Taylor - OF OAK BA =29. Result =L
21. Wade Davis - P TB BA =34. Result =P
22. Chris Carter - 1B/DH OAK BA =28. Result =P
23. Lonnie Chisenhall - 3B CLE BA =31. Result =P
24. Aaron Hicks - CF MIN BA =19. Result =P
25. Matthew Moore - P TB BA =35. Result =W
26. Starlin Castro - SS CHC BA =16. Result =L
27. Kyle Drabek - P TOR BA =25. Result =P
28. Madison Bumgarner - P SF BA =14. Result =L
29. Derek Norris - C WAS BA =38. Result =P
30. Tyler Matzek - P COL BA =23. Result =W
31. Fernando Martinez - OF NYM BA =77. Result =L
32. Casey Kelly - P BOS BA =24. Result =W
33. Freddie Freeman - 1B ATL BA =32. Result =P
34. Alcides Escobar - SS MIL BA =12. Result =W
35. Mike Montgomery - P KC BA =39. Result =P
36. Chris Withrow - P LAD BA =48. Result =L
37. Aaron Crow - P KC BA =40. Result =P
38. Ryan Westmoreland - CF BOS BA =20. Result =N/A
39. Brett Wallace - 1B TOR BA =27. Result =W
40. Devaris Gordon - SS LAD BA =40. Result =P
41. Jordan Lyles - P HOU BA =91. Result =W
42. Julio Teheran - P ATL BA =51. Result =W
43. Todd Frazier - 2B/3B/OF CIN BA =43. Result =lol
44. Casey Crosby - P DET BA =44. Result =P
45. Donovan Tate - CF SD BA =53. Result =L
46. Grant Green - SS OAK BA =52. Result =W
47. Simon Castro - P SD BA =57. Result =P
48. Jacob Turner - P DET BA =26. Result =L
49. Jared Mitchell - CF CHW BA=55. Result =L
50. Hector Rondon - P CLE BA=NR Result =L
51. Wil Myers - C KC BA =NR Result =WWWWW
52. Reid Brignac - SS TB BA =54. Result =P
53. Yonder Alonso - 1B CIN BA =45. Result =P
54. Josh Bell - 3B BAL BA =37. Result =W
55. Jaff Decker - OF SD BA =82. Result =L
56. Arodys Vizcaino - P ATL BA =69. Result =L
57. Alex Colome - P TB BA =68. Result =L
58. Jhoulys Chacin - P COL BA =71. Result =W
59. Brett Lawrie - 2B/OF MIL BA =59. Result =lol
60. Shelby Miller - P STL BA =50 Result =L
61. Kyle Gibson - P MIN BA =61. Result =lol
62. Dan Hudson - P CHW BA =66. Result =P
63. Zach Stewart - P TOR BA =NR Result =L
64. Tanner Scheppers - P TEX BA =42. Result =W
65. Josh Reddick - CF/OF BOS BA =75. Result =P
66. Mike Trout - CF LAA BA =85. Result =W
67. Tim Beckham - SS TB BA =67. Result =lol
68. Tim Melville - P KC BA =NR. Result =L
69. Tyler Flowers - C/DH CHW BA =60. Result =P
70. Mike Moustakas - 3B KC BA =80. Result =W
71. Brett Jackson - CF CHC BA =74. Result =P
72. Jose Iglesias - SS BOS BA =NR. Result =W
73. Zach Britton - P BAL BA =63. Result =L
74. Ryan Kalish - CF BOS BA =NR. Result =W
75. Hank Conger - C LAA BA =84. Result =P
76. Danny Duffy - P KC BA =NR. Result =W
77. Jay Jackson - P CHC BA =98. Result =L
78. Ethan Martin - P LAD BA =NR. Result =L
79. Jake Arrieta - P BAL BA =99. Result =P
80. Josh Vitters - 3B CHC BA =70. Result =W
81. Alex Avila - C DET BA =NR. Result =L
82. Jiovanni Mier - SS HOU BA =73. Result =W
83. Zach Wheeler - P SF BA =49. Result =L
84. James Darnell - 3B SD BA =90. Result =P
85. Matt Dominguez - 3B FLA BA =NR. Result =W
86. Manny Banuelos - P NYY BA =NR. Result =W
87. Mat Gamel - 3B/OF MIL BA =89. Result =P
88. Michael Saunders - OF SEA BA =30. Result =W
89. Nick Weglarz - OF CLE BA =NR. Result =W
90. Hak-Ju Lee - SS CHC BA =NR. Result =P
91. Mike Leake - P CIN BA =72. Result =P
92. Wilmer Flores - SS NYM BA =88. Result =P
93. Jason Castro - C HOU BA =41. Result =W
94. Jarrod Parker - P ARI BA =36. Result =N/A
95. Alex White - P CLE BA =65. Result =L
96. Scott Sizemore - 2B DET BA =NR. Result =P
97. Austin Romine - C NYY BA =86. Result =W
98. Tony Sanchez - C PIT BA =79. Result =L
99. Austin Jackson - CF DET BA =76. Result =P
100. Danny Espinosa - SS WAS BA =NR. Result =W
101. Drew Storen - RP WAS BA =92. Result =P
104. Ike Davis - 1B NYM BA =62. Result =L
105. Thomas Neal - OF SF BA =96. Result =P
108. Nick Hagadone - P CLE BA =44. Result =W
111. Travis D'Arnaud - C TOR BA =81. Result =W
115. Noel Arguelles - P KC BA =100. Result =W
116. Jason Knapp - P CLE BA =64. Result =N/A
121. Miguel Sano - SS MIN BA =94. Result =L
123. Chad James - P FLA BA =78. Result =L
131. Wilson Ramos - C MIN BA =58. Result =L
135. Andrew Cashner - SP/RP CHC BA =95. Result =L
146. Phillipe Aumont - P PHI BA =93. Result =W
HM. Peter Bourjos - OF LAA BA =97. Result =P
NR. Adam Moore - C SEA BA =83. Result =W
NR. Lars Anderson - 1B BOS BA =87. Result =W
Final Tally:
Mine::: 38 wins
BA::: 29 wins
Push/Tie::: 36
Lol/Exact: 8/100
NA: 4
33 comments
|
0 recs |
Do you like this story?
Comments
interesting
seems like you were pretty fair, but realistically there are probably way more pushes (or N/A maybe) than you listed. 80 vs 90 doesn’t matter much. and of course, it is natural to carry over some of the biases from the previous year, though there weren’t too many examples of that in my opinion.
honestly, your list looks a lot like BA’s list – so someone who studies up a bit using other published sources can come pretty close to replicating BA’s list even when he doesn’t think BA does a good job. i think it just shows how inside the box most prospect evaluators are. a list like the project prospect one, is the only real out of the box one, and maybe they’ll be proven horribly wrong (or really right) to think outside the box, or maybe their method will prove to be roughly equal, further showing what a crapshoot prospecting is.
eh
Its a cool idea, but I think all of these success comparison exercises should wait a few years
Bullpen Banter's Top 100 Prospects for 2011!
- - - - - - - -
You can find my musings at Bullpen Banter and Beyond the Box Score.
by JD Sussman on Apr 2, 2011 10:12 PM EDT reply actions 2 recs
This is kind of strange...
I don’t really get your methodology. A difference of 3 spots for a player who graduated (Logan Morrison) is a Win for you, but a difference of 6 spots for a player who fell really far down the rankings (Chris Carter) is a push? But then later a difference of 7 spots for another player who fell down the charts is a W for you (Matzek).
I’m confused. What kind of gap is significant enough to make it a W or a L?
Your list is also very prone to confirmation bias, since the grades of W, L, and P are still tied to your (updated) views of the player, not BA’s, or a disinterested third party with independent knowledge. Two players come to mind. First, you’re giving yourself a Win for being lower on Alcides Escobar, a player who just put up a 0.6 WAR last year as a rookie despite one of his primary skills (speed) being removed from his game due to batting 8th in the NL. If Escobar puts up a 2.5 win season this year, does your W turn into a L? Will this list be revisited in future years? Second, you’re giving yourself a W for Casey Kelly, who scouts still love and rank pretty equally to where he was last year despite the stats. Your ranking, which presumably puts more weight on his performance, leads to a considerable downgrade of Kelly’s status, making it a win for you. Calling it a win is circular, because it presupposes your method of ranking is superior.
I'm sure...
there are many more instances of confirmation bias, but to be honest I stopped looking closely at the list around 50.
I don’t mean to denigrate the exercise – you put a lot of thought into the list and it’s definitely interesting. I just want to point out some of the assumptions that make this a little less than an objective analysis.
I thought the same things
It appeared to me that there is a lot of bias in the analysis.
Big Sexy
Follow KBR and Dewey on Twitter! @KBRandDewey
by King Billy Royal on Apr 3, 2011 1:28 PM EDT up reply actions
agree with Slam
a bit of time lapse and using WAR might help the strength of the analysis
hmm, Carter should be counted as a loss for me
you’re right. Morrison is a win, because he is already significantly better with the bat than most expected… Might be an interesting question to ask Callis/Manuel in a chat, but I know that based on what Morrison is doing in MLB right now, I wished I had ranked him much higher… and I acknowledged the biases in the writeup…. and I asked for any opinions on it if anyone wanted to give any. The Carter ranking is a good catch.
Morrison...
was pretty much ranked the same by BA, though. That was my point. I don’t think a 3-ranking difference qualifies as a “win” any more than the fact that Drabek is 2 spots higher on BA’s list should be counted as a “win” for them. There’s just not a meaningful difference between those rankings. There’s probably a reconizable difference at some point, but two different sources could rank a player 17 and 20, respectively, and not really disagree in any meaningful way about his talent/future. You’re splitting hairs when you get to differences like that.
This isn’t a problem with you, it’s a problem with numerical lists. Some sort of tiered ranking system would be much more appropriate for this kind of analysis. Better yet, formal projections for each player would be ideal since you’d have something approaching an objective basis for comparison. But obviously since BA confines its analysis to a simple numerical list, that would render such a comparison impossible.
from a recent chat with Frankie Piliere someone asked about prospect lists
[Comment From JimJim: ]
Hey Frankie! Could you give some insights into the scouting process? Are there methods that MLB teams use that produce different results and observations than all the lists that get published?
Thursday March 17, 2011 12:51 Jim
12:52
Frankie Piliere:
The lists are pure entertainment. In the real world of scouting departments, guys are grouped using the 2-8 scale. You’re viewed as a future 5 (MLB regular) or a 6 or a 4. They’ll still be in some order but it’s much more about the grade than the order on a list.
yes, no doubt
but at some point, your pick comes up, and you are gonna have how many players ranked a 4/5/6? probably a whole lot of them. how do you choose? numerical rankings are just more finely tuned.
and regarding the second half of your post
I did try to take some general knowledge/consensus rankings into account. do you think the general public would be happier ranking Escobar 12th after the year he had? I think closer to 30 is way more appropriate and I’m pretty confident thats how the consensus would feel. I plan on revisiting this list every year… as I said before, this was just a fun exercise and obviously, its impossible to tell which rankings will be absolutely better right now. need another 10 years at least.
and regarding Kelly, I think many people have lowered him in the 2011 rankings, save maybe Keith Law [iirc]
I don’t have an opinion either way about whether 12 or 30 was a “better” ranking for Escobar. In my opinion, “better” implies that there exists some ultimate standard by which a ranking can be called “correct,” which is a premise I reject at the outset. In the 2010 off-season, both BA and you thought that Alcides Escobar looked like he was going to become a good MLB player. He had a rough transition to MLB and his team didn’t really let him use his legs, and he was still valuable. Even if I were to agree that one set of rankings can be called “better,” I disagree at a fundamental level that one season (or even 3-5 seasons) would give you meaningful information to make that determination.
These rankings are based on different valuations of different strengths. BA chooses a lot of guys who are long on tools but relatively less accomplished statistically, because they don’t want to miss out on superstars. BP’s old rankings used to swing the opposite direction, favoring certainty over upside. There are a lot of different approaches, and I hesitate to call any one of them better than any other (particularly when most peoples’ lists, including your own, use BA for 95% of their source material and just weight different things differently than BA’s authors do).
Casey Kelly is a great example. In BA’s view, he’s still a top prospect (ranked 24 in 2010, 31 now) so if you asked BA they’d call this one a push. You point to a bad ERA and low K-rate and say you were right, BA would point to scouting reports glowing about 3 plus pitches and say they were right if they were engaging in the same exercise.
Kelly
i think the scouting reports are overly optimistic. from what I have seen of him.
and I know you are probably exaggerating about the “use BA for 95% of their source material” … because I certainly do not even come close to that, and if I can avoid it, then I’m sure the experts with way more resources also won’t come close to relying that heavily on them. I have never paid BA a red cent. I think their league top 20s are awful lists, mostly because managers of minor league teams don’t necessarily make good rankings… and they usually are only helpful in finding names I might have missed.
actually, the only expert I’ve paid anything to over the past 3 years would be alex eisenberg at baseballintellect. I base mine much more on statistical analysis and general scouting reports and then videos.
you are missing the point
You thought the scouting reports were optimistic last year and this year. BA thought the scouting reports were pretty good this year and last year. Neither of you have changed your mind about how good Kelly is, so how can you say who was more right about him, even in the most simplistic way of looking at it?
by auclairkeithbc on Apr 5, 2011 6:47 PM EDT up reply actions
oh i see what you/he is saying
but I would still count that as a W… because my ranking more accurately reflects where he is… its even kinda funny that BA’s ranking of him this year is only 1 spot away from mine.
does this make sense? My ranking from last year is more accurate to where he should have been or is now. and BA even agrees with this, so that is a win.
Shouldn't Austin Jackson and Storen be losses?
I just looked at a few names, but those guys seem like they would be losses.
Big Sexy
Follow KBR and Dewey on Twitter! @KBRandDewey
by King Billy Royal on Apr 3, 2011 1:31 PM EDT reply actions
What should the ranking be?
You had Posey at 3, they had him at 7. Okay, he won the rookie of the year. But it’s still hard to say if 3rd is the right spot for him on the list.
Assuming the purpose of a list is to predict the BEST CAREER (or am I wrong?) a better method would be to wait a while and reorganize the list in terms of Warp. But even then, there is a fundamental debate as how one should optimally reorganize an old list.
When people talk about lists as being “High-upside” versus “safe-bets” the analysis should be that the predictions are simply taking different approaches to asses the same thing – likelihood to have the best career.
In retrospect, should every list that ever had King Felix on it have ranked him first over-all? Or was a list that ranked an 18yr old way down still a better list because accounting for his downside?
An interesting case is Jenrry Mejia. You obviously saw his high upside as key, while they saw the risk of a young pitcher. Regardless of how his career pans out, can we come up with a rational reason to decide which position (19 or 56) was a better decision.
Go strikeouts
by providence bruins on Apr 3, 2011 1:59 PM EDT reply actions
the purpose
is whose ranking was better. Posey is a legitimate star and plays the most valuable defensive position. ranking him where I did was absolutely better than where BA had him. thus it is a W for me. that’s just an example.
“In retrospect, should every list that ever had King Felix on it have ranked him first over-all? Or was a list that ranked an 18yr old way down still a better list because accounting for his downside?”
this is an interesting question, and one that I did think about. I just played it straight with this list. Approaching it “as a GM approaches a draft.”, which is how I formulate all my lists. because, isn’t that the main point of ranking prospects? GMs and scouting directors must weigh all the factors, upsides, and risks involved, and then make a selection and live with it. Thats mostly how I tackle any rankings/lists
This whole exercise is bizarre
The most cogent statement in the entire thread is “I think all of these success comparison exercises should wait a few years.” per jar75. The only ‘success’ worth speaking of is predicting which players will post meaningful major league performance. Who cares if a guy has a good/bad year in High-A? What’s he doing at 25?
lets not make any lists ever.
let’s not have any rankings at all. we’ll never know anything until a player has retired or died. remember when Griffey Junior was gonna break the Hank’s record? and he didn’t. we should’ve stopped there.
in all seriousness, i’ve mentioned time and again that i acknowledge the fact that we don’t know how these players will turn out. this is merely a fun exercise for me. i don’t see how its ‘bizarre’ though? is a weekly ‘hot list’ bizarre? no. it is taking a note of a player’s progress at a certain point in time. i chose one year.
is Wil Myers a better prospect now than he was a year ago? Is Mike Trout? Why is it so bizarre to take a look back 1 year prior?
remember when Griffey Junior was gonna break the Hank’s record?
I’m still cautiously optimistic about this. Keeping my fingers crossed for a Junior come back!
Also, its not Hank’s record anymore…
Okay
substitute “borderline meaningless” for bizarre. lol
You say up front that you want to determine “whether the experts are really that great at formulating rankings.” The answer to that lies years down the road, not in an annual review of where players fall on a list. The second most cogent point was " Some sort of tiered ranking system would be much more appropriate for this kind of analysis." I’m much more interested in who is on a Top 100 list than where they’re ranked, which is why C_S_J’s thread noting which players are consensus and which aren’t is more pertinent. His thread also has the added cachet of not seeking to compare his personal assessments against the experts’ based on a debatable system of ‘wins and losses’.
by blackoutyears on Apr 6, 2011 4:55 PM EDT up reply actions
borderline meaninglessness
is the 400-some odd fantasy threads on here asking “who should i keep with these last 2 spots???”
"we’ll never know anything until a player has retired or died."
nothing like hyperbole to prove a point. Yes, clearly we never know whether a player is good until they’re retired. No middle ground between that and “this list has Player X at 31 and I have him at 26…advantage me!”
by blackoutyears on Apr 6, 2011 4:59 PM EDT up reply actions
if you don't like the process thats fine
i’m not trying to fool anyone here. the thread begins with “1 year later”… so what did you expect to find?
and yes, a 5-spot difference within the first 30 spots is significant. moreso than a 5-spot difference between 95 and 100, which is moreso than a 5-spot difference between 495 and 500. whats your point?
we know that Wil Myers is significantly better than anyone thought. I had him at 51, BA had him unranked. I noticed this and decided to take a look back ay my entire list compared to theirs… what I found was that mine is quite comparable, if not better. I’ve come to the conclusion that I don’t like lists formed by many heads. I don’t think several are better than one, in this instance. and yes, for one year, its a small sample, but its a trend i’ve been noticing every year from BA and other sources. I just finally put in enough work last year for it to be anything more than luck that I compiled a better list.
I think you suffer from a pretty severe bias
And also a massive ego. You are declaring wins and loses (and N/As) based on how you feel about players, which is no different then how you ranked it last year.
you are declaring wins and losses
based on how YOU rate the players right now. but that is based, in no small part, on the same biases you used last year in the original rankings. Its like asking a student to grade their own essay test.
Lets look at Parker. you rated him in the 90s, BA rated him at 36. But somehow that merits a “N/A” even if he is universally a top 30 pick atm.
Then lets look at guys like Moustakas and Trout. Both of you are off by nearly 60 spots. just cause you had both slightly higher hardly could be considered a win for you.
Look, I get it, you want to stroke your ego. but there is nothing here of any real depth or value.
is Parker a universal top 30?
you may have a case on that one… I did compile this and the rankings before BA’s list was released, and I am far more weary of injured pitchers than I am on hitters… but you may be right on that one.
as for Trout, I was 19 spots higher on him than BA was… If you want to call that a wash just because he’s a universal top 5 prospect this year, and I had him ranked in the 60s, then I don’t think you grasp the concept of what I’m doing here.
as for Moustakas???? that also seems fair to me… for comparison let’s look at Zach Britton, whom I had at 73, and they had at 63, and I gave myself the Loss for… same 10-spot difference that I gave myself the win for on Moose. Also look at Shelby Miller, whom I had at 60, and they had at 50. gave myself the loss. How about Jared Mitchell, who I had at 49, and they had at 55? Gave myself the loss even though he was injured all year.
the basic premise is, if i pick a guy higher than BA does, and he does happen to move up the rankings, that justifies my placement moreso than their’s, and thats a win. same thing for a loss. I am looking back and these look very fair to me.
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Recommended FanPosts
Recent FanPosts
