Excellent Prospects, 100+
Colby Rasmus (120.7)
Very Good Prospects, 50-100
Chris Perez (54.2)
Good Prospects, 25-50
Jaime Garcia (49.3)
Jarrett Hoffpauir (45.4)
David Freese (39.0)
Clayton Mortensen (31.0)
Bryan Anderson (30.7)
Tyler Greene (28.5)
Mark Worrell (27.8)
Average Prospects, 10-25
Allen Craig (23.0)
Mark McCormick (15.6)
Adam Ottavino (14.7)
Blake Hawksworth (10.5)
Marginal Prospects, 0-10
Mitchell Boggs (8.3)
Tyler Herron (6.9)
Tyler Norrick (6.5)
Mark Hamilton (6.5)
Cody Haerther (2.4)
Brian Barton (2.2)
Pete Kozma (1.8)
Not to be redundant some of AZ’s earlier posts, but I wanted to go a little more in depth for those of you not familiar with how PECOTA and Upside work. This list has some major differences your scouting oriented rankings, but PECOTA obviously isn’t designed to be a replacement to scouting. Personally, I consider it another tool in the prospect evaluating toolkit; a computers way of evaluating prospects based on similarity scores and measuring their worth during the period they are under team control. BP defines it as:
UPSIDE is determined by evaluating the performance of a player’s PECOTA comparables. If a comparable player turned in a performance better than league average, including both his batting and fielding performance, then twice the number of runs he contributed above average is counted toward his UPSIDE. If the player was worse than league average, or he dropped out of the database, the performance is counted as zero.
Silver goes on to explain a little bit more from an earlier article:
Upside gives credit only for performance above league average at the player’s position, and zero credit for everything else. If a player winds up being a bench guy in the majors, or gets stuck at Double-A, or quits baseball to work in a lumberyard–none of these outcomes is desirable. On the other hand, the cost of employing a prospect is relatively low, both in terms of financial outlay and opportunity cost (a player can simply be left in the minors if he’s not good enough for MLB), so assigning negative points for a below-average or below-replacement level performance isn’t quite fair. Upside works around this negative value problem by giving credit for the good, while treating all different types of bad performance as having zero (but not negative) value. The version of Upside that we’re using here is the peak-adjusted variant, which measures a player’s most valuable five-year window up through and including his age 28 season (or simply his next five years of performance if he’s already age 25 or older). I realize that all of this is a bit complicated, and I encourage you to explore the PECOTA glossary if you’re the type that likes the dirty details. But the intuition behind our methodology is fairly simple: we’re attempting to measure the degree and probability of above-average performance while the player is under the control of his parent club. This is the real fruit of the unforgiving labor of scouting and development: getting impact performances from players who are still cheap under the reserve clause, or in arbitration.
Take Mark Hamilton, for example. He’s probably not the best guy to do this comparison with because he’s kind of sketchy, but he’ll have to do for the sake of comparison. Hamilton is the kind of player who will probably age better then Jarrett Hoffpauir. His power projects him possibly have a better career, but it doesn’t make him a favorite to produce more then Hoff while he is under team control. Mark Hamilton may likelier be a better player at 32 years old, but it won’t matter then. Both could be long gone from the Cardinals by then, because both will be able to negotiate their salaries at market price. Do you see what I’m getting at?
A few other things to keep in mind: Apparently not everyone has been run through the system, thus no Joe Mather or P.J. Walters. Or Jose Martinez, etc. Let’s go through each category and see what we got here.
Excellent prospects: Silver explains that an excellent prospect is anyone with an upside score of 100 or higher. Barring any unforeseen fluke, they are all but certain to become future stars, and even possibly future Hall of Famers. We have only one of them, and that is Colby Rasmus. Only Jay Bruce ranked higher among CF prospects, and there are those who think Bruce’s ultimate destiny is in RF. Rasmus’s most exciting comparable was Carlos Beltran. There were also a couple of flameouts-Dee Brown and Chris Lubanksi that are sort of a caution flag.
Very good prospects: These are players who have a reasonable shot at having a meaningful career, with some star potential. The Cardinals also just have one of them, and that is Chris Perez. Perez’s most favorable PECOTA comp is Mitch “Wild Thing” Williams. Frankly, I’m disappointed that I didn’t make that connection earlier. His other comps are two Yankees-Kevin Whelan and Brian Bruney. Whelan came over in the Sheffield trade and is currently ranked the #15 prospect in their system by BA. Bruney was a waiver claim, but was briefly tried out as the Dbax closer in 2005. Both have some serious control issues. Bruney was just mediocre last year in the Yankees’ pen, while Whelan dominated AA.
Good prospects: A very good player is someone likely to have a solid major league career, and a small handful may go on to be stars. Jaime Garcia was .7 points from making the “very good” category. His top comparables are Odalis Perez and Mike Hampton, which is pretty fantastic. Hampton posted ERA+’s of 107, 116, 108, 104, 122, and 154 during his seasons under team control, and as we all know ended up hitting the open market to receive on of the richest and stupidest contracts in baseball history. It seems like a long time ago, but Odalis Perez was an All Star at the age of 25. Perez ranged from middling to pretty good while w the Dodgers. Jarrett Hoffpauir’s comparables weren’t all that inspiring, Joe Lis being the best. He had an 8 year career with an OPS + of 101 as a utility infielder. The rest of his were basically career minor leaguers. David Freese’s had an interesting PECOTA comp with Jeff Cirillo, a 3B who didn’t hit for a ton of power, but hit for average and got on base. Cirillo’s OPS + when he was under team control as a Brewer was 107, 121, 106, 123 and 119 and was an All Star at 27. Another of Freese’s was Ron Coomer, who didn’t reach the majors until he was 28 and in a 9-year career posted a .734 OPS.
Clay Mortensen has some interesting minor leaguers in his comparables, including Jeffery Marquez, who rated #8 by BA for the Yankees, Lance Broadway, a White Sox ex-prospect whose star has faded, and Chris Ray, who was a good starter in the minors who became a fantastic closer in AA, and was concordantly fast-tracked to the big leagues. PECOTA loved Bryan Anderson last year, with some of his top comps being Brian McCann and Joe Mauer. PECOTA soured on Anderson considerably this time around. A couple of his top comps are Jason Belcher and Dioner Navarro. Navarro is a good fielding catcher who hasn’t quite put it together hitting-wise in the majors, but wasn’t a bad hitter in the minors. Belcher was a top catching prospect for the Brewers whose career fizzled out because of injuries. Like Anderson, he was picked out of high school, only a round later (5th). Belcher couldn’t cut it behind the plate for various reasons and was subsequently moved to the OF. He was last was seen as a 25 year old, putting up a .587 OPS in the independent Northern League. Yikes. After Anderson, it’s gets a little less interesting, so for sake of time I’ll stop there.
10-25 upside scores are reserved for players a fair chance at a meaningful career but are more likely to be fringe prospects. 0-10 is where marginal guys who are more likely to be career minor leaguers. There still are some interesting comps worth noting with some of the remaining players-
Allen Craig-Kevin Kouzmanoff
Adam Ottavino-Aaron Sele, Matt Clement.
Mark Hamilton-Dan Johnson
Brian Barton-Chris Denorfia. I still am not certain why PECOTA is so down on Barton.
Mitch Boggs-Zach Day…yuck.
And I wouldn’t read too much into Kozma’s score right now. He has the highest probability to improve of all the prospects.
Filed under: Projections, Prospect rankings
erik
azruavatar
fewgoodcards
roarke49
theredbaaron
Here’s what else Silver says in the same article:
“Are the PECOTA rankings intended to be a substitute for scouting-based prospect lists? Emphatically not. As much fun as it is to play up the scouts-versus-stats angle, I don’t expect the PECOTA rankings to be as accurate as Kevin Goldstein’s Top 100, which you will see in Baseball Prospectus 2007, or the rankings you might get from Baseball America or John Sickels.”
I point this out because I think think the Upside score, while interesting, is really sensitive to inputs….which change dramatically for players without much track record. I almost think it’s a little unfair to refer to it as “upside”, because the metric is not taking into account anything special we know about that player that might not yet be showing up in his stat line. This is a little different from PECOTA in general, or maybe just a lot more exaggerated. Look at how much the Upside will change from one year to the next, for example - guys have a strong year after a weak one, and suddenly they go from a 20 (or a 10) to a 100. Did they really just go from “average” (or worse) to “excellent” as a prospect? No, they just finally played to their potential. So referring to their potential before this as “upside”, but basing it almost entirely on performance to date (while ignoring tools) makes “uspide” seem like a misnomer to me. And, in reality, it looks like Silver agrees (based on his quote above).
I say all this because I really don’t think it’s a very good barometer for gauging the following:
Mortensen
Anderson
Greene
Worrell
Ottavino
Herron
Boggs
Barton
Kozma
These guys either don’t have a sufficient track record for the stats needle to register accurately yet, or they have some obvious caveats that the performance to date can’t indicate effectively.
I got excited when I saw the title of this post. I thought it was going to be our prospects ranked by their ceilings. It would be an exciting yet, fairly nonconstructive way to look at prospects.
erik - “His power projects him possibly have a better career, but it doesn’t make him a favorite to produce more then Hoff while he is under team control.”
to produce more relative to his position. He’s penalized for being a first basemen.
siddfynch - because the metric is not taking into account anything special we know about that player that might not yet be showing up in his stat line.
I’m not sure that’s entirely true. It’s creating an aging curve based on the stats and the comparable players. I don’t know if PECOTA takes body type into account but it conceivably could. In any event, the player specific aging curves should be more accurate than just general aging so it is capturing a certain degree of projection. Certainly not as much as Goldstein relies on projection but I wouldn’t discount it entirely.
I agree fynch. In reality, I would say that Mark McCormick has as much “upside” as any of our prospects not named Rasmus.
That doesn’t mean he’s as good of a prospect as most of these other guys, but he has the “tools” to be a #1 starter.
This list is not a great look at upside for me at all, but more a look at what they should reasonably expected to do if who they are today, is who they will always be without much improvement in their stats.
Not to say I don’t appreciate the listing, it’s just another tool to use while evaluating our guys. I’m glad that the list has Rasmus far and away #1 though, it would kill it for me if he weren’t.
AZ, it’s true that it takes secondary indicators into account, but uses proxies. And I think those proxies (age, position, demonstrated speed, and body type [I think]) are pretty useful/accurate for major leaguers, simply because they have a longer track record and there’s thus a little better baseline with which to calibrate the player.
When I say it’s not taking into account “something special”, the “something special” I mean is more than just a secondary indicator….it’s things like Perez’s or McCormick’s ability to make a small adjustment that has an upwards domino effect….or what might happen if Geo Soto lost 20 lbs (answer: upside score shoots from ~20 to 120)….or if Matt Antonelli could adjust to wood bats given a full season (4 to 60).
Don’t laugh, but I put a ton of thought into PECOTA’s “Upside” last year, when preparing for my fantasy league draft (again, don’t laugh) - I had basically turned over my roster to gain a bunch of high picks in the minor league draft, and couldn’t afford to blow it. And what I found was that the Upside for these really young guys was more useful as a reinforcement of what the stats/scouting combo said than it was as a primary indicator.
Here are some Upside scores going into March of 2007…..I am cherry-picking to make my point, but it is the exact cherry-picking I had to do in my draft. And I think the stats/scouting combo worked out much better than if I had gone with Upside score as the primary filter:
Pitchers around 100:
Slowey (124)
Igawa (104)
Inman (96)
Sonnanstine (70)
Pitchers well under 100:
jacob McGee (42)
Clay Buchholz (34)
Wade Davis (24)
I was lucky enough to take Buchholz over everyone, reasoning that he was new to pitching, already flashed better stuff than the guys at 100, was in an organization that was good at developing pitchers, and had a short statistical track record (that could be undervalued by Upside).
Hitters available around 100:
Jeff Bianchi (207)
Alexi Casilla (172)
Sean Rodriguez (129
Ben Zobrist (113)
All these guys had pretty obvious warts. The list of guys much lower that I knew I’d rather have as prospects was lengthy, and included:
Snider (74),
Colby Rasmus (68)
Carlos Gomez (42)
Dexter Fowler (32)
Matt Antonelli (4)
Chase Headley (13)
I guess I should add that I use stats extensively for my own analysis (and for my everyday job), so I am not “down” on PECOTA in particular or sabermetrics in general….and Upside did a really good job pegging guys that everyone knew were really good….Longoria, Gallardo, Ryan Braun, and even took an accurate flyer on the enigmatic Tim Lincecum (206). But it also put some guys up there that were way too high, and (more importantly for the Cards prospects), failed to identify some guys like Buchholz or Antonelli or Chris Nelson. And that’s what led me to post my list of guys like Greene, McCormick, Herron, Barton, etc. above.
OK, I’m done with the long posts. Thanks for listening.
Hey boys, i was just checking one of my sites with draft predictions and it said we would take Aaron Hicks. Just wondering if anyone knows anything about him?? thanks
I wouldn’t take PECOTA as gospel. I think there is a laugh test that it must pass first. Like when it rated Anderson above Rasmus last year, it’s obvious that while Anderson was very good, Rasmus was the best we had.
Sidd — I understand where you are coming from. I don’t think PECOTA can ever do as good of a job, on balance, as a scout or analyst who closely follows players. As a purely objective measure though, PECOTA is the best there is at this point. It tries to take things into account that allow it to individualize projection but as you point out there are a lot of things that really can’t be measured, which inhibit a truly accurate projection. At the same time, it’ll hit on guys like Pedroia who may be overlooked by subjective analysis.
Whither Jose Martinez? Slick-fielding 21-year-old shortstop hits .300 in AA, with as many extra base hits as strikeouts…I’ll take three, please!
I wonder how PECOTA’s projections compare to Clay Davenport’s Translations. I ask because (1)they’re both courtesy of Baseball Prospectus, and (2)Davenport’s system likes Allen Craig and Jarrett Hoffpauir *very* much–and is pretty positive about Martinez, as well.