It’s a fact of the draft projection business: people will look back at every big board you’ve ever done and point out every mistake you’ve ever made. They will question every projection out of line with the consensus. They will point out how much you’ve failed. Back in 2015, I wrote about the challenges in prospect projection. That was after doing it officially for just one year. Two years later, I’ve learned a whole lot more, and it’s time to revisit just why this is so damn hard.

1. Judging players is ultimately subjective.

I tend to fall back on BPM and WS/48, but those stats are highly imperfect. Last year I started messing around with some numbers to try to figure out what kind of positional adjustment needed to be applied to them – it’s pretty significant. Coaching and teammates influence them. Role highly influences them. They also value certain skills more than others. Traditional stats and traditional opinion have value. How much value? Depends who you ask. I don’t particularly value them, but some people point to 20 PPG and say “how can you say the player is bad?”

Looking at any big board, it’s important to keep in mind that you may evaluate or value players differently than the person who created the board. It doesn’t make one of you more right than the other, but it helps to keep an open mind as to all ways players can be evaluated when judging projection success.

2. It takes a minimum of five years to competently assess a draft class.

Players improve in fits and starts up until they are 23-24 years old. Many players are drafted as 19 year olds. See the problem?

If only people were so patient.

The first draft class I officially ranked was 2014. Only a single player drafted in the top 10 of that class has played their age 23 season (Stauskas). We still have no idea how that class is going to turn out. So far, it’s been pretty abysmal, with Nikola Jokic the only player who has made any kind of real impact. That will change in the next 2-3 years. But most of the guys taken high have just finished their age 21 season. They still have a lot of growth left. I think we’re just now getting to the point where we can look at the 2012 draft class with a strong idea of the talent of every player.

I know it’s hard to be five years patient, but judging any earlier should really just result in an “incomplete”.

3a. Because it takes five years to assess a draft class, it takes a long time to learn lessons.

3b. Because it takes five years to assess a draft class, sometimes the NBA changes and what was projected is no longer relevant.

These two points go together. This is my 4th draft doing this. That means that anything I did wrong in previous drafts…I still wouldn’t really know for certain!

I consume as much draft and basketball content as I can, because we all learn from each other. It really is the only way to learn. Of my 2014 top 12, 5 have suffered major or career-ending injuries and a 6th did not come over immediately. The already limited data is more limited. I try to identify mistakes I make, but it’s just so hard to tell. And you only get one chance to practice each year.

The confounding factor is that the NBA is changing at such a rapid rate that what was projected in 2014 is of only questionable value. What was projected then may look bad, but it may have made perfect sense at the time. For example, players like Elfrid Payton and Aaron Gordon were highly ranked by both scouts and stats. Having them in the lottery was completely defensible. Now? Elfrid Payton would probably project as a late first at best, and I would be projecting Aaron Gordon as a center. The 2014 projections weren’t wrong. The NBA just completely changed in the 3 years since they’ve been drafted. Does that mean the big board was bad?

I have made major changes to my formula and projections every year to try to account for the changing landscape, but I cannot predict the future. Neither can any other draftnik. And by the time we realize what we’ve done wrong, it’s already changed.

4. The single most important skill can’t be scouted in college and is one of the last skills to develop.

NBA players need to shoot the NBA 3.

NCAA players shoot from shorter than the NBA 3.

Welp.

Basically, if a player (other than a shot-blocking C) cannot shoot the 3, their NBA value tanks. But they don’t shoot the shot in college, and they typically don’t develop it, if at all, until they’re between 21 and 25. Yes, that’s a broad range. Yes, that goes even for the best college 3 point shooters. If a guy unexpectedly develops a shot, most people will end up having him ranked lower than he should have been. If a guy expected to shoot well fails to do so, most people will end up having him ranked higher than he should have been. And this development can be greatly affected by the team a player is drafted by and the coaching received.

This is the defining factor of whether 80% of NBA players succeed or fail at the next level, and there is simply no great way to project it. My big board presentation this year is taking this into account significantly more than usual, but if you look at a board and it seems off, this is probably why.

5. Variance.

Sometimes, unpredictable shit happens. Players get drafted to organizations who don’t teach them properly or don’t give them playing time. Others get drafted who use them in a way most others wouldn’t. Players mature as they turn from college age to adult age. Seriously, think about you at 19 vs. you at 22. Huge difference, right? Now you at 22 vs. you at 25. Huge difference again. Those of you who are younger – you won’t realize how much you’re changing until you’ve already changed. But it’s not predictable how these guys will change. Some guys have incredible work ethics. Some guys don’t really like playing. Some end up finding partying more interesting than practicing.

Basically, remember that in the end, this is still guesswork. Educated guesswork. Informed guesswork. But still guesswork.

Conclusion

A consensus tends to build every year as to where guys should be ranked. Ranking guys significantly differently than the consensus can draw questions or ridicule, especially when the rankings end up looking particularly wrong a year or two in. But it doesn’t mean the rankings were wrong. The NBA changes quickly. Players hit or bust for unforeseen reasons. Don’t focus on the players who were highly ranked who busted, or at least if you do, also focus on the players who were highly ranked and hit and the players who were lowly ranked and didn’t. Take all big boards and projections as best guesses – interesting to look at, interesting to read, but not something to be treated as 100% accurate. Not even close. Shooting 36% from 3 makes you a good NBA 3 point shooter. Shooting 36% on rankings would make you a good NBA prospect projector. It’s just that hard.