2016: What We Know We Think We Know But Don’t

I started this series with an article about fallacies. With a month to go before the draft, I want to write a similar article, this time regarding the most common cognitive biases in prospect evaluation. But, as we’re starting to get close, I’m actually going to talk about some specific prospects in the process. Let’s get right to it.

What is Cognitive Bias?

A minor in Psychology and a quick perusal of Wikipedia does not an expert make me, but the concepts I’m covering here are fairly simple and well-accepted, so I’ll provide links where relevant and if you want to do further research on these, please do. These are fascinating to me personally, so hopefully you find it as interesting as I do.

“A cognitive bias refers to a systematic pattern of deviation from norm or rationality in judgment, whereby inferences about other people and situations may be drawn in an illogical fashion…[C]ognitive biases may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or what is broadly called irrationality.”

There’s…well…there’s a lot of these biases. I’m not going to cover all of these, nor am I going to point out every player they apply to. Some players I’m going to bring up multiple times, while many will go unmentioned. I use them to provide concrete examples to concepts so that hopefully we all get a little smarter and better in evaluating prospects. That’s what this is all about, after all.

Bandwagon Effect ft. the vast majority of people and websites

Okay, so I’m going to start not with a bias regarding specific players, but regarding prospect evaluation in general. I’ve said this in many places before, but I’m going to repeat it: prospect evaluation is really fucking hard. You’re going to be wrong more often than you’re right. The goal is to be less wrong more often, not to be right 100% of the time. So why is it that if you look at five top sites, you get this?

NBADraft DraftExpress Chad Ford CBSSports SI
Simmons 1 2 1 1 1
Ingram 2 1 2 2 2
Bender 4 3 5 3 3
Murray 5 6 3 6 4
Dunn 6 4 4 4 5
Hield 3 7 6 8 7
Brown 7 5 10 5 9
Poeltl 9 8 9 7 6

That is truly insane. 8 players appear in the top 10 of each of their big boards. If you go deeper, you will find very few players where there is disagreement by more than 15 spots or so. What should it look like? Well, DX actually had a feature last year on multiple analytical systems. And as you can guess, it looked, well, very different. Counting the five models, 3 different players ranked #1, 3 different players ranked #2, 4 different players ranked #3. At the #4 overall player, there was a 9 rank split. At #9, there was a 17 rank split. At #12, there was a 37 rank split. At #16, there was a 57 rank split. The thing is – that’s what should be expected. After the top players in any draft, there should be massive disagreement – and that only works when the top players in the draft match the preseason expected top players.

19 players drafted in 2015 played 1K+ minutes last season:

WS/48 Rank BPM Rank Average Rank
Towns 1 1 1
Cauley-Stein 2 2 2
Richardson 4 3 3.5
Porzingis 5 4 4.5
Kaminsky 6 6 6
Turner 3 10 6.5
Nance 7 7 7
Winslow 11 5 8
McConnell 10 9 9.5
Lyles 8 12 10
Portis 9 15 12
Russell 17 8 12.5
Booker 12 14 13
Johnson 16 11 13.5
Hezonja 15 13 14
Okafor 13 17 15
Grant 14 16 15
Vaughn 18 19 18.5
Mudiay 19 18 18.5

The DX article does not include non-college prospects. After seeing how I did on them last year and in 2014, I will not be including them going forward. I tried to apply my statistical analysis to non-college stats, but it hasn’t been effective. It is also a good time to note that where a player goes and opportunity to play are probably the 2nd and 3rd biggest factors in ultimate outcome, because many deserving players get put in no position to succeed or are never even given an opportunity.

Looking only at the college players on the list, BPM and DX were the closest to the top 2, with Towns 1 and WCS 4. BPM had Kaminsky 3. Turns out that “college BPM” may be a better predictor than anything else. It also was highest on TJ McConnell by far, and lowest on Vaughn and Jerian Grant. Was BPM perfect? No, of course not. But as a whole, it actually looks, well, pretty fuckin’ fantastic. Even its #2 player, Delon Wright, would have been 4th if qualified for the minutes limit.

But go back a year, and you see, well, it’s not pretty. Among players who entered that draft, Embiid (injured) was #1, Marcus Smart (good!) was #2, Justin Jackson (played well in D-L, now in Finland) was #3, K.J. McDaniels (bad and barely playing) was #4, Shabazz Napier (truly terrible) was #5, Jordan Adams (barely played rookie season + injured this season) was #6, and, well, it doesn’t actually get much better from there, unless you’re looking for guys who probably deserve an NBA look in some capacity (somebody sign Khem Birch already!).

Okay, that was a long tangent to make the following point: no system is perfect, and the effectiveness of any system can vary wildly from year to year. What is clear is that every system will have major hits and major misses. There is quite frankly no reason why five different websites should have anything close to the same big board. Player rankings should vary wildly, especially after the top guys. I know I have been very strident on Simmons over Ingram, but there’s at least one player who could reasonably be ranked above him – it’s just not Ingram. I’m not saying “throw darts at a dartboard”, because it is nowhere near that random as a whole, but I am saying that if you’re going to be in the business of making big boards, you better be rationally backing up your rankings, and I see a whole lotta irrationality, which is why I’m writing this article to begin with.

So what’s the irrationality at play here? The Bandwagon effect. “The bandwagon effect is a phenomenon whereby the rate of uptake of beliefs, ideas, fads and trends increases the more that they have already been adopted by others. In other words, the bandwagon effect is characterized by the probability of individual adoption increasing with respect to the proportion who have already done so. As more people come to believe in something, others also ‘hop on the bandwagon’ regardless of the underlying evidence.” Put simply, the more people who declare something is okay to believe, the more people that will believe it. This is basically how these generally “consensus” big boards end up being put together. A few sites declare the order, and then people believe that to be true, and so it “becomes true”. Only it doesn’t. Avoid jumping on the bandwagon. Draw your own conclusions. As I pointed out last week, when you actually start reading the underlying evaluation of prospects, it sometimes doesn’t even match the rankings.

This is all a long-winded way of saying “don’t join the consensus just because it’s consensus”. Do your own research. Draw your own conclusions. And don’t be bound by what is acceptable, because what is acceptable often isn’t right anyway. Just be prepared to support whatever you conclude.

Anchoring Bias ft. Jaylen Brown, Skal Labissiere, and Denzel Valentine

Whew, okay, that out of the way, let’s move to a related bias – anchoring. “Anchoring or focalism is a cognitive bias that describes the common human tendency to rely too heavily on the first piece of information offered (the “anchor”) when making decisions. During decision making, anchoring occurs when individuals use an initial piece of information to make subsequent judgments. Once an anchor is set, other judgments are made by adjusting away from that anchor, and there is a bias toward interpreting other information around the anchor. For example, the initial price offered for a used car sets the standard for the rest of the negotiations, so that prices lower than the initial price seem more reasonable even if they are still higher than what the car is really worth.”

What’s the difference between this and the bandwagon effect? With the bandwagon effect, more people believe something because it is declared. With anchoring, people are more hesitant to believe something that is wildly different than what was previously believed. Basically, once the first mock draft is done, anything wildly different than that isn’t accepted. And that is, to put it bluntly, really really stupid. This is the most common bias I see on an everyday basis. It usually manifests itself in comments like “how can you rank X so high when nobody else does” or “how can you rank X so low when everybody else does”. Ranking somebody 10th when everybody sees him 5th is seen as an “acceptable” difference of opinion. Ranking somebody 100th when everybody sees him 5th is seen as a sign that a person is unfit to be taken seriously.

Please, I’m begging everybody – stop thinking like this. This is the single biggest obstacle in prospect evaluation and the biggest contributor to really obvious bad picks and inexplicable prospect drops. On Ford’s Big Board 1.0 (the only Big Board I can find from August), Labissiere and Brown were 2 and 3. Labissiere was mainly bad this year – he showed flashes of potential, but nothing more than flashes. Brown was atrocious this year. Labissiere has a weak argument for remaining where he is, but he is far more typically of a late first round prospect than a lottery prospect. Brown really has no argument for being more than a second round flier, yet remains top 10 on every board. Why? Other than anchoring, there truly is no explanation.

On the flipside are guys like Denzel Valentine. Valentine was absolutely incredible this season. But he started outside the top 30, and when you’re there, it’s hard to move up to where you deserve to be ranked. Sometimes it does happen (like with Buddy Hield), but far more often, guys end up ranked lower than they should only because they weren’t ranked highly to start the season. It is very, very easy to argue Valentine over Hield, as Valentine was nearly as good a shooter while being a far more complete player. Why then is Hield ranked higher? Well, from the way it looks reading through Ford’s Big Boards, Hield was playing amazing while big board rankings were still volatile while Valentine was injured. Seems bad to rank prospects this way, doesn’t it?

So please, please, PLEASE stop thinking in terms of “a player is ranked X, any ranking far from that is clearly wrong”. I’m not asking everybody to agree with my evaluations. In fact, you should probably disagree with some number of my evaluations if you read that first part on bandwagoning. But insert whatever players you feel are appropriate. And if you don’t feel any players fit here…well, then I’m not sure you’re doing much evaluating at all.

Confirmation Bias and the Backfire Effect ft. Andrew Wiggins and Jahlil Okafor

Okay, my flameshield is already at the ready. But I’ve never been one to shy away from controversial opinions, so here goes: Wiggins and Okafor are two prospects who were undeservedly anchored at the top of the draft who were bad college players who have been mediocre (Wiggins) or worse (Okafor) pro players. KEEP READING PLEASE DON’T CLOSE THE TAB. I’m not interested in rehashing why they were overrated. What I’m interested in is looking at why people are so stridently defending them as NBA players. And that brings us to confirmation bias.

“Confirmation bias, also called confirmatory bias or myside bias, is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s preexisting beliefs or hypotheses, while giving disproportionately less consideration to alternative possibilities. It is a type of cognitive bias and a systematic error of inductive reasoning. People display this bias when they gather or remember information selectively, or when they interpret it in a biased way. The effect is stronger for emotionally charged issues and for deeply entrenched beliefs. People also tend to interpret ambiguous evidence as supporting their existing position.”

Now, you may be saying “BusterDucks, if you don’t like that, aren’t you going to be using the same confirmation bias?” And, well, all I can say is that the stats I use to judge players I’ve been using for a while and I use it to judge all players, not just these guys. I try to stay consistent in how I evaluate players, it’s the only way I really know how to eliminate these types of biases. So let’s go to the tale of the stats:

Wiggins: (SF) WS/48: 36/59; (SF) BPM: 44/59; (SG) RPM: 46/81

Okafor (C): WS/48: 48/49; BPM: 47/49; RPM: 77/77

By these metrics, Wiggins looks like an average backup wing, while Okafor looks like…well, let’s just say he didn’t belong in the NBA last season. It is amazing how many explanations (read: excuses) people make for these rankings, and how strongly they defend their guys when presented with evidence that they may not be future superstars. “The ‘backfire effect’ is a name for the finding that, given evidence against their beliefs, people can reject the evidence and believe even more strongly.” This is basically what happens. Rather than accept the evidence and seek to understand it, people tend to see the evidence, ignore it, and substitute their own reality.

Look, I get it. Wiggins and Okafor both score a lot of points. You don’t need to tell me that. Wiggins is very good at getting to the rim and the line. Okafor wasn’t particularly efficient, but he has some individual scoring game. If basketball started and ended with how well you could score with the ball in your hands, Wiggins would be pretty good and Okafor may belong on an NBA roster. But there’s more to basketball than that, and while people get excited over every single good game, ignoring all of the bad games is no way to judge players. I actually like Wiggins in certain types of roles that he hasn’t been put in, and think that he could show much better if treated as the player he is rather than the player people want him to be.

I could probably write a whole article just on this, but this is the draft series, so why am I actually bringing this up? Well, most people project prospects by seeing what is working in the NBA and then looking for which college players have those attributes. The problem with this method is that it requires being realistic about what is working in the NBA. Otherwise, you do things like continue to project mediocre superathletes to be worth high draft picks (sorry Jaylen, I’m picking on you again) or project individual offense only bigs to be worth high draft picks (if you really like Jahlil Okafor, just draft Diamond Stone, he’s the same prospect with a lower anchor).

It’s also relevant because of the way it interacts as an amplifier for other biases, such as…

The Serial Position Effect ft. Ben Simmons and Buddy Hield (guest appearance by Actor-Observer Asymmetry)

Serial position effect is the tendency of a person to recall the first [primacy effect] and last [recency effect] items in a series best, and the middle items worst.” I have found that, as far as the primacy effect goes, it shows most in early season games disproportionately affecting draft stock, helping anchor a prospect higher or lower than they otherwise should simply because they had their good games early enough. The recency effect can best be shown by comparing the last game of the season for two high prospects:

Simmons: 31 MP, 4-11 FG, 2-7 FT, 10 points, 12 rebounds, 1 assist, 1 steal, 1 block, 3 turnovers

Hield: 36 MP, 4-12, 1-8 3P, 0-0 FT, 9 points, 7 rebounds, 2 assists, 2 steals, 4 turnovers

Since those last games, a lot of people have talked about how Simmons “isn’t competitive” or “quit on his team” or other, well, complete nonsense. And I know it’s complete nonsense, because Simmons carried that LSU team as far as it could possibly go, and because Hield put up basically the same game in his last game and nobody is calling him less of a competitor. People believed Simmons lacked competitiveness, so that last game confirmed it (see, amplifier). People believed Hield was a “gamer”, so a bad game was just a bad game (it didn’t confirm a pre-existing notion, so it was pretty much just ignored). As an aside, non-Simmons players in that game shot 9-52. That’s 17.3%. LSU got blown out because nobody on LSU could make a shot, not because Simmons “quit on the team” or anything like that.

In fact, there’s a name for this bias as well! “Actor–observer asymmetry (also actor–observer bias) explains the errors that one makes when forming attributions about the behavior of others. When people judge their own behavior, and they are the actor, they are more likely to attribute their actions to the particular situation than to a generalization about their personality. Yet when an observer is explaining the behavior of another person (the actor), they are more likely to attribute this behavior to the actors’ overall disposition rather than to situational factors. This frequent error shows the bias that people hold in their evaluations of behavior.” Put simply, nobody should be judging players based on their personality or attitude or anything like that.

All games matter. I’m not going to say all games matter equally, because level of competition certainly matters, but games don’t matter more or less because of when they happen. Look at all games when evaluating prospects, not just some of them or the most memorable ones.


These are just a few of the dozens of biases that exist. Some of these biases are specific to basketball or sports in particular (biases based on age (ft. Denzel Valentine), height (ft. Kay Felder), or other physical factors), while others are more general. Even objective systems are going to be created with some bias built in. Everybody should strive to become aware of and remove bias as much as possible. Humans will never be perfect thinkers, but it doesn’t mean we shouldn’t strive to be *better* thinkers. And it’ll make us better prospect projectors in the process!

One month until the NBA draft, which means four more weeks of articles. If anybody wants me to cover something in particular, whether it be specific individual prospects, general concepts, whatever ya have in mind, now is a good time to throw it at me. I’m considering doing a grab bag next week (responses to responses to previous articles, updates to previous articles, random thoughts that don’t warrant full articles, etc.), so just let me know and I’ll see if I can fit it in.