You should stop looking at numbers and actually watch the games. You have no idea what you’re talking about.

*sigh*

If nobody ever said this again, the world would be a better place. There is not a single numbers guy out there, myself included, who does not factor in watching games to some extent. Just because somebody disagrees with your opinion of a player doesn’t mean they don’t watch, and just because they haven’t watched as many games as you doesn’t mean their opinion is less valid. The eye test is a good one, but to get the right answers, you have to ask the right questions.

So, while I’ve spent many many words on this site, LibertyBallers, reddit, and elsewhere talking about my statistical system, I’ve never actually explained how I personally use the eye test. After all, it makes up a not insignificant portion of player evaluation. I still start with the numbers, but the tape explains the numbers, and if they don’t, then there can be arguments, but it does not make one side more correct than the other – as with many things in life, the truth lies somewhere in the middle.

First, and most importantly, I start with a simple but critical premise: what makes a player good is not what he can do, but how consistently he can do it.

There are literally hundreds of players coming out of college every year who can make amazing plays. The guys in the D-League are capable of amazing things. But being capable of making amazing plays does not make you an amazing player. There are a ton of guys who are stupid athletic and capable of doing insane things and also not good NBA prospects because they simply lack the skills and/or consistency necessary to become NBA players.

In fact, I even have a name for guys who make incredible highlights but lack consistency: Vine All-Stars. I guess the name is outdated now with Vine disappearing, but the point stands. These are guys who look great if you only ever look at highlights, because the good plays they make are awesome. But there are approximately 195 possessions in a basketball game. If you make amazing plays on 5 of them every game, that still leaves 190 possessions that matter just as much. It is these more typical, standard possessions that define how good a player is.

Everybody knows small sample size applies to stats, but it applies equally to the eye test. A highlight tape or a highlight play simply is not useful in evaluating a player. The amount of times I’ve heard “does a bad player make that play?” or “do you still think he’s bad” after a player makes one good play is greater than 0. It should be 0. An NBA player, even the worst NBA player, should be capable of things that make us go wow. But that is no way to judge a player.

The average college player plays between 1,500 and 2,500 possessions in a season. Some of those possessions were awesome. Some of those possessions were awful. For scouting purposes? They’re worth noting but not worth focusing on. What’s much more interesting is all of those in the middle, because most possessions are not highlight OR lowlight worthy, but have a much bigger impact on the game by volume. That’s what I focus on, because it’s those rote, common plays that really point to a player’s skill level.

When scouting, most people focus on the player with the ball and the player defending the ball. That’s where the action is. And what happens there definitely matters. But most of the action takes place away from the ball, and for everybody other than PGs, that action is much more relevant to projection than what they do with it. On the ball is easy to look for – how well can a player dribble, hit pull-ups, pass, etc., and on defense, how well can a player navigate screens, stay in front of his man, use his hands, etc. But off the ball is where most players make their living.

So, on offense, I look for quality and timing of picks and cuts, positioning, and finding open spots and getting ready to receive passes. These are the things that make an offense work, and players who understand these nuances of the game and these secondary skills help an offense and show that they have an understanding of what they will need to do at the next level.

On defense, I look for ability to track off-ball movement, help and recover (for wings), help in the lane (for bigs), ball-man positioning, and navigation of pin-downs, flares, and other off-ball screens. It is a little harder to translate this to the NBA, as many NCAA teams run schemes that are not legal in the NBA, and many NCAA offenses don’t ask the same questions of the defense that NBA offenses do.

I also watch rebounding, a facet of the game that is too often ignored given its importance and that there is a rebound on more than 50 percent of all possessions. Rebounding is much more than size + length + jump. Boxing out is a skill. Reading a shot in the air and positioning for the bounce is a skill. Sneaking in to crash is a skill. These are all important to being a good NBA player, yet they are too often ignored.

I try to watch 2-4 games of as many prospects as I can. While I would like to watch more, I don’t have the time (and neither does anybody else not getting paid to do it!) Why 2-4 games? Because that is generally a large enough sample to see athleticism, potential top level ability, and everything mentioned above. I use this limited eye test to supplement the stats to try to get the strongest projection of a player I can. While watching more games of a player may give a more accurate picture of that player’s capabilities, if you’re not looking for the right things, watching every game still won’t make a difference. There’s also a tendency to judge players who you watch a lot differently than players you only watch a few times. This tends to manifest a bias for or against the players you watch simply because there is a stronger opinion about them than other players. Watching everybody roughly the same amount can eliminate some of this bias.

There’s a lot more to basketball than just your good/flashy plays. When watching games, look at all facets of a player’s game – to do otherwise can only lead to an incomplete picture.