In undertaking this project, I fully intended to chart one full game of each of the notable QBs. After about 2, it became obvious that I simply did not have the time to do so. So, I charted one game of each of the big 5, watched multiple other games from them, and watched multiple games of each other QB. In charting games, I followed the following rules:

1. I used the game tape in which the QB’s passer rating most closely matched his season passer rating, as long as DraftBreakdown had it. Some small exceptions were made to get enough good tape to use.
2. I charted only passes intended for a receiver. Throwaways, sacks, scrambles, or any other play that was not a pass intended for a receiver was not charted. My only interest was success on passes intended for receivers. Throws with penalties were counted unless the receiver was completely taken out of the play.
3. Each pass was charted for distance (short, medium, long), throw quality (bad, average, good), openness of receiver (open/not), and pressure (pressure/none). Occasionally there is some grey area or subjectivity, but I did my absolute best to keep an objective, consistent standard. Some plays have .5 where there was uncertainty as to how it should be charted, but I tried to keep these to a minimum.
4. A Bad throw is any throw that was uncatchable or ill-advised. Average is any throw within a reasonable catching radius, and more throws should end up in this category than the others. Good is a throw that is on-time, on-target, and precisely where it is supposed to be.
5. Decision-making was factored into throw quality, but only when the tape made it possible to judge. Since this is not all-22 tape, it is not possible to judge on most plays absent obvious factors or replays. Sorry.
6. Notes were be made on every throw. As the categories are rather big, this is intended to give insight as to why the grade was given.

Link to charting

There are 5 tabs there, the information there should be fairly self-explanatory – a link to the game, the stats for the game, the charting, and a summary. Here are some overarching concepts that the tape showed:

It’s incredible how complex every play is

I just want to start with this. To chart even the most simple of plays required 6 or so viewings. To chart some of the more difficult plays required 20+. There’s just so many moving parts on literally every play and it’s so easy to miss things. Sometimes there’s late pressure from somewhere you’re not looking. Sometimes a receiver breaks off a route after the ball is already thrown. With 11 guys on the field and at least 7 offensive players and 6 defensive players involved in every play, it’s very easy to miss things.

By the same token, viewing everything once, it is very easy to miss what “really” happened. Just something to keep in mind.

Do not treat stats as gospel

I do not think they should be completely ignored, but they definitely should not be relied upon. Completion percentage is significantly impacted by both pressure and drops. Baker faced very little pressure, while many of the other QBs were constantly on the run. Lamar Jackson’s receivers dropped 3.5-5.5% more passes than the other QBs because his receivers were terrible. Would he have accuracy questions if he completed 63% of his passes instead of 59%? It shouldn’t matter.

It does make it difficult to objectively evaluate these guys though because they had different advantages and disadvantages and we don’t know how they would look in different situations. Would Rosen look like a top QB behind Louisville’s line? Would Baker have some of the best numbers ever on a less talented team? The answers to these questions are purely speculative, but they should surely be factored in to any evaluation.

Each QB has throws they are better and worse at

As an Eagles fan, I got to watch this season as Doug Pederson completely redid his offense on the fly because Carson Wentz and Nick Foles were good at entirely opposite throws. Not all throws are created equal. Some QBs are better to the middle, some are better to the outside, some are better short, some mid, some long, some are better on timing plays, some are better on sit plays, and so on and so forth.

Why does this matter? Well, I’m not sure any of these guys played in systems that truly emphasized their strengths and minimized their weaknesses. Darnold and Allen were much better throwing medium routes but were constantly asked to throw lateral behind-LOS balls that they simply weren’t good at. Rosen and Jackson’s ability to hit receivers on slants and crosses was not exploited in favor of more sit and hook routes. Jackson didn’t run a single read or RPO.

Whatever team drafts each of these QBs better have a good understanding of their strengths and weaknesses and tailor their offense around them. You wouldn’t buy a towel and complain that it’s a bad blanket. Don’t draft a QB unless you plan on building to him.

Each QB has glaring flaws and we honestly don’t know which will improve and which won’t

There is no such thing as a perfect prospect. Each person (and team) will have to decide which flaws they can live with and which they can’t. It is not safe to assume that with “better coaching” or “more experience” or whatever else that a QB will magically turn their weaknesses into strengths. The important thing here is to be realistic about all prospects. What happens far too often is that some prospects have their strengths highlighted while others have their weaknesses highlighted, and then you end up with a biased, useless view of both of them. And while it’s common in NBA scouting, age is not a common factor in NFL scouting despite it potentially being a huge factor in development.

Conclusion

No matter how much analysis and tape study is done, when it comes down to it, there are so many factors at the next level that contribute or detract from success that every QB has a chance to succeed and a chance to fail. Ultimately, the goal is to try to identify the guys who are most likely to succeed. No analyst is going to be 100% on prospects for so many reasons. But over a period of time, a good process will lead to better results than a bad process. Holding out one wrong projection is bad because it is impossible to be perfect.