This year will offer a glimpse of how the playoff committee will adjudicate the SOS issue, as well as the follow-on political considerations that come with the number of spots allotted to each conference.

In years when Notre Dame qualifies, expectations are for allocations to track as follows:

1. SEC - 4 bids

2. Big Ten - 3 to 4

3. ACC - 1 to 2

4. Big 12 - 1 to 2

5. Notre Dame

6. Highest ranked G5 conference champion

One of the biggest hurdles non-P2 (SEC and Big Ten) teams will face is the perception of the strength of those leagues, and how that perception will guide discussions about strength of schedule.

Making apples to apples comparisons between conferences, which was already difficult, will be all the more challenging this year with conference realignment leading to less non-conference matchups between P4 teams.

If Virginia Tech is really a playoff contender, as I believe the Hokies are, then that begs the question: How many games does the team need to win to secure a playoff berth?

Projecting wins and losses based on SOS

To get a sense of where the top teams are likely to land, record-wise, I ran some calculations that look solely at the SP+ top 25 and group win chances by the opponent’s strength.

Ranked opponents were bucketed by fives (e.g., #1-5 nationally, then #6-10, etc.) for the top 25, while unranked opponents were labelled either P4, G5, or Other.

For each team, I counted the number of opponents in each strength grouping. Those totals were then multiplied by the assigned win chances, which depended on the team under consideration.

For demonstration purposes, here is how top ranked Georgia looks:

The Bulldogs will face six teams ranked in the SP+ top 25, two of which are in the top 5. That’s pretty brutal!

In contrast, preseason #7 Penn St. will face only two top 25 teams. One Nittany Lion opponent is ranked in the top 5, while the other is between 21 and 25.

So you tell me, if both teams end the year with a 10-2 record, whose getting the higher seed and/or last remaining playoff birth?

Georgia, right?

Maybe.

It depends on how all the other SEC teams perform. Five are ranked in the top 10, six in the top 12.

Political considerations mean that it is highly unlikely that one conference will get more than four bids.

Imagine a world in which the SEC gets five teams in, the Big Ten gets three, and Notre Dame punches a ticket. That would leave the ACC, Big 12, and the entire G5 with three spots, one for each.

Under this hypothetical, I could almost guarantee that within 72 hours somebody would file an anti-trust suit. I’m not saying they’d win, but I doubt anyone has a taste for the kind of ugliness that such a move would generate, especially if one considers the 2023 Florida St. snub.

All of this is to say that there is great uncertainty about how teams will be seeded. With so much good-on-good lined up in 2024 during the regular season, things could get muddled in a hurry.

The good news for the Hokies is that there is no clear elite team in the ACC. The bad news is that even if the ACC nets two bids this year, Tech will still face an uphill battle to secure one of them.

What it will take for Virginia Tech to make the playoff

SP+ includes four ACC teams among its preseason top 25. Based on schedule strength, their forecasted records (based solely on my SOS model) are as follows:

• #12 Florida St.: 9.4 wins, 2.6 losses

• #14 Clemson: 9.5 wins, 2.5 losses

• #19 Miami: 9.5 wins, 2.5 losses

• #23 SMU: 8.9 wins, 3.1 losses

Extending the methodology down to Virginia Tech, which is #32 in the preseason SP+ ranking, yields 7.7 forecasted wins for the Hokies and 4.3 losses.

But what if SP+ is wrong?

Tech is #28 according to the coaches, which would mean 8.1 wins and 3.9 losses.

The Hokies’ highest preseason ranking, #16, which comes from Phil Steele, translates to 9.1 wins and 2.9 losses.

All of this leads me to the following conclusions:

1. The ACC champion is likely to go 10-2 in the regular season, therefore entering the playoff with an 11-2 record and a better-than-even chance at securing the #3 seed, even though the team is likely to be ranked around #10 by the Associated Press.

2. The team that losses the ACC championship game will have a minimum of three losses and potentially four, either of which will probably eliminate it from playoff contention.

3. At baseline, Virginia Tech is an eight-to-nine win football team in 2024. That should be good enough to keep the Hokies in the playoff conversation for most of the season, but from the outside looking in.

Putting it all together, in order to make the playoff, at minimum, the Hokies need to:

• Win 10 games, which means stealing a game or two that they would otherwise be expected to lose, the most likely of which are at Miami and at Syracuse

• Go undefeated in the four non-conference games

• Beat Clemson or Miami, and the team they beat needs to play in the ACC championship

• Win the ACC championship game if they play in it

Concluding thoughts

Virginia Tech has, on its face, a pretty light schedule. The only opponents in the SP+ preseason top 25 are Clemson and Miami, and the Hokies get the Tigers at home.

There are, however, a number of tricky road games the Hokies will have to navigate, including the aforementioned trip to upstate New York as well as the cross-country venture to Palo Alto, California to play Stanford.

On paper, a loss in either game would look pretty bad. And like it or not, looks matter when it comes to selecting playoff teams.

If we assume that both of those games will be nerve-rackingly close, then the difference between making the playoff and going to a mid-tier bowl will be Tech’s ability to grind out ugly wins, maybe via a late field goal or defensive stand.

With almost all of last year’s starters returning, as well as excellent special teams units across the board, Tech is positioned to outperform expectations.

Subscribe now

Only the second receiver ever to win the Heisman, Smith caught 117 passes for 1,856 yards and 23 touchdowns.

Smith beat out six quarterbacks, two running backs, and Florida tight end Kyle Pitts, who finished tenth in the voting.

Meanwhile, Virginia Tech left tackle Christian Darrisaw graded out at 95.6 on his way to generating absolutely zero Heisman talk.

Was Darrisaw any less of a player than those who received votes in 2020? I don’t think so, but who’s to say?

Outside of subjective grades, there is a dearth of offensive line statistics that would allow interested third parties to accurately discern the true difference in performance between players.

Enter Artificial Intelligence (AI)

Much of the focus on AI has rightly gone to the various chatbots, such as ChatGPT, which have brought revolutionary advances to many office tasks.

However, the limit of these chatbots, as I noted in a two-part series last year, is in their training data.

The missing piece is using AI to generate 100% accurate data, then using AI tools like ChatGPT to analyze the video data, create metadata (data about data), then run statistics on the metadata.

Now that would be revolutionary.

And believe it or not, the revolution is upon us.

AI-driven video capture

Companies like Pixellot have already brought to market video technology that supports multiple views as well as AI-driven zoom and tracking on specific players.

Cameras like the S3 can now record game film from a tactical angle, while also producing fully zoomed-in game tape of each individual player on the field.

As a result, we can see heretofore hidden details of offensive line play.

Of course, this isn’t anything PFF couldn’t - and doesn’t already - do, albeit manually and at a more basic level. The difference comes in the degree of detail and the ability to automatically create and track new metrics.

AI-driven analytics

Although Pixellot touts an analytics offering, what they are really offering about is better classified as video editing. The true next step is incorporating a chatbot to generate statistics.

ChatGPT-4o can already do that.

Wait a minute - what does fact checking have to do with tracking football statistics?

Actually, a lot.

What ChatGPT does in the above-cited product feature is identify an assertion in a video, then check it against training data, which is OpenAI’s source of truth for determining the veracity of news.

All it takes are some tweaks to the code to have the bot look for whether or not a lineman pulled on a run blocking play, and if that lineman pulled, was he able to seal the edge.

With the correct code set, training data could be established from a few weeks’ worth of game film.

Then, voilà! The chatbot can now track a new offensive lineman metric. And it can do so automatically for all games recorded on digital video (which, these days, is nearly every FBS game).

New statistics are coming

The dark age of measuring play in the trenches is finally nearing an end.

Thanks to advances in AI technology, we can now automatically generate and evaluate play along both lines.

The next step is an integrated solution, something that is surely in the product pipeline for AI-video companies like Pixellot and its competitors (yes, this is a competitive market; numerous companies offer similar solutions aimed at different market segments).

After that, the final step will be the creation of new statistics. At present, Kaggle has regular competitions challenging enthusiasts to create new football metrics. I expect such competitions to kick into overdrive in the coming years as AI-driven video technologies are more widely adopted.

As far as an overall timeline, I expect everything described above to be mainstream within ten years.

Who knows, perhaps the first Hokie Heisman winner is a right guard preparing to enter middle school in the fall.

Subscribe now

Coverage of the 2023 season will feature a new-from-the-ground-up score prediction model.

Whereas the 2022 in-season model was a fairly simple, albeit extremely clunky and difficult to manage, linear linear regression setup, this year’s version is an Extreme Gradient Boost (XGBoost) model .

In this article, I will open the hood on this latest model, highlight what it does (and does not) do well, and share examples from the test results.

Background on XGBoost models

XGBoost is a machine learning algorithm that is specifically designed for supervised learning tasks, where one has a labeled dataset and is aiming to make predictions or classifications.

The algorithm was developed by Tianqi Chen, an Assistant Professor of Computer Science in the Machine Learning Department at Carnegie Mellon University. It builds upon the concept of gradient boosting, which is an ensemble learning technique that combines the predictions of multiple weak models (typically decision trees) to create a strong predictive model.

XGBoost improves upon the traditional gradient boosting algorithm by incorporating various enhancements and optimizations, making it extremely efficient and effective.

Advantages of XGBoost models include:

1. High Performance: XGBoost is designed to be highly efficient and can handle large datasets with high dimensionality.

2. Regularization: It includes regularization techniques that help prevent overfitting, making the model more generalizable.

3. Flexibility: XGBoost supports both regression and classification tasks, making it versatile for various predictive modeling scenarios.

4. Feature Importance: It provides insights into feature importance, helping users understand which features have the most impact on predictions.

5. Handling Missing Values: XGBoost can handle missing values in the data during training and prediction.

6. Parallel and Distributed Computing: It can be run in parallel on multi-core CPUs and distributed environments, which speeds up training.

Last year’s model was assembled with the computing equivalent of duct tape and a Sawzall. It was built to predict Virginia Tech games only, and some of the variables, notably QB Health, were subjective and prone to bias.

Key goals for this year’s model were:

• Applicability to all FBS games

• Ease of Use - no more manual data entry or subjective variables

• Flexibility in handling null value fields

• Accuracy

The 2023 XGBoost earns high marks in each area, and fully removes any and all bias.

Accuracy

The model was trained on regular season games from 2013 through 2020 and tested on all games from the 2021 and 2022 seasons. The testing results are as follows:

Of course, some games are more difficult to predict than others. For example, there tend to be more foreseeable blowouts early in the season, before conference play heats up.

While performance on the test data varied by week, the ranges all remained in the acceptable range.

And in the toughest games to predict, those with a spread ranging from -2 to 2, the model got more picks right than it did wrong.

Those results are pretty solid, especially considering the detailed mechanics of the model. Technically, the model makes two predictions - predicted home points and predicted away points.

The model is not actually predicting which team will win or lose, or by what margin. However, it is not predicting points in a vacuum either, as away team points are a variable in the home team predicted points algorithm, and vice versa.

One major note of caution: the score prediction model should not be used to bet over/under or against the spread. It is not good at either task. At some point in time I might create models to predict one or both of those.

Results from the testing data

Understanding predictions from the testing data, and how they compared to the final results, will help contextualize the 2023 predictions. Most predictions aligned nicely with the actual results. Below are some instructive deviations from the norm.

Near miss - 2021 Notre Dame at Virginia Tech

The model predicted a 29.3 to 28.4 Virginia Tech win. The final result was a gut wrenching 32-29 Tech loss. How close was the model? The Hokies led 29-21 late in the fourth quarter. Notre Dame drove down the field and scored a touchdown. Had the Irish failed on the two point conversion attempt and the Hokies gotten a first down, they could have taken a few kneel downs and secured a big 29-27 win. As it was, the Irish converted the two=point attempt to tie the game, the Hokies went nowhere on their ensuing possession and punted the ball back to the Irish, who drove far enough into Tech territory to set up a game winning field goal.

Right result, wrong score - 2022 Boston College at Virginia Tech

The BC game was the only real bright spot in Brent Pry’s first season as head coach. The Hokies won the game 27-10, and it was never in doubt. In comparison, the model predicted a 29.9 to 26.4 Hokie win. I’d like to meet the person who has created a model that accurately predicted just how heinously last year’s BC offensive line would play.

Didn’t see that one coming - 2022 Virginia Tech at Old Dominion

Tech lost 20-17, but the predicted outcome, 32.2 to 23.7 in favor of Tech, made sense in a world in which the Hokies do not commit 15 penalties and turn the ball over 5 times. That’s why they play the games!

Key injury - 2021 Virginia Tech at Boston College

This game was the straw that broke the camel’s back in terms of Justin Fuente’s tenure as head coach. On paper, the game was an even matchup. Phil Jurkovec was a game-time decision after missing multiple games due to injury. The model predicted a 29.2 to 27.4 Virginia Tech win. In reality, Braxton Burmeister got injured early and missed the rest of the game. Knox Kadum stepped in and struggled mightily. The Hokies could not move the ball, so once BC put up a few points, they just sat on their lead, cruising to a 17-3 win.

Overall, the model’s performance in predicting the scores of Virginia Tech games was reflective of all the test data. And even when the prediction did not match the actual result, the prediction made logical sense, with clear narratives explaining the deviation from the predicted scores.

Coming up!

Next week is finally game week!

I am finalizing new game matchup visualizations and planning a few new bells and whistles.

While I have not decided on a publishing cadence for the season, I am leaning toward a midweek, quantitative and data visualization heavy look at the week’s VT game as well as others involving ACC teams.

I will likely keep publishing columns each Friday, with a tweak here or there due to Thursday night games.

The season is upon us. Finally!

Subscribe now

Share Hokie Analytics

The discussion at hand is the limits of AI’s ability to be useful in the college football analytics space, specifically in the predictive space. Covid is controversial, and Covid vaccination even more so, and as such, when married to college football it is a perfectly illustrative of my point.

Since I have a Master in Public Health and worked in hospitals during the pandemic, I was able to connect a clinical decision made by a head coach (in this case, Michigan’s Jim Harbaugh) with the outcome of a game vs. the Vegas line.

Absent a connection like this showing up in an AI model’s training data, the AI will never be able to make such a connection. And since the landscape is always evolving and new factors emerge seemingly daily, AI is unlikely to crowd out the best human analysts in college football. (That sound you hear is the collective deep exhalation of a roomful of analysts working for Nick Saban at Alabama).

Now then, let’s connect the wisdom of Leonard Cohen to published science, the expectations of sports betters, and the outcome on the field. And in the end, I’ll bring it all back to Virginia Tech.

Connecting the Dots

I mentioned the medically illiterate in last week’s article. Perhaps that seemed harsh. Here is what I was talking about. This study showed that fully vaccinated healthcare workers, whose average age at first dose was 47 years, experienced 90% vaccine efficacy at least seven days following the second dose. Fully vaccinated nursing home residents, whose average age at first dose was 84 years, saw 64% efficacy.

Both figures are in broad alignment with the findings in other studies examining short-term efficacy in the fully vaccinated. In contrast, both groups showed negative vaccine efficacy during the 14 days following the first dose — meaning here that those who got the vaccine were more likely to have a laboratory confirmed positive SARS-CoV-2 test than unvaccinated controls during that time period. For healthcare workers the vaccine was -104% efficacious, while for nursing home patients it was -40% efficacious.

Other studies, like this Lancet preprint, showed that the period of positive efficacy lasted around 6.5 months following the second dose.

After that, it went negative (defined here as the vaccinated being more likely than the unvaccinated to get a symptomatic Covid-19 infection). In late 2021, the wave of Omicron infections nationwide suggested that the third shot was having a result more similar to the first shot than the second, i.e., an initial period of negative efficacy. So, in strict scientific terms, assuming the entire Michigan squad had already received a two-shot mRNA series back in the spring, they were likely to be in a window of negative efficacy regardless of whether or not they all got a third shot. But, while that might have been the risk Harbaugh sought to minimize, it was not the biggest concern. Safety was, or should have been.

Particularly alarming at the time (and still to this day) were reports of major cardiovascular (blood clots and heart attacks) and neurological (strokes) injuries. One example was then Wake Forest, now Notre Dame, QB Sam Hartman, who had surgery to remove a blood clot, causing him to miss the start of the 2022 season. A study presented at the Scientific Sessions of the American Heart Association’s annual conference in the fall of 2021 found a near doubling of the five-year risk of Acute Coronary Syndrome in patients who received an mRNA vaccine.

It was clear from the original Pfizer trial data that any potential protective benefit against Covid-19 would be more than erased by increased cardiovascular deaths. In fact, the FDA’s Summary Basis for Regulatory Action document revealed higher all-cause mortality in the vaccine group (21) than the placebo group (17). Pfizer’s trial showed three cardiac deaths for every one Covid death prevented, a stat available in the literature from the beginning, but not noted in the mainstream press and very unlikely to find its way into AI training data, especially given the political data adjustments that are now en vogue in the AI world.

Unlike the U.S., many European countries keep comprehensive and detailed mortality data by vaccination status. A number of these countries noted year-over-year increases in all-cause mortality in 2021, with a study out of Sweden published to a pre-print server in 2021 providing a good example.

The study noted that 3,939 out of the 4,034,787 people who received both doses of the vaccine before August 5, 2021 died within two weeks of vaccination. From 2015–2019, during the same spring to summer period observed in the study, an average of 3,300 out of 10.6 million Swedes died every two weeks. That is 600 more vaccinated deaths in less than half the number of total people.

The CDC had, from the beginning, said that mild to moderate side effects could occur following vaccination, but one could dig through media reports all day and still find no mention of severe adverse events. That information was all out there, though, hiding in the medical literature.

So, on the side effects front, considering the mass vaccination of Michigan players was to take place just days before the game, my public health training told me that the best case scenario was that the team would play sluggishly. The CDC’s VAERS database already contained tens of thousands of injury reports, and a 2022 FOIA request brought even more clarity via the V-Safe database, which was populated by vaccinees via a phone-based app.

Thirty-three percent of users reported an adverse health impact of some kind:

• 12.1% of people participating in V-Safe reported a mild adverse impact

• 13.3% reported a moderate adverse impact

• 7.7% reported an adverse impact requiring medical care

Had this dataset been publicly available in late 2021, I would have been even more bearish on the Wolverines.

It’s not like the actual outcome of the game was unimaginable. I ran this game through my latest multiple regression model (which, I should note, is distinct from machine learning based artificial intelligence) and the prediction came back 40-12, Georgia.

Rather, the human ability to identify and apply a connection that AI could not would put the human at a major advantage when it came to adjusting predictions. The AI would factor in the base x% chance that Michigan pulls the upset, whereas the human would realize that at this point an upset was basically impossible. The human could then adjust the expected point spread range accordingly.

Virginia Tech and Asymmetric Information

No dataset has ever existed, or likely ever will exist, that can make the connections I have drawn above. For AI to do this, it would need to ingest something like a study evaluating the impact of vaccinating an entire football team with a novel biologic days before a big game. Obviously, at the time there was nothing to evaluate - Michigan was the first known team to opt for this course of action. Without a robust collection of scientific data or significant commentary in the media or an online space, one will have nothing on which to train an AI model. AI can only look backward and recognize connections already made in the ingested data, then apply them moving forward.

The end result, as far as I was concerned, was the best case scenario for Michigan. They played sluggishly and got trounced.

As an analyst, I am always on the lookout for variables that will supersede models. Examples like the one presented above are rare. More common are X-factor variables, the most common of which is an injury to an important player, e.g., the starting quarterback. In a previous model that looked only at Virginia Tech data, I actually included a QB health field. The problem was that I had to manually enter the data, quantifying my own understanding based on public reports. As a result it was biased and often inaccurate (undisclosed injuries were a gaping hole).

Player eligibility/disciplinary actions are other factors that can have a major impact on the expected outcome of a game. My ears always perk up when I hear Chris Coleman of Techsideline begin a sentence with the words, “We’ve heard…”

Again, intuitively, fans get all this. Hence the long-running joke on message boards and Twitter about sauces. For this reason, when I share a modelled prediction for a game outcome, I also offer my own thoughts. I often agree, at least directionally, with the model, but when I disagree, it is usually because of a hard-to-quantify factor or a Leonard Cohen variable.

Like any dealer he was watching for the card that is so high and wild
He'll never need to deal another
He was just some Joseph looking for a manger

Brent Pry has stated on the record that he is not much into analytics. I’d love it if one of the beat reporters asked him why, and how defines analytics. If he means it in pure science terms, I’d agree with him. As the maxim goes:

All models are wrong. Some models are useful.

I’m in the camp that sees analytics as both an art and a science. The science part can get you in the ballpark in regard to most any question. The art part is what takes you the last mile. Unless and until AI takes up art, its utility in the college football analytics space will remain limited.

Subscribe now

I would see this neighbor mostly in the spring, summer, and early fall, when the Connecticut weather was nice enough for our kids to play outside (our sons are 6 weeks apart in age and are best friends). So, while I never got a detailed breakdown of his thoughts leading up to the CFP semifinal game between #2 Michigan and #3 Georgia, I do remember him mentioning that he thought it was a tough matchup, and that he had been rooting for Georgia to win the SEC Championship, which likely would have afforded Michigan a more favorable matchup with Cincinnati. Still, I would say his mood was cautiously optimistic leading up to the game.

Consensus among the pundits was that Georgia would win, and likely cover the -7.5 point spread. Still, there was a clear path to victory for Michigan, and had the Wolverines won, it would have been considered an upset, not a miracle.

In the days between the CFP selection show and December 21, I thought Michigan was set up well to be competitive and make it a four-quarter game. They were coming off the big win over Ohio St. and had completely dominated Iowa in the Big Ten Championship. They seemed to be playing their best football when it mattered most.

Then I saw this headline and knew all bets were off:

Up until November 2021, Covid was my full-time job. I earned my Master of Public Health in Health Policy & Management at Columbia University and worked for two different health systems during the course of the pandemic. The healthcare data I had at my fingertips was every bit as rich as the college football data that underpins this Substack. I vociferously read then, and continue to read now, the medical literature on the topic (research and clinical journals).

On its face, to the medically illiterate, Michigan’s decision to mass vaccinate all its players just days before the biggest game of their lives was a logical one. Harbaugh’s biggest worry was an outbreak that could cause key players, or even entire position groups, to miss the game due to a positive test.

“The card that is so high and wild…”

Artificial Intelligence (AI) is on a lot of people’s minds these days, and there seems to be a lot in the way of fear around its capabilities. I went from healthcare to managing Analytics for the fintech startup Vic.ai, which hopefully lends some credence to my thoughts on the topic. As I wrote in a LinkedIn post a few weeks back:

The New Yorker ran this article in [a recent] issue about the prospect of using AI to treat mental illness. Severe mental illness was an area I focused on during my MPH studies at Columbia University Mailman School of Public Health - I completed a geospatial analysis of state inpatient psychiatric hospitals during an internship at the Milbank Memorial Fund, and I volunteered in the evaluation center at NewYork-Presbyterian Hospital's White Plains psychiatric hospital (what NYP calls The Westchester Division). And yet, I found that the history of AI-mental health initiatives goes back further than I would have guessed. Still, the results, when positive, have tended to be at the margins. I couldn't help but compare those results to what we see at Vic.ai with our clients in the accounts payable space. For example, the 80% decline in per invoice processing time that some of our established clients have realized is light years ahead of anything achieved in the mental health space. When I asked myself, "Why?", the answer I came up with was: "Because Vic.ai leverages AI to improve performance on a tightly defined set of human-to-machine interactions. At the present time, AI has not demonstrated an ability to consistently solve issues governed by human-to-human interactions."

To expand on the above, AI has also not demonstrated an ability to reach beyond its training data. By definition, it cannot, and that will always limit its utility in dynamic environments, such as college football. There are factors that can crop up out of nowhere, factors that were heretofore considered unrelated, if they were considered at all. And in key moments, these factors can emerge in a way so as to render all others moot. Leonard Cohen captured the importance of these factors to the skilled analyst in The Stranger Song when he wrote:

Like any dealer he was watching for the card that is so high and wild
He'll never need to deal another
He was just some Joseph looking for a manger

Think about it, the average fan plays this game as well. So do talk radio hosts and television and print commentators. Data analysts are just a bit more structured in how they consider it.

My central argument is that it takes a human, and not just any human, but a human with some know how, to recognize that card when it is dealt, and to either incorporate it into the dataset, such that the magnitude of the determination can be captured, or toss the dataset entirely.

To Be Continued…

Next week, in Part 2, I will, in somewhat painstaking detail, present the folly of Harbaugh’s attempt to maximize Michigan’s chances of victory in the 2021 CFP semifinal by employing mass vaccination, and discuss why AI would fail to pick up on the impact this decision was likely to have on the game’s outcome. Then I will bring it back to Virginia Tech and discuss how this line of thinking impacts how I interpret models.

Subscribe now

Then, last week, I ran simulations of every Virginia Tech team since 2004 against that year’s National Champion. The key takeaway from that exercise is that outside of the 2004 team and a couple others when a comparatively lesser team won the title, Tech has not fielded a team that could compete at a National Championship level.

Now, to wrap up this multi-part series examining how good Tech has been and how far they have to go to get back to competing at that level, I will share the results of simulated games between the 2004 Hokies and every National Championship team since then. We know that Tech football has gotten much worse over the last two decades, but have the National Champions gotten better (or worse) over that same time period? And what is the key driver of those changes?

Let’s dive in…

How good was the 2004 team?

How good were the 2004 Hokies? Watch this 10 minute clip of the USC offense vs. the VT defense from the opening game of that season and then you can tell me.

I had an endzone seat 23 rows off the field that night in Landover. I taped the broadcast, but I don’t think I ever watched it all the way through. It wasn’t clear from my vantage point in the stadium just how hard the Hokies were hitting that night. Matt Leinart won the Heisman Trophy that year and the Trojans beat undefeated #2 Oklahoma 55-19 in the National Championship after sharing the title the year before. They were a juggernaut. And yet the Hokies led 10-7 at the half and trailed by a single point (14-13) with six minutes remaining in the game. Reggie Bush’s final touchdown reception came on a wheel route in which Blake Warren fell down in coverage. Warren, a career backup, started that night at Backer alongside Mikal Baquee at Mike, but Vince Hall and Xavier Adibi played a lot of first half snaps (it was the first regular season game for both). Adibi, as a redshirt freshman with incredible athletic ability, was shutting down Bush, but as the video shows, he tore his biceps late in the first half and would miss about six weeks. Without Adibi shadowing him, Norm Chow was able to scheme Bush open for two TD receptions in the second half - the deciding scores in the game.

Notice how big, strong, fast, and hard hitting that defense was. That was what it looked like in the glory days. The offense was average, mainly due to all the youth at wide receiver. They were uber talented, but Eddie Royal, Josh Morgan, and Justin Harper were true freshmen, Josh Hyman was a redshirt freshman, and David Clowney was a true sophomore.

Here is some trivia. Who started at WR for the Hokies that night against USC opposite Josh Hyman? The answer - Chris Clifton, a junior and converted quarterback. Hyman caught the Hokies’ only touchdown, and Clifton did not appear in the box score. Jeff King was the main target through the air - 4 catches for 65 yards. The following week against Western Michigan, Royal, Harper, and Morgan caught the first passes of the collegiate careers, and from then on the young guys got all the snaps out wide. As a point of comparison, imagine what the 2022 Hokies would have looked like had they featured four freshmen and a sophomore at WR.

So when the model says in a postseason rematch USC would be expected to eek out an 18-17 win over Virginia Tech, now you can see why. As close as Tech played the Trojans to open the season, they were not yet the team they would become. USC was the same team all season. Keep that all in mind as I walk you through the results of 19 years of simulated games against a slate of title winners.

Dominant in the 2000s

From 2004 through 2012, the Hokies went 6-3 in modelled games against each year’s National Champion. The only losses were to 2004 USC (18-17), 2005 Texas (22-11) and 2008 Florida (17-12). During this period, the average score was 17-10 in favor of Virginia Tech. The largest margin of victory for the Hokies was 19 points (over 2006 Florida) and the largest margin of defeat was 11 points against 2005 Texas. Notably, Tech went 3-0 against Nick Saban-coached Alabama, winning by an average score of 15-7.

National Champions are Getting Better

There was a clear inflection point in 2013. That was the year Jameis Winston led Florida St. to an undefeated season, culminating in a last minute win over Auburn. Alabama lost to Auburn that year in the “Kick Six” game. In 2014, the Crimson Tide lost to Ohio St. in the inaugural CFP semi-finals. The Tide won the title in 2015, but then lost the rematch to Clemson in 2016. In those four years, Alabama transitioned away from their ground and pound offense and adopted increasingly more spread concepts. Then, in 2017, Lane Kiffin was hired as offensive coordinator, and he brought a fully modern offense to Tuscaloosa.

Alabama was a late adopter, but once they took the plunge, that was it. There was no going back for anyone in college football. The 2017 Crimson Tide struggled on offense through most of the season and half the title game, then Tua came in, provided the necessary spark, and that was that. Every National Champion since 2018 has had a high powered offense. Outside of 2019 LSU, they all also had elite defenses. All of that is to say that National Champions today are better than they were 15-20 years ago.

From 2013 through 2022, VT 2004 was 2-7-1 against the National Champions. The wins came over 2016 Clemson and 2019 LSU and the tie was against 2015 Alabama. During that time period, the average score was 16-10 in favor of the opponent. Two teams shut out the 2004 Hokies: 2018 Clemson (30-0) and 2022 Georgia (12-0).

The table above suggests that National Champions have gotten better over time. Other data point in that same direction. Although ELO ratings have nudged up only slightly, net EPA per play has increased significantly among National Champions since 2004.

The gains, however, are all coming from the offenses, as defenses have seen their EPA per play decrease over time. Georgia’s defensive EPA in 2022 was slightly better than Auburn’s in 2010, but Georgia’s offensive EPA was nearly double that of the Tigers. And as we saw throughout this year’s playoff, better offenses impact opposing defenses directly, and also indirectly via offenses taking chances they wouldn’t have taken before (more aggressive on fourth down, attempting longer field goals, etc.). Sometimes those gambles pay off, and sometimes they do not.

Closing Thoughts

The key takeaway from this series of analyses is that while Virginia Tech has faded in the ranks of college football teams, the issue is two-fold. The Hokies, as a team, have gotten worse over time, but the best teams have also gotten better. So, while getting the team back to playing at a mid-2000s level would be a great achievement for Brent Pry and his staff, rather than returning Tech to being a perennial top-10 team, it would not return them to being a National Championship caliber team.

NIL and the transfer portal could accelerate the rebuilding process, but that is far from a given. Now that the portal has closed, I will turn to the roster and examining the types of players available in VT’s six-hour recruiting radius. It is no secret that Penn St. has taken over as the dominant program recruiting in the Commonwealth of Virginia. And yet and still, the Nittany Lions have not challenged for a National Championship under James Franklin. They currently stand at least a notch, and maybe two, below Ohio St. and Michigan in their own conference. So what is going on? I’ll begin a deep dive on that next week.

Subscribe now

In addition to modelling expected outcomes, I will also examine ELO and Net EPA trends. By the end of the article, you should have a precise understanding of the heights to which Frank Beamer took the team and the depths to which it has fallen in recent years.

ELO Trend

ELO is one of my favorite methods of comparing one team to another because it is a straight forward metric. ELO rises when a given team defeats a quality opponent by a significant margin, and it falls when the opposite occurs. Since 2004, the ELO stats are as follows:

Only two National Champions, 2006 Florida and 2010 Auburn, finished with an ELO below 2000. In contrast, only two Virginia Tech teams have finished a season with an ELO over 2000: the 2004 and 2005 teams. The 2005 team had the highest ELO, 2091, but that was still 316 points less than Vince Young-led Texas, which won a thrilling title game over USC and finished with the highest ELO of any team in this era (higher than 2019 LSU and 2020 Alabama). That 300+ point ELO difference is similar to the difference between Virginia Tech and Ohio St. going into their 2014 matchup in Columbus (the one VT won 35-21).

In two years, 2004 and 2006, the ELO difference between VT and the National Champion was less than 100 points. On five occasions (2013, 2018, 2020, 2021, and 2022) the Hokies finished with an ELO rating more than 800 points less than the National Champion, including a max difference of -982 points in 2022.

Overall, the trends are as follows:

While National Champion ELOs have increased slightly in the last 20 years, Virginia Tech’s has declined by about 700 points, or approximately one-third. Winning a game against a team with an ELO 300 points higher is a major upset. Defeating a team with an ELO 900 points higher gets into Miracle on Ice territory.

EPA

Another way of comparing Virginia Tech to the eventual National Champion is to look at offensive, defensive, and net expected points added (EPA). The proliferation of explosive offenses is reflected in the rise in Offensive EPA and decline in Defensive EPA. However, the net figure, as well as the interplay between both sides of the ball is still revealing. As this scatter plot shows, only the 2004 Hokies are really in the conversation with a decent segment of the National Champions. Some of the best Fuente era teams also sneak into Tier 3, but just barely. Only two National Champions fell into Tier 4 (2008 Florida and 2012 Alabama).

While ELO is a great indicator of strength over the course of a season, it is less informative when it comes to postseason and playoff games. As we have seen in recent years, defense still wins championships, but only if you have an offense that can move the ball and score. That assertion is clearly demonstrated in the above scatter plot, which shows the difference between the Hokies and the National Champs on both sides of the ball, and especially highlights Virginia Tech’s offensive deficiencies through the years.

Turning to Net EPA, the story is even more bleak. Only one Tech team (the 2004 Hokies) had a better Net EPA than the National Champions. In most years, VT trailed by a significant margin.

The Net EPA Difference trend mirrors what we see in ELO ratings, only to a greater degree. The Hokies have gone from slightly favorable in their comparison to the National Champions to distinctly unfavorable in 19 years. Half of the difference has come since 2018, when the VT program took a steep nosedive.

Expected Outcomes

The third way I examined the difference between Virginia Tech and the National Championship team from each year is by running the teams through the model I used for the VT CFP. I have said on numerous occasions that one of the outputs of the model is a score prediction. However, it would be more accurate to call this output an expected result. A score prediction would factor in qualitative data like motivation, matchup (some teams give others fits year after year regardless of the relative strength and talent of each) as well as data points that are not currently quantifiable, such as injuries (not just players who are unable to play but also those who are playing hurt).

The aggregated findings from modelled expected outcomes since 2004 are:

• Closest Game: 2004 USC 18 vs. VT 17

• Biggest Blowout: 2018 Clemson 61 vs. VT 0

• Average Game: National Champion 37 vs. VT 13

• Game with the Most Points Scored by Virginia Tech: 2019 LSU 43 vs. VT 31

• Shutouts by the National Champion: 3 (2018 Clemson, 2021 Georgia, 2022 Georgia)

• One-Score Games: 2004 (USC wins 18-17), 2006 (Florida wins 22-19), and 2007 (LSU wins 29-21)

• 35+ Point Margin: 2008 (Florida wins 42-6), 2013 (Florida St. wins 48-10), 2018 (Clemson wins 61-0), 2021 (Georgia wins 45-0), and 2022 (Georgia wins 40-0)

According to the model, no Virginia Tech team would be expected to beat the National Champion from that year. The closest the Hokies would come is the 2004 team playing a rematch against USC. On average, the Hokies lose each matchup by a convincing 24 points, with the gap growing significantly over time. Somewhat surprisingly, the 2007 Hokies would be expected to join the 2004 team in playing a close rematch from an earlier game against the eventual National Champion. In both cases, Tech was stronger at the end of the season because the Hokies were giving significant playing time to stud freshmen (Eddie Royal, Josh Morgan, and Xavier Adibi in 2004 and Tyrod Taylor in 2007) that had sat earlier in the year in favor of middling veterans or due to injury.

While there are peaks and valleys in expected outcomes, the trend in point differential is unmistakably negative.

A few more tidbits about specific expected outcomes:

• If it could, the model would have predicted 2022 Georgia holding Virginia Tech to -8 points. The only other team that would be limited to negative points if it could would be the 2018 Hokies (-1).

• The 2019 team would be expected to fare rather well against LSU, a team that featured perhaps the greatest offense of all-time. While the model looks at the totality of each team’s games, I’ll go ahead and qualify that those 2019 Hokies were a decidedly different (undefeated!) team when they started both Hendon Hooker and Caleb Farley. Take at least one of them away, and that team was looking at almost certain defeat (albeit, Duke aside, in a close game).

• The 2005 team, which in early November of that year was ranked third in both polls behind USC and Texas, would be expected to lose to the Longhorns 42-13 in a hypothetical postseason game. Given how that Tech team melted down on big stages against lesser Miami (27-7) and FSU (27-22) teams, a 29-point loss to Vince Young & Crew feels about right.

The Takeaway

The 2004 Hokies finished the year 10-3 overall, 7-1 in the ACC, and #10 in the final AP and Coaches polls. However, that team played at a near National Champion level, so much so that we would expect those Hokies to lose to USC in a hypothetical title game by a single point. That 2004 team, when considering the three metrics presented in this article, would be around the 25th percentile of National Champions since 2004 had they in fact won the title.

From 2005 to 2011, the remainder of the 10-win streak seasons, the expected outcome of matchups with the National Champion yielded an average defeat of 17 points. So, while it felt like Tech was in the highest tier of college football teams during that stretch, the Hokies were really a step or two down from the very best teams.

Things got worse from there. From 2012 to 2022, the expected margin of defeat extended to 31 points, with only two games (against 2016 Clemson and 2019 LSU) expected to be decided by less than two touchdowns.

So, clearly, the difference between how well the Hokies have played in recent years and how well National Champion teams have played has increased. Both the eye test and advanced stats tell us that Tech has gotten worse over the years. But what about the National Champions? Have they gotten better or worse over time?

Next week I will compare the 2004 Hokies to all the title winning teams. Yes, individual matchups matter, but by holding one Hokie team constant, in this case the best one, we should be able to uncover any trends regarding the strengthening or weakening of National Champions since 2004. At that point, we will have a clear and quantified answer as to how far Brent Pry and his staff have to raise the level of play by his team to have a legitimate shot at winning a National Championship.

Subscribe now

It’s championship Friday - time to crown the best Virginia Tech football team of the last 20 years. There has been a lot of real (not simulated) football since the semi-finals a week ago, so let’s quickly review how we got here.

Both the 2004 and the 2009 team had a first round bye. The 2004 team knocked out the #5 and #1 seeds by an average margin of 17 points. In those games, the team averaged 31.5 ppg while allowing just 14.5. The two teams VT 2004 defeated (VT 2007 and VT 2005) were probably the closest Virginia Tech has come to fielding a National Championship team on the field since 2000.

Meanwhile, on the other side of the bracket, the 2009 team won a couple of nail-biters over two of the most high-powered Virginia Tech teams in recent memory: the Jerod Evans-led 2016 team and 2010 Tyrod Taylor-Ryan Williams-Darren Evans-David Wilson team.

The Pregame Chatter

Looking first at the pregame matchup of full season stats, one would have to give a slight advantage to the 2009 team from an offensive perspective:

However, the VT CFP stats (two games for each team) tell a more intriguing story:

The 2004 team had a tougher road to the championship, with an average opponent seed of 3 and ELO of 1999. Given the 67-point average ELO difference, it should not be a surprise VT 2004 is 2-0 in the playoff, but the 17-point margin of victory is eye opening. However, considered alongside a 2:1 run/pass ratio, 87.5 average total yards per game margin, and a 9.2 to 5.2 edge in yards per pass attempt, it is not particularly surprising.

The 2009 team, despite playing a lower average seed (5 vs. 3), only averaged an ELO advantage of 10 points. This team squeaked by two opponents that were its overall equal. The main difference was Ryan Williams and Tyrod Taylor in the run game (47.5 carries per game as a team for 174 ypg). In most other areas, the 2009 team was on par with, or slightly behind, its opponents. If I were 2009 Bud Foster, I would be losing sleep over the defense, particularly the 417 yards given up per game and 6.3 yards per play. VT 2004 completely shut down both of its opponents. Points will likely be at a premium in the championship game for the 2009 team, but at the same time, the 2004 offense is not nearly at the level of the 2016 or 2010 units. Still, seeding aside, the 2004 team enters the championship game with a 172-point ELO advantage. That’s the sort of difference that shows up on film. Entering this game, both team know the 2004 team is favored.

3 Parts Coronation, 1 Part Competition

Too often, championship games end up being blowouts. While that was not necessarily the case in this game, one could not really call it close either. The 2004 team led 7-0 after one quarter and 13-0 at the half. While the deficit was only two scores, it felt like more, as the 2009 offense could not sustain a drive on offense.

Bryan Randall struggled in the passing game, completing less than 50% of his pass attempts on the day, with one interception. The 2004 Hokies also lost two fumbles, marking their third straight playoff game with three turnovers. And just like in the previous two games, it did not matter one iota against the 2009 team, which did not convert any of those turnovers into points.

Trailing 13-0 midway through the third quarter, the 2009 offense finally put together a good drive, going 70 yards in 12 plays, capped off by a Ryan Williams three-yard TD plunge. The momentum, however, was short-lived, as the 2004 offense responded with a touchdown drive of their own to extend the lead to 20-7.

At this point, with 11:28 to go in the fourth quarter and down by two touchdowns, the 2009 team tried to open up the offense some. Taylor attempted a pass on first down, but Williams got flagged for holding on a blitz pickup. On first and 20, Foster again dialed up a blitz, and this one got home. On second and forever, the 2009 team tried to set up a screen play, but it was snuffed out for a minimal gain. On third and 26, Taylor looked long, but seeing no open receivers, he attempted to run for the first down. He gained 15 yards, but was forced out of bounds well short of the first down marker.

With victory less than 10 minutes away, the 2004 team set about eating up as much clock as possible. They ran the ball eight times and gained 43 yards before being forced to punt. The drive took nearly five minutes off the clock, and the ensuing Vinnie Burns punt pinned the 2009 team inside its own ten-yard line. Nearing desperation mode, Tyrod Taylor attempted to hit Danny Coale on a first down slant pattern. The ball was tipped at the line of scrimmage and intercepted by LB Xavier Adibi, who returned it 17 yards for the game-clinching touchdown.

The final score was 27-7, and the 2004 team left no doubt as to which was the best Virginia Tech football team in the post-Michael Vick era.

VT CFP Analysis

So, if you’re like me, you’re probably thinking - why did the 2004 team win the playoff? And how?

The simple answer is good enough offense and elite defense, as measured by advanced statistics. The more complex answer is as follows:

1. The 2004 team had nearly double the Net EPA (0.1714) of any other team in the playoff (second place was the 2017 team at 0.0996). This is an unadjusted number (ELO is used to control for strength of record in the model), but taken on its face, it demonstrates just how good the 2004 team was. Tech finished 5th nationally in Net EPA in 2004, two spots ahead of National Champion USC.

2. Defense wins championships, and the 2004 defense was National Championship-level good, finishing second among major conference teams that year to NC State in Defensive EPA (note: the 2004 VT defense scored five touchdowns). Remember, the Wolfpack beat Tech that year 17-16 in Lane Stadium, when VT’s Brandon Pace missed a 43-yard field goal at the buzzer. I was in the stadium that day and remember how the ball was spotted incorrectly at the 26-yard line, two yards back of where Tech should have scrimmaged. Since the kick missed by inches, it is possible this snafu by the officials cost Tech the game. Anyway, NC State sacked Tech QB Bryan Randall 10 times that day. Football has changed a lot over the last 20 years, thanks to the proliferation of air raid and spread offenses, but as a point of comparison, VT 2004’s Def EPA of -0.1760 was almost twice as good as 2009 Alabama’s -0.0990 (lower is better), which led the nation that year. And that Alabama defense held Tech’s 2009 team to 155 yards of total offense. Tyrod Taylor was 9-20 for 91 yards passing, and the Hokies ran the ball 31 times for 64 yards. Is it any wonder that the 2009 Hokies lost by 20 to VT 2004 after losing by 10 to 2009 Alabama (in real life)?

3. In a playoff comprised of middling to atrocious offenses, the 2004 offense was good enough. Out of 19 Hokies teams (remember, there was not sufficient data from 2003), here is how the 2004 team ranked on five key offensive advanced stats:

   1. Explosiveness - 11th (0.9912)

   2. Power Success - 1st (0.7907)

   3. O-Line Yards - 4th (3.0315)

   4. Open Field Yards - 12th (0.9061)

   5. Points per Opportunity - 13th (-0.1239)

Tech defenses after 2017 were not even remotely in the same ballpark as the earlier teams, but the same can be said for the offenses. Although Covid made 2020 a weird year, there is no denying that Tech was stacked on offense. Compare the 2004 team’s numbers above to the 2020 team’s:

1. Explosiveness - 2nd (1.2439)

2. Power Success - 11th (06792)

3. O-Line Yards - 1st (3.2385)

4. Open Field Yards - 1st (2.0626)

5. Points per Opportunity - 2nd (3.6835)

It’s a shame that Covid had such a severe impact on the defense (player absences, conditioning issues, etc.), and the transition from Foster’s to Hamilton’s defense could not have come at a more unfortunate time. That team could have very easily won nine games, in which case Justin Fuente might still be coaching in Blacksburg.

Up Next

So, that’s it. As demonstrated quantitatively over the last four weeks, the 2004 team is the best Virginia Tech team of the last 20 years, and by not a small margin. Yes, I know, the 2003 team got the last second playoff boot, but they were not a realistic threat to win the whole thing. Same with the 2019 team that did not qualify.

While it is fun to play with models and simulate hypotheticals, there is a broader reason for embarking on this journey. If the logical first question is, What was the best team of the last N years? then a natural follow-up would be, What does that mean for the program in 2023? Coming up, I will connect the results of the VT CFP to the program as it stands in 2023 via two analyses:

1. What would be the expected outcome if each year’s Virginia Tech team played against the National Champion from that year (e.g., 2019 VT vs. 2019 LSU)? In 5 out of the last 19 years, the Hokies have played the eventual champion (Tech went 1-4 in those games), but only the 2016 game against Clemson was played in the postseason on a neutral field.

2. What would be the expected outcome if Virginia Tech’s best team (the 2004 team) played against the National Champion from each year in the postseason at a neutral site?

The first analysis is designed to quantify how close VT has gotten to fielding a National Championship contender since 2004, and the second analysis is designed to demonstrate how much better (or worse) National Champions have gotten over the last 19 years (here, the 2004 team will serve as a control). Overall, the goal is to discover how close the program got to the pinnacle of college football post-1999, and how far it must travel to get back to the top.

Once we have that context set, I expect to dive into recruiting and the roster from an analytics perspective. I am particularly interested in geospatial trends and dynamics that will help shed light on both strategy and outcomes for the Hokies and their primary competitors out on the recruiting trail.

Subscribe now

In August 2004, with Virginia Tech about to begin its first season in the ACC, conventional wisdom had it that the Hokies were drifting toward mediocrity. The media picked Tech to finish mid-pack in the conference, and hopes were not much higher in Blacksburg. Little did we all know that that season would be the first of eight straight in which the Hokies would win at least 10 games. Over that stretch they would play in five BCS bowls, although they only won one. During the streak, there were two distinct cores of players. The Vince Hall, Xavier Adibi, Josh Morgan, Eddie Royal teams and the Tyrod Taylor, Danny Coale, Jarrett Boykin teams. Of course there was some overlap in 2007, but for the most part, the teams were led by different players. In the moment, 2005 and 2010 felt like peak years - seasons in which the Hokies could really challenge for the National Championship. In contrast, 2004 and 2009 felt like ascending years - great teams that had yet to peak. Time and distance has me reconsidering this whole construct. How convenient then that the seeding and early round games would give us a semi-final that will decide the two best teams of the two biggest post-Michael Vick peaks.

Semi-Final 1

#3 seed VT 2010 at #2 seed VT 2009

On paper, according to conventional stats, the 2010 team was better than 2009 team. The Hokies went 8-0 in the ACC in 2010 and won the conference. That said, the 2009 team was sneaky good. Tyrod Taylor didn’t really come into his own as a passer until midway through the season, but Ryan Williams was electric all year. The 2009 team also had Jason Worilds at DE, and while he only had 4.5 sacks that year, he created such havoc that he became a second round draft pick by the Pittsburgh Steelers following the season.

CFP semi-final games are too often blowouts. That was definitely not the case in the matchup between the 2009 and 2010 Virginia Tech Hokies. The two teams went back and forth in the first half, with the 2009 team scoring first and later taking a 14-7 lead early in the second quarter. The 2009 team had the ball and the lead with 9:47 to go in the second quarter when Tyrod Taylor avoided a rush and took off down the field. Rather than slide, Taylor extended his body to try to get the first down. In so doing, he was tackled from his right side, and the ball was jarred loose. In addition to loosing the fumble, Taylor landed wrong on his ankle as he was tackled. He limped off the field gingerly and did not return for the rest of the half.

Meanwhile, the 2010 team got its offense revved up and scored the last 10 points of the half, including a 35-yard field goal as time expired, to take a 17-14 lead into the break.

During the extended halftime, all the talk centered on Taylor’s injured ankle and whether Frank Beamer would elect to pull Logan Thomas’s redshirt. The consensus opinion was that backup QB Juju Clayton would not be able to lead a comeback. In two drives at quarterback in the second quarter, Clayton was 1-3 passing for 2 yards (a wide receiver screen).

The 2010 team took the opening kickoff of the third quarter and marched 71 yards down the field, capped off by a Darren Evans five-yard touchdown run. The 2010 team, having extended their lead to 24-14, could almost taste a championship game berth. With a heavily taped ankle, Tyrod Taylor began warming up on the 2009 team’s sideline, but he would soon sit down. That’s because Dyrell Roberts completely changed the complexion of the game with a 97-yard kickoff return for a touchdown.

With the score now 24-21, the 2010 team embarked on a 14-play drive that chewed up nearly eight minutes. The 2009 team got a much needed goal line stand from its defense, conceding a field goal that extended the 2010 team’s lead to 27-21, but nonetheless kept it a one-score game.

Taylor returned for the next drive, but, walking with a noticeable limp, did not attempt a pass. Rather, he handed the ball off to Ryan Williams three times, with the final carry going for one yard on 3rd and 2. With more than six minutes remaining in the game, Beamer elected to punt and play defense.

After two quick first downs, it looked like the 2010 team was on its way down the field to secure the win. However, a safe wide receiver screen call turned disastrous when Marcus Davis fumbled as he fought for extra yardage. VT 2009 linebacker Bruce Taylor fell on the ball at the 2010 team’s 47-yard line.

On the ensuing drive, Taylor hit Ryan Williams on screen pass into a blitz that went for 34 yards. From the 13=yard line, Offensive Coordinator Bryan Stinespring called five straight running plays for Williams, the last of which resulted in the go-ahead touchdown.

Now down 28-27, with 31 seconds remaining and no timeouts, the 2010 team drove to midfield in 29 seconds. However, the last chance heave for the endzone was knocked away, and the 2009 team emerged victorious. Ryan Williams, who carried the ball a career-high 41 times for 173 yards, was named the player of the game.

Semi-Final 2

#4 seed VT 2004 at #1 seed VT 2005

The main difference between the 2004 and 2005 teams, from a personnel standpoint, was the quarterback position. Bryan Randall won ACC Player of the Year as a senior in 2004. He was the unquestioned leader of the team and displayed great toughness throughout the season. Vick was suspended for the year, meaning that true freshman Sean Glennon was the second string QB. Randall could not get hurt, and he didn’t. He played every meaningful snap and was the main offensive threat for the Hokies.

In 2005, Marcus Vick was named First-team All ACC, but lost out to Wake Forest’s Chris Barclay for Player of the Year. Randall’s TD/INT ratio was a little better, while Vick completed a higher percentage of his passes. Ultimately, the difference boiled down to composure in high pressure situations. Randall was cool, calm, and collected in big games against USC, Miami, and Auburn, willing his team to stay in each game. In contrast, Vick lost his composure in games against Miami, Florida St., and Louisville. His stomp on Elvis Dumerville in the Gator Bowl against the Cardinals ended his days in Blacksburg, and for all intents and purposes his football career, as he was an undrafted, short-term practice squad player in the NFL. I mention all this because it is critical in understanding how the second quarterfinal played out.

The 2004 team took the opening kickoff and drove down to the 28-yard line of the 2005 team before the drive stalled. A Brandon Pace 45-yard field goal gave the fourth seed a 3-0 lead less than five minutes into the game. The 2005 team went three-and-out on its opening possession, and Eddie Royal returned the punt 39 yards to the edge of the redzone. The offense took advantage of the good field position, with Randall completing a seven-yard pass to tight end Jeff King to take a 10-0 lead.

From there the game settled into a defensive slugfest. Both quarterbacks faced constant pressure, and the running games only appeared in fits and starts. With 4:15 remaining in the second quarter, defensive end Darryl Tapp hit Randall just as he released a second down pass. The ball fluttered right into the waiting arms of Xavier Adibi, who returned it 16 yards to the 2004 team’s 41-yard line.

Trailing by 10 and badly needing a score before halftime, VT 2005 drove down to the four-yard line. Facing third and goal, Marcus Vick looked right on a designed rollout. His first read was covered and a blitzing Vince Hall made sure Vick didn’t get a chance to look to his second read, sacking him back at the 11-yard line. The 2005 team settled for a FG and a 10-3 deficit at half-time.

VT 2005 got the ball first in the third quarter and netted a field goal to draw within 10-6. After stopping the 2004 team, VT 2005 got the ball back and was driving when Vick made an ill-advised throw back across his body. Safety Vinnie Fuller intercepted the pass, and returned it 27 yards. With great field position following the sudden change, Randall led another short touchdown drive, this time scrambling in on a QB keeper from three yards out.

Now trailing 17-6 late in the third quarter, the 2005 offense, keyed by 22-yard Cedric Humes rush up the middle, drove into VT 2004 territory. Again, a key sack, this time by Chris Ellis, ended a promising drive. The 2005 team salvaged a 49-yard field goal to make it a one score game.

From there the teams traded punts. The 2005 team got the ball back with just under five minutes to play, still trailing 17-9. A first down holding call put them behind the chains, and after a second down screen pass was snuffed out for no gain, Vick and the 2005 offense faced 3rd and 20. In desperation mode, Vick looked long for David Clowney, who initially had a step on cornerback Eric Green. The pass hung up in the air a bit, though, and Green recovered enough to tip the ball up in the air. Garnell Wilds, in the game as a fifth DB, snagged the ball out of the air and took it to the house to secure the victory for the 2004 team.

Final Thoughts

The two semi-final winners had one thing in common - they were both unflappable. They did not win every game, and in fact, both lost their opening game. It is, rather, who they lost to and how they lost that made all the difference. They played everyone close. There were no meltdowns. They did not just beat up on bad teams. They both went toe-to-toe with the eventual National Champions in the first week of the season.

Meanwhile, the 2005 team, when it lost, lost ugly. The 2010 team faced only limited punishment from the model for the JMU loss, as FCS games are not included. The only hint of that loss was the ELO drop. The 2010 team faced two great opponents. They lost to Boise St. in a week 1 thriller and got blown out by Andrew Luck-led Stanford in the Orange Bowl (after trailing 13-12 at halftime). In sum, the two teams that will play for the VT CFP Championship each lost three games, but none of those losses was a blowout, and they played really tough schedules. In semi-final matchups against teams that were their equals on paper, that made all the difference. Next week’s title game should be a good one!

Subscribe now

While the entire point of this simulated tournament is to determine the greatest Virginia Tech team of the last 20 years, a byproduct of the process is tiering the teams themselves. Seeding is one thing, but the actual results were sure to draw lines of separation that were not clear before. There will likely be a small group of elite teams, a larger group of contenders, and still a larger group of also rans. This round, probably more than any, should go a long way toward separating the contenders from the elite and quantifying the differences between these two tiers.

Quarterfinal 1

#5 seed VT 2007 at #4 seed VT 2004

In my mind, these two teams always felt so different, but when scanning the stats, they are actually a lot a like, at least on the face of things. If we peel back the onion some, the differences become a bit more noticeable. The 2007 Hokies threw the ball a lot more than the 2004 team did, and it’s no wonder, given the 3.3 yards per carry any time they tried to run it. It’s not like the 2004 team was dynamic on the ground. In fact, at the time, the running game experienced a major decline from the 2003 season, when Kevin Jones ran for more than 1,400 yards, at an average of nearly six yards per carry. Ultimately, that last bit, yards per carry, is the most important distinction between the two teams in this first quarterfinal, at least entering the game. USC and Auburn had defenses that were as stingy as the 2007 Hokies, and they both shut down the VT running game. Might the 2004 Hokies struggle again to run the ball against a good defense?

In a word, sorta. In four words, sorta, but not really. Mike Imoh, Cedric Humes, and Bryan tallied 163 yards on 43 carries. The 3.7 YPC was down from their season average, but if the line struggled to get a consistent push, it did limit stops behind the line of scrimmage. With the run game churning out 3-5 yards on each attempt, Bryan Randall was able to hit on some big throws downfield, especially in the play action game.

For the 2007 Hokies, a team that many had considered a potential dark horse to make a run to the VT CFP Championship Game, things started badly and went downhill from there. On a third-and-six play on the 2007 team’s opening drive, Sean Glennon was hit from behind by a backside blitzer, fumbling in the process. The 2004 Hokies recovered, and four plays later, they were in the endzone.

After trailing 20-3 at the half, the 2007 team played Tyrod Taylor exclusively at quarterback in the second half. The freshman phenom did some damage with his legs, but the downfield passing game was nonexistent. Taylor was under constant pressure, and was often forced to bail on passing plays if his first read wasn’t open. Taylor did manage to lead a few scoring drives, which kept the score respectable, but from the second quarter on, this game was not in doubt.

Quarterfinal 2

#6 seed VT 2006 at #3 seed VT 2010

The second quarterfinal featured a matchup between one of Bud Foster’s best defensive units and undoubtedly the most prolific Frank ‘n Stiney offense. The flip side - the stoppable force (2006 offense) vs. the movable object (2010 defense) - is also intriguing, if less exciting. It is hard to quantify just how bad the 2006 offense was, but I’ll try. Brandon Ore ran for 1,100 yards, and yet the team yards per carry (3.2), matched that of the 2022 team. Ore, in fact, struggled on the road that season, running for only 266 yards on 3.7 YPC. The 2006 team’s 295 yards of total offense was only six yards more than the generationally bad 2022 team’s 289 yards. But here’s the kicker - the 2006 team turned the ball over nearly twice as many times per game (2.1) as the 2022 Hokies did (1.3). The 2006 team won 10 games because of Bud Foster and special teams. Surely there is no way this team could hope to make a game of it’s quarterfinal matchup, right?

Wrong. The defense was that good, and they were again in their game with the 2010 Hokies. After registering four shutouts during the regular season, the 2006 Hokies were not intimidated by Tyrod Taylor, Ryan Williams, Darren Evans, and the rest of that great 2010 offense. They harassed Taylor on every dropback and turned the game into the football equivalent of trench warfare.

The 2006 defense was at its best following turnovers by its offense. Twice they took the field on their side of the fifty in a sudden change situation, and twice they kept the 2010 team off the scoreboard. Still, Taylor was efficient at QB for the 2010 team, despite throwing a couple interceptions (only one of which was really his fault - the other was a tipped pass at the line of scrimmage). No surprise - the difference in the outcome resulted from the 2010 team’s ability to run the ball, especially down the stretch. Clinging to a one-point lead, Darren Evans converted two crucial third downs with runs of four and five yards, respectively, to help seal the win.

Quarterfinal 3

#7 seed VT 2016 at #2 seed VT 2009

The third quarterfinal game matched Justin Fuente’s best Virginia Tech team (2016) against one of Frank Beamer’s best post-Michael Vick teams (2009). On paper, the 2016 offense was stronger, with the notable caveat about snap issues and stupid penalties. Averaged out of the entire season, turnovers were not a huge problem, but the 2016 Hokies lost a ton of fumbles in the first half of the season. In addition, I do think there is some bias in the season stats.

Both teams played that season’s eventual National Champion and lost. The 2016 team played Clemson in the ACC Championship game, losing 42-35 after failing to convert a fourth-and-medium opportunity at the edge of the redzone in the final minute. The 2009 team lost to Alabama 34-24 in the opening week of that season, letting a 17-16 lead entering the fourth quarter evaporate as fatigue set in for the defense. The big difference was that the 2016 team played Clemson in the postseason, when the offense was clicking on all cylinders. The 2009 team played the Crimson Tide and Nebraska (which had an elite defense that year) in the first three weeks of the season, when the Hokies were still trying to gel as an offense and fully grasp what they had in Ryan Williams. All of that is reflected in the game-by-game stats, and the model should give it some credence.

This game turned into another nail-biter. Back and forth they went, trading punches. With five minutes to go in the game, and trailing 23-21, the 2016 team faced 2nd and 8 from their own 34-yard line. Jerod Evans ran what looked initially like a pop pass. The 2016 Hokies, however, broke tendency with a brilliant wrinkle. As 2010 safety Eddie Whitley followed his key and sprinted forward to cover Sam Rogers coming out of the backfield, Evans threw a deep post to Cam Phillips in the vacated zone. The throw was on target, Phillips made the reception, then he outran the defense for the go-ahead score.

Now down 28-23, and needing a touchdown to win, the 2009 Hokies embarked on a 17-play, 86-yard drive that chewed up all but 19 seconds. Tyrod Taylor scored the game winning touchdown when he scrambled away from the initial rush and outran the spying middle linebacker, Andrew Motuapuaka, to the pylon. The two-point conversion failed, but the lead held, as did the sense that the 2009 team stole one.

Quarterfinal 4

#9 seed VT 2017 at #1 seed VT 2005

The last of the quarterfinal games, another Beamer-Fuente matchup, looked more even on paper than many might suspect. Both teams had excellent defenses, loaded with future NFL stars. However, the 2017 offense was young at the skill positions and prone to dry spells, while the 2005 offense was experienced and stacked at every position. To some degree, the differences in the various offensive stats reflect advances in offensive philosophy (the adoption of spread principals by Fuente). The 2017 team threw the ball 111 times more than the 2005 team did. Not captured in the Pregame Matchup below are any stats around special teams, which were indeed special back in 2005. Combined with the #1 defense in the country, the 2005 punt return team (featuring Eddie Royal) gave Marcus Vick and the offense lots of short fields to work with. Thus, action (points) required less motion (yards). And the model knows that, as two separate special teams advanced metrics are included.

If there was going to be a blowout, I thought this game was the most likely candidate. Clearly, I underestimated the 2017 defense. They were masterful in this game, holding the 2005 offense to just 91 yards rushing on 44 carries. They also intercepted two Marcus Vick passes, both on the plus side of the field. However, on both occasions, the 2017 Hokies were held to field goals. On the day, three trips to the redzone resulted in three field goals for the 2017 team.

The 2005 team controlled this game from the first quarter on, but they struggled to put the game away. Their 17-6 halftime lead felt simultaneously insurmountable and insufficient. After the 2017 team took the opening kickoff in third quarter and drove down for their third field goal, cutting the deficit to eight points, fans in Lane Stadium sensed, for the first time, the possibility of an upset. Those fears were quickly erased, as the 2005 team answered with 10 points of their own over the their next two drives. The lead was 27-12 late in the fourth quarter when Josh Jackson connected with Sean Savoy for that team’s only touchdown on the day. The two-point conversion failed, rendering the onside kick attempt moot.

Final Thoughts

I am still scratching my head over the 2007 team getting blown out by the 2004 team. I could see that being a 7- to 10-point defeat, but 19? That surprised me. But should it have? The 2007 team had a major flaw (offensive line) and did not play well in high profile games. The 2004 team did not have any major flaws, had great team chemistry, and played very well against top competition. Mental toughness, and composure in pressure situations were not factors that I had considered going into the VT CFP, mainly because they cannot be directly quantified. However, at this point, I think we have enough simulated examples to hypothesize that these factors are key drivers behind teams that overperform expectations.

The higher seeds all advanced out of the quarterfinals, although two did just barely. Through two rounds (eight games), the higher seed is 7-1. What does this tell us? Seeding is not a variable in the model, so it does not merit a direct benefit. The real benefit of being the higher seed has been homefield advantage. The 2017 team’s win over the 2013 team (which replaced the 2003 team, the original #8 seed) was the only road victory. I wonder if the trend will carry over to the CFP once it expands to 12 teams.

Now that we have reached the semi-finals, the remaining games will be played at neutral sites. The 2004 and 2010 teams were both 0-2 in high profile neutral site matchups, while the 2005 and 2009 teams split their two neutral site games. If you’re looking for an early advantage, I would start there, but don’t forget about x-factors like performance in big games (regardless of venue). The 2005 team wilted, while the 2004 and 2009 teams took every big game down to the wire. Perhaps a harbinger of things to come.

Subscribe now

Finally, the coding, testing, and quality assurance on the model is complete. It’s time to simulate some games and find out which is the best Virginia Tech team from the last 20 years. Just a reminder, the format follows what news reports suggest will be the next CFP format. First round games involve seeds 5-12 and are played at the higher seed’s stadium (in this case always Lane Stadium, obviously, but the higher seed gets the home team benefit within the model).

Game 1

#12 seed VT 2015 at #5 seed VT 2007

Frank Beamer’s final team seemed pretty average in real-time. The 2015 campaign began with excitement, as Tech opened the season by playing host to defending National Champion Ohio St., a team it upset the previous season. That excitement evaporated the moment Michael Brewer went down with an injury, early in the third quarter. Brendan Motley took the QB reins and in subsequent games surprised many with his playmaking abilities. These Hokies, however, racked up a lot of early losses, and just before the midway point of the season, Beamer announced his retirement, effective at the end of the year.

So how did that team make this playoff? Two of its six losses were in overtime against ranked teams (the game against #23 Duke went to 4 OTs). Two other losses (both one score games) came in driving rainstorms, and Motley had to play hurt late in the Pitt game. All of that is to say the 2015 team could have easily been 9-4, or even 10-3.

Just looking at the conventional statistics, one would expect a close game between the 2015 team and the 2007 team. There are some key differences, though. The 2007 team played a much more difficult schedule (at #2 LSU, vs. #2 BC, at #22 Clemson, at #16 Virginia, vs. #12 BC in the ACC Championship game, and vs. #8 Kansas in the Orange Bowl). What felt like an underachieving offense at times back in 2007 was in retrospect a very hit or miss unit that faced really tough competition. Tech’s 2007 team finished the regular season by scoring 40, 44, 33, and 30 points against Miami, FSU, #16 UVA, and #12 BC, respectively. However, this team also scored 17 points or less in four games (East Carolina, #2 LSU, UNC, and #2 BC).

Therefore, it should not come as much surprise that the 2007 team dominated the 2015 team in a game that was not as close as the 30-20 score suggests. Michael Brewer hit on a few big passing plays, but was under constant pressure all game long, with sacks dragging down the team’s rushing totals. Fumbles (two lost) were also an issue, as the 2015 Hokies struggled to consistently move the ball. They hung around, but were never really a threat to pull the upset.

For the 2007 team, it was a workmanlike effort, with the offense slowly grinding through the 2015 defense. The 2007 offense ran 86 plays, including 52 runs, as they controlled the clock. Third-down conversions were a big part of this win, as the 2007 team’s predicted yards per play (4.6) was actually slightly less than the 2015 team’s.

Talent won out here, but the 2007 team’s flaws seem bound to trip it up eventually. The Sean Glennon/Tyrod Taylor combo only completed 50% of its passes on the day, and the 2015 defense is one of, if not the, weakest defense in the playoff. The 2007 team will likely lean on its own strong defense in the quarterfinals, but as the #5 seed, there is no telling as to whether that will be enough to advance.

Game 2

#11 seed VT 2008 at #6 seed VT 2006

Game 2 featured a matchup of two teams that, prior to 2022, had amongst the worst offenses fielded by Virginia Tech since the turn of the millennium. The 2006 team was loaded at receiver and featured peak Brendan Ore at tailback, but the offensive line really struggled and, for the first time since 2001, there was not a mobile quarterback to make something out of nothing when things broke down. Defensively, the 2006 Hokies were as stingy as they come. However, the offense, which averaged less than 300 yards and more than two turnovers per game, often put the defense in tough positions, which ultimately led to losses (see: Chik-fil-A Bowl, in which the defense gave up just 200 yards to Georgia and VT still lost 31-24, mainly because the Hokies only gained 189 yards, and they turned the ball over four times).

The 2008 team was young everywhere on offense and counted on its experienced defense to hang in games. After four years of Vince Hall and Xavier Adibi at linebacker, Brett Warren and Purnell Sturdivant took over as starters in their redshirt senior seasons. Stephan Virgil and Macho Harris were lockdown corners and Kam Chancellor manned the deep secondary. Offensively, this team was a mess for the first two-thirds of the season. Gone were the four experienced receivers from 2007, replaced by freshmen Jarrett Boykin, Danny Coale, and Dyrell Roberts. Sophomore QB Tyrod Taylor and first-year starting tailback Darren Evans did most of the damage when the Hokies had the ball. The 2008 team ultimately won the ACC Championship game and the Orange Bowl, but most of its wins were ugly, and the team did lose four games.

In the VT CFP matchup between these two offensively challenged teams, the 2006 team scored just 21 points and nonetheless won comfortably. Neither team was able to generate a consistent rushing attack, and it was the battle out wide that determined the winner. Macho Harris played for both teams, but on the 2006 team he combined with Brandon Flowers to shut down the young receivers for 2008 Virginia Tech, who were still learning the offense even at season’s end. Tyrod Taylor completed 19 of 32 passes for the 2008 squad, but everything was underneath, resulting in a paltry 4.4 yards per pass attempt.

On the other side, the 2006 team, with five good upperclass receivers (four of which would play in the NFL), was able to isolate the 2008 linebackers and safeties on a few intermediate throws down the middle of the field, one of which went for a touchdown. Once they got out to the early lead, with the 2008 team showing no threat of a downfield passing attack, the 2006 team was content to run the ball, throw wide receiver screens, and generally avoid risky play calls. Still, the 2006 team turned the ball over three times, two of which were fumbles. They got away with such sloppiness in this game, but against the high powered 2010 team in the quarterfinals, they will likely be made to pay for any and all turnovers.

Game 3

#10 seed VT 2011 at #7 seed VT 2016

The 2011 Hokies entered the ACC Championship Game with an 11-1 record and ranked #5 in the country, but they lost to Clemson for the second time that season. Then, in the Sugar Bowl, Danny Coale caught the ball in overtime, but instead of rewarding the Hokies with a touchdown, the referees gave them an opportunity to miss a field goal, which they did, and the rest is history. That year, the Hokies started high four-star level players at quarterback (Logan Thomas) and tailback (David Wilson), had four outstanding wide receivers (Coale, Boykin, Marcus Davis, and D.J. Coles), and a bunch of seniors on the offensive line. The offense lacked depth, though, and the play calling felt dated compared to the new spread attacks proliferating across the country. The defense was good, but they were young.

The 2016 Hokies featured the best Virginia Tech offense since the 2000, Michael Vick-led squad. Jerod Evans rewrote the record book at quarterback, and with Isaiah Ford, Cam Phillips, Bucky Hodges, and Sam Rogers, the Hokies were an exciting team to watch in Justin Fuente’s first season as head coach. Defensively, the Hokies looked like a Bud Foster unit for the first time in several seasons. Woody Baron wreaked havoc from his defensive tackle position, and there were no glaring weaknesses for opposing teams to attack. The main warts were offensive mistakes, especially bad snaps from center and stupid penalties, of which quite a few were accumulated by future NFL All-Pro Wyatt Teller.

In a surprising outcome, the 2016 team handled the 2011 team with ease. Jerod Evans had a game to remember, completing 80% of his passes for 245 yards while also leading the Hokies rushing attack, which went for just over 200 yards on 41 carries.

The 2016 Hokies took the lead early and never trailed. Woody Baron and Nigel Williams blew up run play after run play, and David Wilson was never able to get going. A second quarter safety was indicative of the type of day the 2011 team had on the ground. Wilson took a handoff at the two-yard line and, finding no hole up the middle, attempted to bounce the play outside. Tremaine Edmunds came around the edge, beat the block by tight end Chris Drager, and tackled Wilson a yard deep in the endzone. That extended the VT 2016 lead to 19-3 and forced the 2011 team to abandon the run. For the game, Thomas completed 26 of 44 passes for 215 yards. with an interception. VT 2011 was held to just 289 yards of total offense, and the team bowed out of the playoff quietly in the least exciting first round game.

Game 4

#9 seed VT 2017 at #8 seed VT 2013

As I suspected, there were not enough data to include the 2003 Hokies in the playoff, so they were replaced by the 2013 team, which though technically the #13 ranked team, took the vacated #8 seed.

The 2013 offense went only as far as Logan Thomas could carry it. The defense, though, was very good. That unit’s high point was the opening game of the season against Alabama in the Georgia Dome. Although Tech lost that game 35-10, Bud Foster’s defense held the two-time defending National Champion Crimson Tide to 206 yards of total offense.

The 2017 Hokies were similar in that they struggled at times on offense, but were stout on defense. In fact, this was the last of the great Bud Foster defenses. The team finished with a 9-4 record, but without goal line stands to preserve wins against West Virginia and Pitt, Justin Fuente’s second team would have been a very pedestrian 7-6.

On the whole, Josh Jackson played well in his only full season as the starting quarterback, but outside of Cam Phillips, the offense lacked weapons at the skill positions. In Fuente’s offense, Travon McMillian was a shell of his 2015 self, and he would transfer to Colorado after the season, where he ran for over 1,000 yards in his final collegiate season.

In the closest of the four first round games, the 2017 Hokies defeated homestanding 2013 Virginia Tech 20-16. Leading 16-13 with six minutes remaining in the fourth quarter, the 2013 team was driving with a chance to take a two-score lead. On third-and-seven from the 2017 team’s 21-yard line, Logan Thomas threw a 10-yard post to D.J. Coles, who was hit by Terrell Edmunds just as the pass arrived. The ball popped straight up in the air and was intercepted by Mook Reynolds, who had been bracketing Coles on the play. Down three and backed up deep in their own end, Josh Jackson led his team 89 yards in four and a half minutes to score the game winning touchdown.

Up until that point, Jackson had completed just 9 of 23 passes, but when it mattered most, Jackson delivered. He completed five of seven pass attempts on the final drive, the longest of which was a 45-yard completion to Cam Phillips on a double move (2013 freshman CB Brandon Facyson slipped and fell on the play).

Logan Thomas would get one more chance with 90 seconds to play, but a first down sack put the 2013 team behind the chains and chewed through valuable seconds. Thomas drove his team to nearly midfield and had a chance to throw a Hail Mary at the end, but that pass was knocked away, and with it the last remaining chance for the 2013 team to advance in the playoff.

Final Thoughts

Reflecting on the opening round, a few things really surprised me. In the moment, without the assistance of advanced statistics, it was hard for me to gauge how good the 2008 and 2011 teams were, and how many of their wins were the result of playing so many bad ACC teams. The simulation makes clear that the 2008 team was majorly flawed, and the 2011 team was not as good as its record would suggest.

On the flip side, the 2013 team was probably better than most perceived it to be (look at that 1809 pre-game ELO - it was 55 points higher than that of the 2011 team that won 11 games and played in the Sugar Bowl). I am also intrigued at how the 2015 team was outplayed and outclassed by the 2007 team, yet still ended up only losing by 10. The ELO difference between those two teams was 575 points, which is massive. That would be like 2022 Virginia Tech playing against 2022 Oregon. I can’t imagine that being a 10 point game! So, I think the takeaway is that it really is possible for a team to outperform its record - sometimes the ball, or in this case the breaks (literally), do not go your way.

Looking ahead to the quarterfinals, storylines abound. Justin Fuente’s best teams are slated to play two of Frank Beamer’s best post-Michael Vick teams. The Fuente teams are both seeded lower, but the 2/7 matchup especially holds the potential for an upset. The 4/5 game features Bryan Randall and a bunch of freshmen against Tyrod Taylor and all those freshmen, who were then seniors. I have long believed that the 2004 team was the toughest, most unflappable post-Michael Vick VT team. On paper, it looks like an even matchup, but I fully expect the 2004 team to win by 7-10 points. That team would not have won at 2007 LSU, but it also would not have lost 48-7. Finally, the 3/6 game matches great offense against elite defense, and terrible offense against average defense. They say that defense wins championships, and that is generally true. We’ll see if it also wins quarterfinals.

Subscribe now

1. Guys with one year of remaining eligibility looking for an extended NFL tryout (i.e., they need to pad their stats)

2. Guys buried in the depth chart looking for a fresh start and maybe a better fit

So far, there is nothing of interest on the portal front from a data analysis perspective (which is probably a good thing). Pry & Co. have done a good job of retaining their young core.

Portal prelude aside, before launching into the Virginia Tech CFP, I thought it might be helpful to provide a thorough examination of what the new model can and cannot tell us. My goal is to keep this at high enough level that the general fan can fully understand and processes everything, but provide enough detail so that when I launch the simulated games next week, everyone will have the necessary background to contextualize the results.

How it Works

Like the in-season model, this new one is a multivariate model that looks a whole host of advanced statistics and creates the best “fit” for the predicted field. Each formula includes at least 20 variables. Variables are either game specific (pre-game ELO rating, home/away, regular/postseasons, etc.) or team specific (offensive passing play EPA, defensive front seven havoc, etc.). Team specific stats encompass the entire season. Remember, the aim is to predict how Team X will do against Team Y. Team X is the aggregation of all its advanced season stats (each team plays one season) and how it will perform will depend on Team Y’s unique strengths and weaknesses (the aggregation of that team’s advanced season stats) when placed in a particular game environment (game specific variables). Two variables of interest that are not included are injuries, for which there is no quantitative stat tracked, and betting lines (the dataset is incomplete). One variable currently not in the model that I might add before the games commence is weather (likely temperature at kickoff).

What Should Happen

The greatest value of this model is that it predicts what should happen in a given matchup. In most cases, what should happens is what actually happens, and that is reflected in the following predictions vs. the actual scores:

• 2004 VT vs. Western Michigan - Predicted 63-0, Actual 63-0 (no variance in score differential)

• 2005 VT vs. Boston College - Predicted 34-13, Actual 30-10 (1 point variance)

• 2008 VT vs. Duke - Predicted 21-9, Actual 14-3 (1 point variance)

• 2014 VT at North Carolina - Predicted 35-21, Actual 34-17 (3 point variance)

• 2020 VT at Wake Forest - Predicted 25-34, Actual 16-23 (2 point variance)

However, sometimes what should happen does not. A perfect example of this is the 2005 Sugar Bowl (following the 2004 season), which saw Virginia Tech face off against Auburn.

What did and did not happen at the Sugar Bowl

Entering the game, Auburn was undefeated, ranked #3 in the final BCS standings, and still smarting from their National Championship game snub. Virginia Tech, conference champions in their first year as ACC members, entered with a 10-2 record and #9 BCS ranking.

The Hokies were decided underdogs, or so we all thought. And when I say all, that includes the Hokies’ head coach:

Frank Beamer said he thought Auburn should be in Miami playing for more than pride, and had so indicated in his ballot in the coaches’ poll.  “I certainly believe, and the way I voted, is that (Auburn) should be playing in the national championship game,” Beamer said.  “I think anytime you go through the SEC and the title game in that league, you deserve to play for the national championship.

“But things are as they are.”

The question is, was Auburn really that much better than Virginia Tech? Were they better, period? The model doesn’t think so. The model picked the game 24-16 in favor of Virginia Tech (the away team on a neutral field). It nailed the Auburn score, but overshot the Hokies’ point total by 11. Or, perhaps more accurately, the Hokies undershot their expected point total.

Virginia Tech got off to a slow start in the game. Finally, in the second quarter, the offense gained some traction. Trailing 6-0, QB Bryan Randall drove Tech inside the Auburn five yard-line.

On fourth down, passing up a field goal and with everyone expecting another power run, Randall rolled out and flung a pass between two defenders to sophomore fullback Jesse Allen – who had it for an instant, then dropped it.  The ball may have been brushed by a defender’s finger, but no one was certain.

“I wasn’t sure if it was tipped or not,” Allen said.  “Randall usually puts the ball in there where you can get it.  It was my job to catch it.”

Coach Frank Beamer explained his rationale in going for seven points instead of the almost sure three at that point thusly:  “We had a chance to go up 7-6, and I felt we were lucky to be down (just) 6-0 at that time,” Beamer said.  “I felt even if we don’t make it, we’d have them backed up.”

And Auburn was backed up, until it wasn’t. The Tigers drove 92 yards to set up a short field goal. They took a 9-0 lead into halftime. At this point, Tech had left seven points on the field.

Auburn drove the length of the field on their opening possession of the second half to score a touchdown and take a 16-0, which they held at the end of three quarters. Early in the fourth quarter, the Hokies drove to the Tiger 10-yard line. However, the drive stalled, and Tech brought on First Team All-ACC placekicker Brandon Pace, who promptly missed the 23-yard attempt wide left (by an inch). Auburn’s lead remained 16-0, and the Hokies had now left 10 points on the field.

On the ensuing drive, the Hokies drove the length of the field and finally got on the board, courtesy of a 29-yard touchdown pass from Randall to Josh Morgan. Tech went for two, but failed to convert. That made the score 16-6, Auburn, and the Hokies had now left 11 points on the field (the extra point would have been nearly a sure thing).

A few minutes later, with time dwindling and Tech desperate, Randall connected with Morgan on a pump-and-go route that resulted in an 80-yard touchdown. This time Tech kicked the extra point to draw within three. The ensuing onside kick was recovered by Auburn, and the Tigers ran out the clock to secure the 16-13 victory.

It’s easy to say, well, the model expected the Hokies to convert on the 11 points they left on the field. That would have lined up perfectly with the 24-16 score. I think a more accurate conclusion would be that the model expected both teams to land on the most likely outcome for points in regard to their opportunities, i.e. convert short field goals and score touchdowns from a yard out. Once I finish coding for predicted game stats, I may find that the model expected more or less penalties or turnovers, or perhaps more success passing or rushing by the Hokies, and that accounted for the predicted point differential.

How Useful is the Model?

All of the above serves as a prelude to this section, where we quantify the usefulness of the model. Again, I have only finished coding for the predicted points, but the variables will be similar or the same for the game stats. For example, instead of looking at Offensive Explosiveness, which is part of the points predict formulas, the model will include Passing Play Offensive Explosiveness in the passing yards predict formulas.

Overall, the top line game outcome (score prediction) stats for the home team in Virginia Tech games played since 2004 are as follows:

Keep in mind these definitions:

• Sensitivity Rate - % of games the model picked the home team to win and the home team actually won

• Specificity Rate - % of games the model picked the home team to lose and the home team actually lost

• Type I Error Rate - % of games the model picked the home team to win and the home team actually lost

• Type II Error Rate - % of games the model picked the home team to lose and the home team actually won

As a check against the model I used during the regular season, I looked at how often the model predicted the correct outcome of VT home games (the in-season model’s rate was 74.7% of all games when I launched it after the UNC game).

So, at least for home games, this model improves on the accurate game outcome rate of the previous model.

I also looked at how the model fared against Las Vegas with regard to predicting the game spread. Spread data is only available going back to 2012, but that sample size is sufficient to judge performance. Here I was pleased to find that the model (barely) outperforms those who actually put their money on the line in predicting outcomes:

In sum, this looks like a pretty useful model. It does very well at predicting expected outcomes, and most of the time the expected outcome aligned with the actual outcome. We can expect it to provide useful insights for each matchup in the Virginia Tech CFP, which will be key to understanding which VT team in the last 20 years was the best and, most importantly, why.

Subscribe now

For example, can anyone name, with certainty, the best college football team of all-time? 2019 LSU looked unstoppable. So did 1995 Nebraska. Who would win a matchup of the two? We’ll never know!

But thanks to the wealth of data now tracked and easily accessible (with some programming skills), we can build models that predict how such games would go, at least among teams from the last 20 years.

As always in the world of analytics, the availability of data dictates the conclusions we can, and cannot, draw. For the purposes of this exercise, which aims to crown the best Virginia Tech football team since the 1999 National Championship team, data availability is sufficient to support an analysis of the last 20 Virginia Tech teams. That means, unfortunately, the Michael Vick-led 2000 team that went 11-1 and finished #6 in the final AP poll, as well as the 2001 and 2002 teams, will be excluded. Still, that leaves nine 10-win teams, and a few others that came close.

The Format

The Virginia Tech CFP will hew closely to the format scheduled to take effect in by the CFP in 2024. That means 12 teams (or years of VT teams, in this case) will make it to the playoff. The top 4 teams will get a bye, and rounds one and two will be played on campus (with the home team getting the full Lane Stadium statistical advantage). The semi-finals and finals will be played at a neutral site.

The Big Reveal

I went into this exercise with a pretty good idea of which teams would make the cut. In fact, I thought about just making a subjective list. However, that would be running counter to the mission of Hokie Analytics, which is to quantify and analyze all things Virginia Tech football. Therefore, I settled on a BCS-style set of computer rankings, each taking a slightly different perspective on what makes for a “great team”. Those metrics are:

• Football Power Index (FPI) - a predictive rating system developed by ESPN that measures team strength and uses it to forecast game and season results (created in 2005)

• SP+ - a tempo- and opponent-adjusted measure of college football efficiency created by Bill Connelly (ESPN)

• ELO - a rating system originally developed for chess players that rates each team based on their wins and losses

• Simple Rating System (SRS) - a rating that takes into account average point differential and strength of schedule; the rating is denominated in points above/below average, where zero is average

Each metric has its unique strengths and weaknesses, so for the sake of simplicity, I weighted the rank of each Virginia Tech team’s rating an equal 25% across the board. So, let’s say that the 2022 team’s ratings, when compared to the other 19 VT teams’, were: 20, 18, 18, and 20. That would result in an average ranking of 19. Wherever that average ranking fell in comparison to the other teams would be the seeding for the 2022 team, e.g., if every other team had a numerically lower average ranking, then the 2022 team would be seeded 20th, which in a 12-team field means they stay home for the holidays.

The top 4 Seeds (bye in the first round, home field advantage in the second round)

The following teams, listed in order of seeding, secured first round byes and will host quarterfinal matchups:

1. VT 2005 - ACC Coastal and Gator Bowl Champions, 11-2

2. VT 2009 - ACC Coastal Runners-up and Chick-fil-A Bowl Champions, 10-3

3. VT 2010 - ACC Champions, 11-3

4. VT 2004 - ACC Champions, 10-3

Seeds 5-8 (home field advantage in the first round)

5. VT 2007 - ACC Champions, 11-3

6. VT 2006 - ACC Coastal Runners-up, 10-3

7. VT 2016 - ACC Coastal and Belk Bowl Champions, 10-4

8. VT 2003 - 4th place in the Big East, 8-5

Seeds 9-12 (away team first round)

9. VT 2017 - ACC Coastal Runners-up, 9-4

10. VT 2011 - ACC Coastal Champions, 11-3

11. VT 2008 - ACC and Orange Bowl Champions, 10-4

12. VT 2015 - 4th place ACC Coastal and Independence Bowl Champions, 7-6

Teams that did not make the cut

13. VT 2013 - ACC Coastal Runners-up (tie), 8-5

14. VT 2020 - 8th place ACC, 5-6

15. VT 2014 - 5th place (tie) ACC Coastal and Military Bowl Champions, 7-6

16. VT 2012 - 4th place ACC Coastal and Russell Athletic Bowl Champions, 7-6

17. VT 2019 - ACC Coastal Runners-up, 8-5

18. VT 2018 - 3rd place (tie) ACC Coastal, 6-7

19. VT 2021 - 3rd place (tie) ACC Coastal, 6-7

20. VT 2022 - 6th place (tie) ACC Coastal, 3-8

Controversy

Fans of the 2013 team are already up in arms about their snub. Yes, they were ranked as high as 13th nationally at one point in the season, but they faded in the second half of the season and got blasted by UCLA in the bowl game. There was also a stunning lack of talent on offense, especially at wide receiver.

If any team’s outrage is justified, it has to be the 2019 Hokies. That team started slow, then almost came all the way back to win the Coastal, which would have merited an Orange Bowl birth because Clemson made the playoff that year. I thought they were a dark horse to make the VT CFP, and I could have seen it going either way, but 17th place out of 20? Ouch!

Finally, I do want to offer some comment about the most accomplished team in the field - the 2008 squad. That team, though (severely) offensively challenged, won both the ACC Championship and the Orange Bowl, making it the only VT team in history to win its conference outright and win an Alliance/BCS/New Year’s Six bowl (the 1995 team was a co-Big East Champion and received the Sugar Bowl berth because the other co-Champion, Miami, was ineligible for postseason play due to NCAA rules violations). That 2008 team barely snuck into the Virginia Tech CFP with an 11 seed. I know 2008 was a ridiculously down year for the ACC, and that team was young and at times struggled mightily to move the ball, but this seems extreme.

Ok, without further ado, here are the final playoff rankings from each of the metrics:

Looking Ahead to First Round Games

I am still building out what I believe to be a much more intricate model than I used during the 2022 regular season. Using a host of advanced statistics, I seek to accurately predict not only scores, but also standard statistics to help describe how the game went. Did one team rack up a bunch of rushing yards or live in the opposing team’s backfield? I want to know! The point of this exercise is to quantify not just what would happen, but also how it would likely happen. The date of play for first round games is TBD - perhaps next Friday, but if not, hopefully the week after. It depends on how quickly I can finish constructing the model and how crazy things get in the transfer portal (therefore necessitating some analysis and an interlude article). There are four rounds of games, and I am hoping to run them over the four-week bowl season as a fun distraction from this second postseason in three sans VT football.

Here is an early look at the first round matchups:

#12 VT 2015 (7-6) at #5 VT 2007 (11-3)

#11 VT 2008 (10-4) at #6 VT 2006 (10-3)

#10 VT 2011 (11-3) at #7 VT 2016 (10-4)

#9 VT 2017 (9-4) at #8 VT 2003 (8-5)

Alternates Policy

Since I have not yet finished building the model, I cannot say for sure that all of the seeded teams will have enough data to play. The 2003 team’s lack of standard season stats is an ominous sign. All data are sourced from collegefootballdata.com or espn.com via the R API wrapper cfbfastR. If the datasets used to power the model are not sufficient for any seeded team, that team will be replaced by the highest ranked team currently outside of the playoff. There is no reseeding. For example, if the 2003 team is a no-go, it would be replaced by the 2013 team, which would step into the #8 seed and play the #9 seeded 2017 team. In reality, I think the only teams at risk of not being able to play are 2003, 2004, and 2020 (there is a bit of weirdness in some datasets due to Covid).

Subscribe now