The NFL has handed Carnegie Mellon University something no other research institution has: official game film, terabytes of player-tracking data, and a seat at the table to build football AI that can learn the sport the way a devoted fan learns it.
This football AI article breaks down what CMU actually received, how the researchers plan to turn film and sensor data into a model that understands plays, predicts them and eventually simulates them, and where the football AI effort could still fall short. Every fact is sourced from the research team and the league; everything else is clearly flagged as analysis.
Table of Contents
- Inside CMU’s Football AI Project
- What Game Film Can Teach a Model
- The Tracking Data Behind the Plays
- Can Foundation Models Learn Football Intelligence?
- Two Building Blocks, One Model
- From Watching Plays to Predicting Them
- Why Coaches and Scouts Care
- What Fans Could See Next
- Limits, Risks and Open Questions
- What to Watch Next
Inside CMU’s Football AI Project
The access nobody else has
The NFL gave CMU researchers its official film footage and on-field data, and the researchers made the exclusivity explicit. “We are the only institution that’s conducting research with this data,” said Ron Yurko, director of the Carnegie Mellon Sports Analytics Center. That single sentence tells you how the league is running this football AI experiment: one partner, one lab, one controlled look at what modern machine learning can do with a century of football.
The people building it
Two researchers lead the push. Chenyan Xiong is an associate professor in the Language Technologies Institute at CMU’s Kenneth C. Griffin School of Computer Science, and a self-described football nerd who keeps footballs signed by Ben Roethlisberger in his office. Yurko directs the sports analytics center and teaches in the Department of Statistics and Data Science. Their combined pitch for the football AI project to the league: pair language-and-multimodal model expertise with sports statistics, and build AI that genuinely learns the game.
The program behind the deal
The work runs through the NFL Forward Initiative, a program that lets entrepreneurs and researchers pitch technological solutions for competition, training, player development and growing the sport. Mike Lopez, the NFL’s senior director of football data and analytics, said the league is “honored to work with a prestigious institution like CMU” and called innovation “central to how we advance the game.”
| Project snapshot | Detail |
|---|---|
| Host institution | Carnegie Mellon University (only research partner with this data) |
| League program | NFL Forward Initiative |
| Data provided | Official game film + Next Gen Stats player tracking (several terabytes) |
| Research leads | Chenyan Xiong (Language Technologies Institute) & Ron Yurko (Sports Analytics Center) |
| Stated goal | A foundation model that understands, predicts and simulates football plays |
Why one partner matters
A single exclusive research partner is a deliberate control choice by the NFL, not an accident. One lab means one point of failure, one publication pipeline and one relationship to manage while the league tests whether modern machine learning can actually respect the texture of football. For CMU, the arrangement raises the stakes in the opposite direction: the football AI work will be measured against a league that watched every step and can close the door quietly if the results disappoint.
What Game Film Can Teach a Model
A hundred years of tape
Every play of every game, kept on film, is one of the largest structured records any sport owns. The NFL gave CMU access to that archive in full: game footage that shows down, distance, formation, motion and outcome, thousands of times over. For a field trying to teach machines to see, few datasets this rich exist outside of driving and medicine.
From watching to understanding
The first job for the football AI pipeline is comprehension. “The first step is essentially to understand what’s happening on the field,” Xiong said. Before any model can predict a blitz or a coverage rotation, it has to recognize what a blitz and a coverage rotation look like from the booth camera angle, across weather, stadium lighting and a hundred broadcast quirks.
Why video is the hard part
Film carries everything, but it hides it in pixels. Teams that work with video at scale keep learning the same lesson: extraction is brittle, and recognition models drift when the visual world changes. A football AI video system, trained on the league’s own footage, is the answer CMU is betting on. AI video tools keep maturing fast elsewhere too, as we covered when Pangram expanded its detection models to video, and the same architectural wave is what makes a football model feasible now.
Film also carries a practical advantage over pure sensor feeds: it is the same input a human coach uses. When the model and the coordinator are looking at the same angles, the same formations and the same outcomes, explaining a football AI model’s read of a play stops being a translation problem. The comparison that matters is not model against truth but model against the people who already study tape for a living, and that is a benchmark a football audience can intuitively trust.
The Tracking Data Behind the Plays
RFID in the pads
Alongside the film, CMU received the NFL’s Next Gen Stats player-tracking data, powered by Amazon Web Services. It is sourced from RFID chips embedded in players’ shoulder pads, in game equipment and in the ball itself, producing real-time location information for everything that moves on the field.
Coordinates the eye cannot keep
Where film shows the play, tracking data measures it: every route, every separation, every shift in the secondary, sampled continuously. A human scout watching the same rep sees the ball; the tracking feed remembers all twenty-two players at every instant. That is exactly the signal a football AI model needs to learn concepts like spacing, leverage and coverage shells without being hand-labeled.
Terabytes change the training math
Xiong and Yurko received several terabytes of combined film and tracking data, and that volume is why they are turning to foundation models rather than hand-built features: more data lets one model absorb more situations and handle more complex tasks. It also mirrors a broader shift across AI infrastructure, where new communication and memory tools keep making heavy data traffic cheaper to move and store.
Can Foundation Models Learn Football Intelligence?
The football nerd plan
Xiong does not hide the motivation. He wanted to build this football AI because he loves the game, and his plan is unusually honest about the bar: “We want to see whether we can train foundation models on the data to learn the intricacies of football intelligence. In this case, football intelligence is comparable to the knowledge of the average fan.”
Understanding, prediction, recognition
The stated football AI capability ladder is concrete. The model should understand plays; potentially make play predictions; and, once it sees a play, identify whether that play has been run before, perhaps by another team. Recognition of a play from memory is the fan benchmark in its purest form: the thing a backyard analyst does by shouting “I’ve seen this before!”
Why sport is a good testbed
Football is a rare domain with complete records, bounded rules and unambiguous outcomes, which makes it a cleaner evaluation target than most open-world AI problems. The open-model movement has lowered the cost of trying experiments like this, and as we argued in our profile of Chris Alexiuk’s open-models work, accessible tooling keeps shifting who gets to build serious AI systems.
The “average fan” bar deserves a second look, because it is both modest and enormous. Modest: no fan calls audible-proof plays or grades every rep. Enormous: an average fan carries ten thousand remembered snapshots of formations and reactions, a gut sense of which looks precede which outcomes. Reproducing that library statistically, without hand-built features, is precisely the bet foundation models make. If football AI can match a couch, it strengthens the case that pattern intuition itself is learnable from raw experience.
Two Building Blocks, One Model
Start smaller
Before attempting one football AI model that swallows film and sensor readings together, the team split the problem. They built two pretrained systems, and each does the same job from a different sense: learn football. One system works from the game film. The other works from the player-tracking data.
The fusion problem
The real football AI target is a model that learns from the combination of both, and the researchers are candid that getting there could require more data and more time. Multimodal fusion is where most “it works in the paper” systems stall: two strong single-modality encoders do not automatically compose into one coherent understanding of a play.
Building blocks compared
| Building block | What it consumes | What it must learn |
|---|---|---|
| Film system | Official NFL game footage | Plays, formations and outcomes from pixels |
| Tracking system | Next Gen Stats coordinates from RFID chips | Routes, spacing and movement structure |
| Fusion model (goal) | Both streams combined | Football intelligence: understand, predict, simulate |
Pretraining first is the standard modern recipe for exactly this shape of problem: collect huge raw data, teach one representation to compress it, then move on to harder tasks. The CMU team describes both systems as building blocks for a final model, which means today’s deliverable is competence at learning football from two very different senses. The eventual fusion step inherits whatever each block learned, so weaknesses in film recognition or movement understanding will compound rather than cancel.
From Watching Plays to Predicting Them
The next-play question
Once the model understands what happened, the team wants it to look forward. “Once the model understands what happened on the field after the play, can it make predictions before the next play?” Xiong asked. “Is there going to be a blitz? Will this safety rotate to a different coverage?” Those are pre-snap questions, and answering them means the football AI model has internalized tendencies, down-and-distance behavior and formation logic.
Simulation and play calls
Further out, Xiong described a football AI model that could simulate how a play progresses given certain calls, or even suggest tactical play calls based on estimated outcomes. That is the same world-model logic now spreading through AI research, where systems learn to predict what happens next; our piece on 42AI’s world model for predicting human behavior covers the broader trend, and football is a natural proving ground because every down is a labeled experiment.
How far off is it?
The chart below rates each rung of the capability ladder, using only the information the team has made public plus this author’s assessment. Read it as a map of ambition versus readiness, not as measured accuracy.
Capability ladder for the CMU football AI project (directional, author’s assessment)
Xiong’s two example questions are well chosen for skeptics and optimists alike. A blitz is a visible, binary, pre-snap event that a good model can learn to smell from pressure counts and safety depth. A coverage rotation is subtler: it is a coordinated change of responsibility among defensive backs, often disguised until the snap. Predicting the first is a tendency problem; predicting the second is a theory-of-mind problem about how defenses conceal intent, and both live inside what the football AI pipeline must eventually model.
Why Coaches and Scouts Care
The hours problem
Yurko put the status quo bluntly: studying football data has historically meant coaches and scouts sitting through hundreds or thousands of hours of footage, manually annotating what happened on each play. That work is paid, repetitive and lossy, and it is exactly the kind of task modern vision systems are built to absorb.
Focus on what matters
The promise of football AI here is not a robot coordinator calling games. It is automation of the boring middle: “It’s going to help coaches and scouts focus on what matters most and can help the league be more efficient,” Yurko said. When annotation happens at machine scale, humans spend their hours on judgment instead of bookkeeping.
The efficiency pitch
The league frames the same upside from its side, with Lopez calling innovation “central to how we advance the game.” Efficiency language matters here: an NFL that believes AI cuts film-room costs while keeping football’s texture is an NFL that keeps opening more doors to researchers.
Scouting adds a second pressure point on top of film study. Front offices evaluate thousands of college players each year, and every evaluation ultimately rests on limited college tape translated through scheme differences. If automated annotation can describe what a linebacker did against a given front, decision-makers can ask consistent questions across seasons and programs instead of trusting whichever clips a regional scout chose to highlight.
What Fans Could See Next
From draft chatbot to game oracle
Fans have already met Yurko’s work. During the 2026 NFL draft he integrated research into the NFL IQ AI Chatbot, which let users ask questions and get answers from a model. The CMU football AI project is the next rung of that same ladder: from answering questions about prospects toward answering questions about the game itself, live.
Watch the game differently
The stated goal of the whole program is to change how professionals and fans annotate, analyze and understand games. A football AI model that recognizes whether a play has been run before, by any team, turns casual viewing into pattern spotting at archive scale. Broadcasts get a new layer of storytelling; fans get arguments settled by retrieval instead of memory.
Broadcast’s hungry machinery
Leagues adopt whatever deepens engagement without breaking the product, and football’s broadcast layer has been hungry for fresh statistics for two decades. Analytics already moved from front-office curiosity to pregame show; a football AI model that explains plays in plain language fits neatly into the same slot.
Limits, Risks and Open Questions
Research, not product
Nothing in the football AI pipeline is shipping this season. This football AI effort is a research proposal accepted into a league program, with two pretrained systems and a roadmap. Anyone expecting a play-calling bot on broadcast by opening week has misread the stage, and the team is careful not to promise timelines.
The exclusivity question
CMU is the only institution with this data, which protects the experiment and narrows its science at the same time. Findings that cannot be reproduced on the same footage by other labs will be harder to verify, and the league controls what gets published. That is a normal tension between proprietary data and open research, not a scandal, but it is worth naming.
Football resists prediction
Football is the most stochastic of the major sports: single-play outcomes hinge on bounces, whistles and one-on-one collisions that no model fully controls. Expect the football AI model’s prediction ceilings to be humble even when its understanding is strong, and judge the project on comprehension first.
Reproducibility cuts both ways in a deal like this. Outside labs cannot rerun published results on the same film or the same tracking feeds, so the field inherits the NFL’s accounting of what the data showed. Peer review can evaluate method and claims, but it cannot independently verify them. The healthiest version of this arrangement would see the league publish aggregate benchmarks even while keeping raw data closed, giving the football AI community something to argue against.
What to Watch Next
The fusion milestone
The football AI signal to watch is the first published result that combines film and tracking streams in one trained model. Until that exists, everything else is componentry; the moment it exists, the league’s stance on opening the data further will be tested in public.
The league’s next moves
The Forward Initiative exists to produce results the NFL can use, and a visible success here invites the next cohort, maybe a second institution, maybe other sports copying the offer. Watch for the program’s next class and for any public benchmark the NFL lets researchers share.
Why this matters beyond football
Domain-specialized foundation models are the quiet trend of 2026, and a football AI model that reaches fan-level understanding would be one of the most quotable proofs that foundation models transfer beyond language. If the CMU football AI effort gets there, expect every sports league to start cataloguing its own terabytes.
The ripple effects may matter more than the model itself. A league-embedded research program that survives its first public season becomes a template: other sports own comparable archives, comparable tracking investments and the same film-room economics. Each adoption decision then rests less on whether the science works and more on whether executives believe an external lab can be trusted with the crown jewels. The CMU experiment answers the first question in public; the second is left to every league watching from behind.
References
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.
