This post has been a long time coming – partially because we haven’t posted in a while - but also because we’ve evolved our Expected Points model to include work done at the breakdown and we want to share our findings with you. One of the limitations of our previous framework to evaluate player performance was the lack of measurement around the actions of players at the breakdown. However, there has been a relatively recent initiative from 3rd party data providers to include greater detail around player contributions at the breakdown, so it only makes sense to start incorporating this new data into our framework to provide a more complete picture of performance at both the individual and team levels. The breakdown data included information around both attacking and defensive breakdowns as well as qualifiers for how effective the players were in their participation. For this study, a total of 1,203,249 breakdown involvements taking place across multiple competitions from 2020-2023 were analyzed to generate an Expected Points model.
For our evaluation framework, attacking breakdowns included the following details:
Order of arrival
Field position of breakdown
Player effectiveness at the breakdown (successful cleanout, failed cleanout, attended ruck, secured ruck)
Was a penalty conceded?
Defensive breakdowns included the following details:
Order of arrival
Field position of breakdown
Player effectiveness at the breakdown (got cleaned out, not cleaning, nuisance)
Was a penalty conceded?
Was a penalty won?
Was a turnover won?
Contributions to the breakdown can be positive or negative and can vary in magnitude depending on how much the action increases/decreases a team’s probability of scoring. Details around individual player involvements at both the attacking and defensive breakdowns are assigned an expected points value and can be summed to produce an overall score. This total score can be referred to as the “Breakdown Presence” which was a phrase coined by Tabai Matson back in 2015 when he was an Assistant Coach with the Crusaders hand coding games and trying to measure player contributions to the breakdown. As an initial test for the model, we identified the top 5 players in Total Breakdown Presence for the current 2024 Gallagher Premiership season. Does this list pass the eye test?
TABLE 1: Top 5 players for Breakdown Presence in the Gallagher Premiership for the current 2024 season after 13 rounds of play. NOTE: Values may not sum exactly to the totals in the table due to rounding artifact.
To further test the model, we summed the Breakdown Presence scores at the team level in the following competitions for the 2022 and 2023 seasons and correlated them to Table Points.
Gallagher Premiership
Super Rugby Pacific
Top 14
United Rugby Championship
The total Breakdown Presence scores had a moderate/strong correlation to Table Points (Figure 1; r = 0.6781). While this is a promising finding for a preliminary study, further work will be required to see if there are ways to remove some noise and potentially strengthen the relationship. Perhaps this could be achieved by filtering out “low value” contributions to rucks (e.g. attending a ruck that is already won or possibly any ruck contribution with an |EPA| < 0.01) and focusing more on those actions that have greater positive/negative contributions.
FIGURE 1: The relationship between Breakdown Presence and Competition Table Points in Gallagher Premiership, Super Rugby Pacific, Top 14, and United Rugby Championship (2022, 2023).
Our original EPA evaluation model (herein referred to as “EPA OG”) was last revised here. For this same analysis, if we took all player EPA OG values and rolled them up to the team level, we would find that there’s a very strong correlation between Total EPA OG and competition Table Points (Figure 2; r = 0.8308). This validates the EPA OG metric as a potential tool for player evaluation despite a lack of detail around the breakdown. Will adding the breakdown data improve the model?
FIGURE 2: The relationship between Total EPA OG and competition Table Points in Gallagher Premiership, Super Rugby Pacific, Top 14, and United Rugby Championship (2022, 2023)
One of the advantages of using Breakdown Presence to evaluate performance at the breakdown, is that since it uses the Expected Points framework it can easily be combined with EPA OG. Since they both use the same units of measure, they can simply be added to achieve a combined score. When this combined score (Total EPA OG + Breakdown Presence) is plotted against competition Table Points, the resulting correlation is very strong (Figure 3; r = 0.8604). What is remarkable about this result is that the correlation coefficient for the combined (Total EPA OG + Breakdown Presence) measure is actually GREATER than the individual correlation coefficients for each measure when looked at in isolation.
FIGURE 3: The relationship between (Total EPA OG + Breakdown Presence) and competition Table Points in Gallagher Premiership, Super Rugby Pacific, Top 14, and United Rugby Championship (2022, 2023)
Thus, the combined (Total EPA OG + Breakdown Presence) appears to be an improvement from the Total EPA OG metric. Obviously, we will need to further evaluate the measure as we accumulate more data over the coming seasons. However, for now, we will use this as our new measure to evaluate both player and team performances. To clarify, this new combined (Total EPA OG + Breakdown Presence) will now simply be referred to as “Total EPA”. If this results in confusion with the Total EPA OG measure then perhaps we will have to consider renaming the metric altogether?
To bring things full circle, let’s revisit the players identified as the top 5 in Total Breakdown Presence in the Gallagher Premiership and see how much their TOTAL EPA scores change (Table 2).
TABLE 2: Top 5 players for Total EPA OG, Breakdown Presence, and Total EPA (revised) in the Gallagher Premiership for the current 2024 season after 13 rounds of play.
Alex Dombrandt scored the highest in Total EPA as his Total EPA OG was nearly double the nearest player in the table - giving credit to his excellent work away from the breakdown and providing further support for his selection as an England international. Will Evans, Hanro Liebenberg, and Miles Reid have reputations as back row players that are breakdown specialists – and these numbers support that perception. While these players are experts in their craft (i.e. the breakdown) perhaps they have other aspects of their game that need to be brought to another level (team roles permitting?) in order to be considered for international selection. The interesting case here is Thomas duToit who is a prop. Historically props typically do not achieve high scores in Total EPA OG. There are many reasons for this and perhaps this is a topic that could be discussed in a future post. Regardless, duToit’s Total EPA OG have him ranked 3rd amongst props which is very respectable. However, duToit’s point of difference is his work at the breakdown as he is the top ranked prop in Breakdown Presence. As a result of this, duToit is currently the top ranked prop in terms of Total EPA.
Breakdown Presence is a metric which assigns expected points to various actions that occur at the breakdown in both attack and defence. Our original player evaluation model did not include details around breakdown contributions, and this was an acknowledged gap in the model. The recent inclusion of breakdown data by 3rd party data providers have given us the means to include this important aspect of the game. The combination of our former model (Total EPA OG) with Breakdown Presence have resulted in a revised metric which has a stronger correlation to competition table points. The revised “Total EPA” will be the new standard for how we will evaluate both player and team performances.
Future work in this area could include:
Further optimization around which specific actions around the breakdown have the most impact and which ones are low value
Does Breakdown Presence and Total EPA predict performance at the international test rugby level?
Does Breakdown Presence and Total EPA translate to 7s rugby?
What other areas do you think Breakdown Presence and Total EPA could be used to provide new insights in rugby?