What Substitution Data Tells Us About Squad Depth and Managerial Impact
Digging into the data to understand whether substitution impact reflects managerial skill or the depth of your squad
Louis van Gaal believes that once a match starts there’s nothing more that he can do to influence a game.
There are tactical tweaks a coach can make here or there to help his team, but for the most part Van Gaal is right. Once the game is underway, the result is in the hands of the players. That’s not to say the manager doesn’t have a big impact on the game - after all the manager is the one who picks the players. Once the match starts the manager can still make a big impact because he can change up to five of those players.
The question is how much of an impact does changing those players actually make? That’s what we’re here to explore.
As always a note about the data sample.
I started tracking substitution data during the 2020-21 season. At first it was simply to gage how Manchester United’s substitutions were stacking up against their peers. Thus the original sample only included United’s direct rivals, plus Everton who at the time were managed by Carlo Ancelotti. Over time I slowly began adding teams like Brighton, Newcastle, Aston Villa, and Brentford to the database. This season I added Ipswich Town, Nottingham Forest, West Ham, Crystal Palace, and Fulham and back filled them for last season as well. That gives us data on 17 teams/managers. The most comprehensive sample yet.
Most football analysts agree that substitutions are good. Managers should be maximizing the amount of substitutions they make, and even making them earlier in matches to have greater impact. Kwest Thoughts published a newsletter with this argument two summers ago.
This past season the average number of subs made per game rose a smidge from 4.01 in 23-24 to 4.1 per game. They did max out their subs far more often - just over 47 percent of the matches saw a manager use all five substitutions compared to 40.5 percent a year prior. The amount of subs made at halftime also increased.
Managers who were behind in the match made their first change over a minute and a half earlier this season compared to the one before.
When the scores were level things stayed pretty much the same compared to last year. The average time of the first sub was 62.24 compared to 62.60 the year before. The times for the second and third subs when scores were level were also within a minute of what they were a year before.
Overall managers made slightly more subs and did so a little bit more aggressively and the result was… pretty much the same number1 of matches had the result change after a coach introduced his first substitution.
That’s substitution as a whole. The question is - in terms of being able to impact the match in your - is there any skill to making substitutions? Are certain coaches better at it than others?
Substitution Impact
As I always state. It’s incredibly difficult to measure the impact a managers change’s truly make on a game. To try and measure this, I use a +/- metric based on the state of the game when the managers makes his first substitution. (For a full explanation of how the metric is calculated, click here).
At first glance the +/- chart for this season makes a lot of sense. The manager of the best team in the league on top, the lowest team in the sample on bottom, and everyone else sandwiched in the middle but interestingly, this is the first season where it’s ever looked like this. Usually we get something interesting at the top of the table - such as last season when Ange Postecoglou and Erik Ten Hag topped the table with scores of +20 and +16 respectively - which speaks to the volatility of this stat.
As I have some level of data on this stat going back six years now, we can finally look at some historical data which really sheds some light on what’s probably going on here.
Let’s start with the top 15 plus/minus scores from the entire database, where those teams finished, and what happened the following year. There is one club whose results have been removed from this list which we’ll explore a bit more later.
The top three scores all come from the 2023-24 season where a sharp increase in added time lead to the explosion of goals - specifically in the second half of matches.
Marco Silva’s +13 comes from this past season so we don’t know what comes next for Fulham. Of the other 14 teams, nine* of those managers left the club the following season.2 Ange Postecoglu likely would have been sacked earlier had his team not made the Europa League final.
Ralf Rangnick was an interim who was never going to return, Graham Potter left after six games to take a “better” job at Chelsea, while Maurizio Sarri chose to leave Chelsea buttttt was that choice really his own?3 That’s still 65 percent of managers not making it through the next season with their clubs and over two out of five getting sacked.
There’s only three other managers on the list: Pep Guardiola who’s won six of the last eight Premier League titles, Mikel Arteta who has finished runner up three years in a row, and Unai Emery who previously appeared and was sacked.
Now let’s take a look at the other side of the table - the bottom 15 plus/minus scores.
Five managers got sacked in the midst of their poor substitution seasons plus the aforementioned situation with Ange Postecoglou. Interestingly five of these scores came this past season. If we again eliminate the ones from this season we’re looking at just four of 10 managers getting sacked. More interestingly, none of the managers who survived the season got sacked the following season either.
Eleven different managers comprise the bottom 15 positions. Five of those 11 also appear in the top 15. The most noteworthy of them all is Mikel Arteta. In six seasons managing at Arsenal Arteta has had five seasons in either the top or bottom of this metric. He lives at the extremes.
What’s going on there? If pressing the right buttons *substitution wise* is such a particular skill, how can one person constantly fluctuate between being so good at it one year and so bad the next?
Diving deeper into the numbers
If we go back to the top 15 plus/minus scores and look at the managers who got sacked the following season you’ll notice that for the most part their teams finished the following campaign a bit further down the table. Compare that with the managers from the bottom 15 list who got sacked. Their clubs tended to finish the following campaign pretty close to where they finished in the season where the manager got sacked.
On that chart of the top-15 scores, only two coaches managed to crack the list multiple times without getting sacked the following season. Pep and Arteta have combined to manage 12 and a half seasons at their current club but only crack the list four times. Substitution impact just doesn’t seem to translate year over year.
Then there’s Liverpool.
Remarkably consistent. If we did a true Top 15 list Liverpool would account for six of the 15 spots. No other club would have made the list more than twice! This past season they changed the manager and still ended up with the same results.
The answer we’re looking for lies right there in the chart. There’s only one season where they finished with a plus/minus less than 10 - 2020-21. That was the year they were decimated by injuries to their defense, with Virgil van Dijk, Joel Matip, and Joe Gomez all missing huge parts of the season along with midfielders Thiago, Jordan Henderson, and Fabinho missing time in front of them. As the chart on the right shows, Liverpools ability to impact games for the better wasn’t effected that season, but their ability to hold onto positive results was.
Substitutions are just the in game application of the next man up theory. The next man up theory only works if that next man’s ability is comparable to the first man. If there’s a drastic drop off in quality you’re going to have a problem.
Substitutions are far more impacted by squad depth than who is actually making the subs. For the most part, managers will typically select the best players they have available for a given game. There may be one or two changes for specific tactical reasons, but ultimately nine or ten of the best available players are going to be out there.
That means with any substitution you’re likely making your team worse. However you also have to factor in that throughout the match the players on the pitch get more tired, making them worse versions of themselves.
For example, Arsenal have Bakayo Saka, Gabriel Martinelli, and Leandro Trossard to play in their two wing roles. For arguments sake, let’s say Saka’s ability is quantified as 90 out of 100, Martinelli 80, and Trossard 70.
Saka and Martinelli are selected to start the match. As the match goes on and the two get tired, their overall ability goes down. If after 65 minutes Martinelli is now a 75, the gap between him and Trossard is much smaller, minimizing the impact between the two. Maybe after 65 minutes Martinelli is now a 67, making Trossard an improvement on him. Factor in that the opposing team is tired too and suddenly Trossard’s 70 can play more like a 73 or 74.
Now imagine Saka gets injured forcing Martinelli and Trossard to start. Instead of having a 90 and 80 starting on the wings you have an 80 and a 70, with the player on the bench only being a 60. When you get deeper into the match that 60 may be better than what you have on the pitch, but they’re still a 60. The overall composition of your team is still far worse than it used to be especially at the end of games, impacting your ability to either change a game for the better or hold on to a positive result.
For the most part team’s live in the middle of the substitution impact tables and play to their squad depth. Having a low plus minus score early in the season leads to coaches getting sacked mostly as an overreaction to bad luck.
Having a low score over the course of a season says more about your squad depth than your ability as a coach. In 2022-23 Thomas Frank made 4.44 subs per match - third most in the sample. His plus minus was +10 - largely fueled by having the 20 goal Ivan Toney, who scored seven of his 14 non-penalty goals in the final 30 minutes of matches. A year later Frank’s subs per match was 4.42 but his plus minus dropped to -12. That’s largely because he didn’t have the squad depth. Toney missed a chunk of the season serving a suspension and upon his return he just didn’t score. Every game where he made lots of subs made his team weaker and they fell apart late.
This wasn’t held against him though. Brentford didn’t sack him. They stayed the course and went from finishing 16th to 10th and Frank ended up getting a “better”- or at least higher paying - job at Tottenham.
Kieran McKenna was a -11 this season. Despite relegation there’s not been any concern about him losing his job. On the contrary actually, he’s probably come out of this season with more plaudits than he had before.
Everyone knew coming into the season that McKenna’s Ipswich Town side were completely out of their depth in the Premier League. Their sole hopes for survival were entirely based on McKenna doing some spectacular magic. They just didn’t have the talent in their squad to compete with Premier League clubs.
Yet week after week McKenna had these guys ready to punch above their weight. The Tractor Boys had a habit of starting matches very strong. Had Premier League matches ended after 75 minutes, Ipswich would have finished 17th and avoided the drop.4
It was the second half where Ipswich’s lack of talent did them in. McKenna made 4.66 subs per match this year - the third highest total - to try and keep his team fresh. But bringing on a player who isn’t good enough to start over a player who isn’t Premier League quality himself just dooms you to the inevitable.
But what about old Mikel who appeared on both lists multiple times?
Looking at Arteta and Pep is the perfect bridge to bring us back to the top of the table. Pep Guardiola does not make substitutes. He consistently ranks at the bottom of the list for subs per game. He picks the 11 players he feels will win that game and will only make a change or two, unless City are winning.
His plus minus score reflects that with one outlier. It doesn’t take long to figure out injuries played a role in that campaign. City’s three leading scorers that year were Kevin du Bruyne, Raheem Sterling, and Riyad Mahrez, but none of them even played two thirds of the season. The lack of availability meant Pep was constantly trying out different things, which lead to an increase in winning games late. The following season he rebuilt his team around Erling Haaland and just blew everyone away earlier in matches.
Arteta's worst score came in his first season when he inherited a sh*t show of a squad. As he got his team in place the scores followed suit, peaking in 2024 when Arsenal didn’t concede goals and therefore didn’t have results change late in games. They took a tumble this season when they spent long stretches with Martin Odegaard and Bakayo Saka on the sidelines.
And that volatility is exactly what makes Arteta the perfect bridge between the managers with low substitution impact scores and those with the highest. He’s one of the few to appear at both ends of the spectrum showing that high or low scores often say more about the squad context than the coach. So what happens when a manager posts a high impact score?
When you look at the low scores where the manager wasn’t sacked, Crystal Palace are the only team to come back and finish worse the following season that comes with a giant asterisk in that they won just two of their last eight games as they prioritized the FA Cup which they won. They also finished with four more points than they did the previous year. Safe to say they had a better season.
The opposite is true for teams at the top. Outside of the top three teams over the past seven years, high scores in this metric usually proceed a fall down the table, suggesting the high score is giving you an artificial boost up the table.
There are certainly situations where teams are built to win games late. Sometimes managers will willingly start a worse player every week because the better one just can’t go 90 minutes and his impact is greater coming off the bench. Ole Gunnar Solskjaer used Edinson Cavani to great effect in this role in 2020-21.
But for the most part, high plus minus scores seem to come from one of two things. The manager is often getting his starting XI wrong, either because he doesn’t know his best XI or because he’s so married to a system that other teams have figured it out. The system consistently doesn’t work leading to him to say fuck it and make wholesale changes in the second half where he gets lucky.
That’s not a sustainable formula. In 2018-19 Unai Emery made a halftime substitution in 18 different games - by far the most of any manager in a single season. This was before we expanded to five substitutions and encouraged teams to make subs at halftime! That doesn’t scream someone who knows what their best team is, and it shouldn’t be a shock that a year later Arsenal were -13 across two managers.
In the end, Louis van Gaal might have been more right than he realized. Once the match begins, the manager’s influence dwindles. Substitutions are one of the few tools left at a manager’s disposal during a game, but their effectiveness depends far more on squad depth than managerial genius. Managers with high substitution impact scores aren’t always magicians; sometimes, they’re just the ones with more cards to play. And if those cards are weak, even the smartest tactical moves can’t change the result.
proportionally - the amount of “results changed” rose from 33.68% to 34.06%
Ange was technically sacked after the season but it’s well known he would have been sacked earlier had Spurs not made it to the Europa League final
Had Sarri not chosen to go to Juventus were Chelsea really going to bring him back?
The team that would have finished 18th and gone down? Manchester United