Statistical Domination

[repman 634]

43 minutes ago, nutty nigel said:

I would imagine there's a big team at Colney looking at stats and breaking them down to areas where they're useful. Xg won't be as simple as the crude figures we look at. It seems to me our right side is targeted a lot. It could be the right side of our defence is weakest at stopping crosses or it could be that the left side is weakest at dealing with those same crosses. Only the stats guys could know.

This is exactly it for us fans. Looking at xG stats which are freely available to the public is good, and goes at least a little way to helping understand the performance of the team. These xG stats though only measure when a player takes a shot. Plenty of people have posed questions like what if x had passed to y instead who would've had an open goal? or maybe the player simply didn't get a shot off, therefore it doesn't show up in the xG. Clubs will be well aware of these oversights, and lots of work will be being done to model everything correctly (it has been done already I'm fairly sure). The data and models required for this information is time consuming and hard to develop, meaning that it also costs plenty of money. We are still very much in a growth stage of data in football, behind the scenes there is an arms race between data companies to provide the best data so for a while we will still be seeing reasonably basic public stats.

[hogesar 9,618]

23 minutes ago, littleyellowbirdie said:

If there's one upside to the downturn in our XG it's that it has restored the credibility of statistical modelling in football almost overnight. 

Hahahah I like this 😂

[littleyellowbirdie 2,511]

9 minutes ago, repman said:

This is exactly it for us fans. Looking at xG stats which are freely available to the public is good, and goes at least a little way to helping understand the performance of the team. These xG stats though only measure when a player takes a shot. Plenty of people have posed questions like what if x had passed to y instead who would've had an open goal? or maybe the player simply didn't get a shot off, therefore it doesn't show up in the xG. Clubs will be well aware of these oversights, and lots of work will be being done to model everything correctly (it has been done already I'm fairly sure). The data and models required for this information is time consuming and hard to develop, meaning that it also costs plenty of money. We are still very much in a growth stage of data in football, behind the scenes there is an arms race between data companies to provide the best data so for a while we will still be seeing reasonably basic public stats.

But these factors do automatically get taken into account in xg. If a player stops taking 0.1 shots in favour trying to progress the ball to a position where there can be a 0.5 shot, but doesn't manage to get as many shots in then it's automatically taken into account; the 0.1 shots disappear from the stat and the 0.5 shots not being there to make up for it will be evident, so you've actually seen the outcome of changing the approach in the XG stats. And if he never gets a shot off in the first place then there's a 0 probability of scoring, so it works: You know you were better off going for the 0.1 shots. 

When you think about it, the modelling of the shots is pure common sense; turning it into numbers just allows you to process the outcomes in an objective mathematical way to find out what's most effective. 

Edited October 27, 2022 by littleyellowbirdie

[repman 634]

22 minutes ago, littleyellowbirdie said:

But these factors do automatically get taken into account in xg. If a player stops taking 0.1 shots in favour trying to progress the ball to a position where there can be a 0.5 shot, but doesn't manage to get as many shots in then it's automatically taken into account; the 0.1 shots disappear from the stat and the 0.5 shots not being there to make up for it will be evident, so you've actually seen the outcome of changing the approach in the XG stats. And if he never gets a shot off in the first place then there's a 0 probability of scoring, so it works: You know you were better off going for the 0.1 shots. 

When you think about it, the modelling of the shots is pure common sense; turning it into numbers just allows you to process the outcomes in an objective mathematical way to find out what's most effective. 

By your reckoning shooting is, objectively, always the best thing to do. I bet anyone watching a game of football could point to at least 1 occasion where a players takes a shot instead of choosing a 'better' passing option. But when it comes to measuring performance, clubs will want more than just a shot based xG in their analysis. 

[littleyellowbirdie 2,511]

4 minutes ago, repman said:

By your reckoning shooting is, objectively, always the best thing to do. I bet anyone watching a game of football could point to at least 1 occasion where a players takes a shot instead of choosing a 'better' passing option. But when it comes to measuring performance, clubs will want more than just a shot based xG in their analysis. 

Not necessarily. If shirking the 0.1 chances in favour of trying to create 0.5 chances means you finish up with enough 0.5 chances to outweigh the lost 0.1 chances then it'll show in the overall xG as having been a good move. 

Edited October 27, 2022 by littleyellowbirdie

[repman 634]

1 minute ago, littleyellowbirdie said:

Not necessarily. If shirking the 0.1 chances in favour of trying to create 0.5 chances and you finish up with enough 0.5 chances to outweigh the 0.1 chances you've lost then it'll show in the stats as having been a good move. 

Yes but my initial post was about the quality of data used behind the scenes in clubs being far ahead of what's publicly available. 

Generally, even if you don't get a shot off you would be better off not taking pot shots from outside the box, it's one of the first things xG pointed out.

[nutty nigel 7,509]

But all this probably changes after considering what the defensive teams xg against stats are.

It's ridiculously complicated and gives me a skull ache...

🙃

Edited October 27, 2022 by nutty nigel

[littleyellowbirdie 2,511]

12 minutes ago, repman said:

Yes but my initial post was about the quality of data used behind the scenes in clubs being far ahead of what's publicly available. 

Generally, even if you don't get a shot off you would be better off not taking pot shots from outside the box, it's one of the first things xG pointed out.

Back to first principles. Forget statistics for a minute. 

What's the most important thing in football? Goals. Not passes, not pretty nutmegs, not a nice lob over the keeper; it's the ball going in the net. 

In reality every goal is different, but you can say that the probability of scoring close to goal in the box. The shot from the halfway line that happens to go over the keeper and into the net is 'lucky' or in other words you would expect it to actually work out 1 time in 1000 for the sake of argument. 

Then again, you've got the shot from a yard in front of the goal that somehow comes off his foot wrong and goes over the bar. That's 'unlucky', or the shot you'd expect to go in 9 times out of 10. 

So lots of nerdy people decided that maybe it'd be better to try and take luck out of the equation and start looking at probabilities of success from different shots and consider what you'd 'expect' to happen with the shots that are made analysing the probabilities of shots from any given position going in based on real world data on how frequently shots from different positions go in. 

At this stage, you have a numerical way of comparing the game where you tried taking more pot shots from the half way line to the game where you tried working the ball up into the box more. 

To summarise xG is attempting to model how many goals you should have scored purely based on performance, taking 'luck' entirely out of the equation. 

 

Edited October 27, 2022 by littleyellowbirdie

[It's Character Forming 1,160]

On 26/10/2022 at 14:58, hogesar said:

So, continuing the somewhat downward trend, here is the graph from Sheffield Utd away.

Note that our xG includes one pen which carries (I think) 0.76 xg score. So let's take that out, making our xG 0.45 for the night.

First thing to note is, anyone criticising Pukki for the penalty miss should probably think twice because he's scored 2 goals he had no right to score. They were far from clear chances.

Secondly, none of our players created a clear chance for anyone. The xA list is Sheff Utd players only. 

The point, or perhaps problem, is this xG is now marrying up with what people are seeing on the pitch.

This differs from the start of the season where people were bemoaning the team and the manager after the Wigan game at home. At that point it was clear we weren't getting points we perhaps deserved to get.

This time, we were lucky to get a point.

 

This is grim reading.  For me the problem with the idea that "we should give Smith more time", is that we've been on a downward trajectory for quite some time now, there's no sign of improvement in any aspect.

[daly 500]

Might have different statistics had Sargent taken the penalty 

[repman 634]

16 minutes ago, littleyellowbirdie said:

Back to first principles. Forget statistics for a minute. 

What's the most important thing in football? Goals. Not passes, not pretty nutmegs, not a nice lob over the keeper; it's the ball going in the net. 

In reality every goal is different, but you can say that the probability of scoring close to goal in the box. The shot from the halfway line that happens to go over the keeper and into the net is 'lucky' or in other words you would expect it to actually work out 1 time in 1000 for the sake of argument. 

Then again, you've got the shot from a yard in front of the goal that somehow comes off his foot wrong and goes over the bar. That's 'unlucky', or the shot you'd expect to go in 9 times out of 10. 

So lots of nerdy people decided that maybe it'd be better to try and take luck out of the equation and start looking at probabilities of success from different shots and consider what you'd 'expect' to happen with the shots that are made analysing the probabilities of shots from any given position going in based on real world data on how frequently shots from different positions go in. 

At this stage, you have a numerical way of comparing the game where you tried taking more pot shots from the half way line to the game where you tried working the ball up into the box more. 

To summarise xG is attempting to model how many goals you should have scored purely based on performance, taking 'luck' entirely out of the equation. 

 

Yes I know what xG is.

My point still remains though. Free, publicly available xG models are good for analysis, easily the best way for anyone outside of professional football to try to predict future performance. However, clubs will be using much more granular data, combined with more complex modelling in order to do their analysis. Will this produce wildly different answers to the basic xG? I don't know but I'd suspect probably not. All I was saying is that the data analysis done within professional football, by people employed by clubs to do such a thing, is being done to a far deeper level than we can here.

[littleyellowbirdie 2,511]

33 minutes ago, nutty nigel said:

But all this probably changes after considering what the defensive teams xg against stats are.

It's ridiculously complicated and gives me a skull ache...

🙃

XGA is literally just turning it on its head. It's looking at where and how often the opposition are shooting at your goal. If most oppositions are getting off the odd 0.1 you're defending well. If they're getting off lots of 0.6s and 0.7s then there's probably something wrong with your defence. 

As to whether there's a way of trying to marry up teams' XGs against the other team's XG against, I confess to having no idea. 

[nutty nigel 7,509]

1 minute ago, littleyellowbirdie said:

XGA is literally just turning it on its head. It's looking at where and how often the opposition are shooting at your goal. If most oppositions are getting off the odd 0.1 you're defending well. If they're getting off lots of 0.6s and 0.7s then there's probably something wrong with your defence. 

As to whether there's a way of trying to marry up teams' XGs against the other team's XG against, I confess to having no idea. 

Well there is xg difference I guess...

[littleyellowbirdie 2,511]

13 minutes ago, repman said:

Yes I know what xG is.

My point still remains though. Free, publicly available xG models are good for analysis, easily the best way for anyone outside of professional football to try to predict future performance. However, clubs will be using much more granular data, combined with more complex modelling in order to do their analysis. Will this produce wildly different answers to the basic xG? I don't know but I'd suspect probably not. All I was saying is that the data analysis done within professional football, by people employed by clubs to do such a thing, is being done to a far deeper level than we can here.

Ah, gotcha. There's no way they'll be wildly different, just that little bit more accurate in the long haul. To use an analogy, you can model the orbit of a satellite perfectly well with Newtonian mechanics for the purposes of a TV satellite, but you can do it that little bit better if you throw relativity into the mix, and if you're dealing with atomic clocks on GPS satellites then relativity becomes important. 

By the same token, the guy looking at placing a bet on winning or losing will be alright with ballpark probabilities; the manager trying to decide the relative merits of encouraging more shots from outside the box or working on more buildup needs as good a model as possible. 

Edited October 27, 2022 by littleyellowbirdie

[nutty nigel 7,509]

I wonder if my grandad could have used all this for his pools coupon. He used to have a little roulette wheel to pick the numbers. No wonder he never won...

🙃

[TIL 1010 4,722]

12 minutes ago, nutty nigel said:

I wonder if my grandad could have used all this for his pools coupon. He used to have a little roulette wheel to pick the numbers. No wonder he never won...

🙃

My dad used birthdays so we never got down as far as the Jock fixtures.

[nutty nigel 7,509]

12 minutes ago, TIL 1010 said:

My dad used birthdays so we never got down as far as the Jock fixtures.

True story...

When @Duncan Edwards was little he asked a proud Scotsman if teams relegated from the fourth division went into the Scottish League🙃

To be fair that's how it appeared by looking at the league tables in the paper...

[hogesar 9,618]

4 hours ago, It's Character Forming said:

This is grim reading.  For me the problem with the idea that "we should give Smith more time", is that we've been on a downward trajectory for quite some time now, there's no sign of improvement in any aspect.

Yes, I agree. There was a point midway-ish through our good run where we were improving pretty consistently on chance creation and generally conceding few. I defended Smith at this point believing this trend alongside players coming back from injury should see further improvement. Unfortunately the complete opposite has happened.

[Robert N. LiM 4,243]

Interesting article on this topic. @hogesar

https://www.theguardian.com/football/blog/2022/oct/31/data-has-improved-our-understanding-of-football-but-it-remains-a-sport-of-luck?CMP=Share_AndroidApp_Other

[hogesar 9,618]

45 minutes ago, Robert N. LiM said:

Interesting article on this topic. @hogesar

https://www.theguardian.com/football/blog/2022/oct/31/data-has-improved-our-understanding-of-football-but-it-remains-a-sport-of-luck?CMP=Share_AndroidApp_Other

Thanks Robert!

Luck is such a huge factor and I think we / I discussed that earlier on in the thread.

If you're interested, this book is really worth a read:

https://www.amazon.co.uk/dp/1089883188/ref=sr_1_2?crid=3IY8FZB3FNOS6&keywords=the+expected+goals+philosophy&qid=1573369702&sprefix=The+Expected+goals%2Caps%2C145&sr=8-2

 

[Robert N. LiM 4,243]

1 hour ago, hogesar said:

Thanks Robert!

Luck is such a huge factor and I think we / I discussed that earlier on in the thread.

If you're interested, this book is really worth a read:

https://www.amazon.co.uk/dp/1089883188/ref=sr_1_2?crid=3IY8FZB3FNOS6&keywords=the+expected+goals+philosophy&qid=1573369702&sprefix=The+Expected+goals%2Caps%2C145&sr=8-2

 

cheers, will check that out - have been meaning to read it.

While we're doing book recommendations, I read this recently and found it very interesting. https://www.amazon.co.uk/Making-Decisions-brilliant-smart-thinking-leadership/dp/0008530149

It's by the former England cricket selector Ed Smith, and is mostly cricket focused. I think you'd have to be interested in cricket to really enjoy it. But he does touch on football (Liverpool and Brentford get a fair few name-checks). But it's basically about how you use data to help you make decisions, and where the data and the human judgment interact. I thought of this thread quite a few times while I was reading it. 

Pink 'Un Book Group!

[It's Character Forming 1,160]

The way I think of it is that it's like rolling the dice. 

 

Say you've got an xG outcome of Norwich 2 vs Stoke 1.5.

You could simulate this by rolling for Norwich with 5 dice, using 10-sided dice, and Norwich "score" on a 1-4.  Whereas Stoke also have 5 dice but only get to score on a 1-3.  So on average, you'll get a draw or a narrow Norwich win, but in a one-off game Stoke could easily win via a couple of lucky rolls.  Equally you can get an outcome like Saturday with Norwich winning 3-1 which is a bit flattering to us.

 

xG tries to look under the bonnet by seeing how well the two sides have really done in creating chances, accepting that on average, it's rare to have a chance that is over a 0.5 chance of scoring (except penalties) so typically a striker will miss any given chance more often than he will score.  And the outcome will generally turn on a handful of chances and whether they actually go in the net or not.

 

One interesting thing about football is that often the entire season can come down to a handful of results or even a single game, which in turn are decided by a small number of chances, and which could go either way.  People say "it all evens out over the season" but often it doesn't.

 

xG is an awfully long way from perfect but it does add to our understanding.  When you compare it to the other stats that we're always given like possession % or number of corners, xG is streets ahead.

[dsr-burnley 25]

On 27/10/2022 at 15:19, Robert N. LiM said:

Obviously the penalty was, to a great extent, Hanley's error in terms of him raising his hand. But it was also the result of Burnley getting in another decent position, the wide player getting the other side of McCallum and creating a decent crossing opportunity. So I think it's a little unfair to say it wasn't 'determined' by the attacking team.

But most of the xG charts I've seen do explicitly include penalty details in their summary, as Hoggy's Sheff Utd chart does, so you can see when the overall xG includes the 'gift' of a pen.

One of the difficulties of xG in raw from is that big differences come from little variables.  For example, one of Burnley's last year, we had a corner.  Westwood curled in in his usual style, the defender mysteriously let it past, and it crept in at the near post; the goalkeeper got a hand to it an inch too late, Ben Mee smashed it in from a foot.

Because the keeper was an inch too late, xG was 0.01.  If the keeper had been a fraction faster, or if the ref hadn't had his wrist buzzer goal line tech device, xG would have been 0.8 or 0.9 for the rebound.

Or another one - yer man has a shot from distance, and it's a blinder right in the top corner. xG 0.03.  Except the keeper made a stunning stop and tipped it on to the post, from where it rebounded to the forward who tapped it in.  xG 0.03 + 0.75, the extra 0.75 generated because (and only because) of the stunning save.

So like has been said, the club experts will have to study it in detail to get full value.

[Badger 2,405]

On 27/10/2022 at 13:26, repman said:

I thought I'd bring back the 5 game rolling average xG/xGA chart. Should be unsurprising to most that its looking worse by the week and after the Sheff U game our 5 game xG differential was in the negative for the first time this season. I also added the trendlines in green and red which don't paint the best picture either.

This is very interesting - did you construct it yourself or is it taken from a source that I am unaware of?

[Badger 2,405]

4 hours ago, hogesar said:

Thanks Robert!

Luck is such a huge factor and I think we / I discussed that earlier on in the thread.

I would argue that strictly speaking it's not luck but

1. Random probability. The problem is that a football season is not long enough to produce a perfectly "fair season." If you tossed a coin 10,000 times it would be pretty much a 50-50 split, but if you only toss it 4 times there is a far greater likelihood of it being 100% one way or a 25-75 split. We should demand 1000 game seasons! 😀

2. Different quality of finishing/ keeping and blocking etc.

[king canary 7,580]

44 minutes ago, Badger said:

1. Random probability. The problem is that a football season is not long enough to produce a perfectly "fair season." If you tossed a coin 10,000 times it would be pretty much a 50-50 split, but if you only toss it 4 times there is a far greater likelihood of it being 100% one way or a 25-75 split. We should demand 1000 game seasons! 😀

Baseball gets pretty close...

[littleyellowbirdie 2,511]

5 hours ago, hogesar said:

Thanks Robert!

Luck is such a huge factor and I think we / I discussed that earlier on in the thread.

If you're interested, this book is really worth a read:

https://www.amazon.co.uk/dp/1089883188/ref=sr_1_2?crid=3IY8FZB3FNOS6&keywords=the+expected+goals+philosophy&qid=1573369702&sprefix=The+Expected+goals%2Caps%2C145&sr=8-2

 

The better you play, the luckier you get. 

[It's Character Forming 1,160]

1 hour ago, dsr-burnley said:

One of the difficulties of xG in raw from is that big differences come from little variables.  For example, one of Burnley's last year, we had a corner.  Westwood curled in in his usual style, the defender mysteriously let it past, and it crept in at the near post; the goalkeeper got a hand to it an inch too late, Ben Mee smashed it in from a foot.

Because the keeper was an inch too late, xG was 0.01.  If the keeper had been a fraction faster, or if the ref hadn't had his wrist buzzer goal line tech device, xG would have been 0.8 or 0.9 for the rebound.

Or another one - yer man has a shot from distance, and it's a blinder right in the top corner. xG 0.03.  Except the keeper made a stunning stop and tipped it on to the post, from where it rebounded to the forward who tapped it in.  xG 0.03 + 0.75, the extra 0.75 generated because (and only because) of the stunning save.

So like has been said, the club experts will have to study it in detail to get full value.

I think your first example is actually what xG is trying to capture.  In real life football, sometimes that 1% chance comes off (assuming xG has correctly estimated the probability is that low).  So you can get skewed results from a goal coming from a very unlikely situation.  I suppose if a team were (hypothetically) to put in crosses at every conceivable opportunity, all those small xGs would add up ?  Do you miss the old pre-Kompany days ?

 

The second one is a good point, if you have someone take a shot with a very low xG chance and it doesn't score but sets up a very high % chance, it seems strange to get a very high xG for the 2nd chance.  I can't see a way around that.

 

Glad to see the fantastic level of intellectual debate on the pinkun (🤣🙂😲) is attracting even fans from other teams. 

[Robert N. LiM 4,243]

15 minutes ago, littleyellowbirdie said:

The better you play, the luckier you get. 

Or, the better you play, the less important luck is

[Robert N. LiM 4,243]

1 hour ago, dsr-burnley said:

One of the difficulties of xG in raw from is that big differences come from little variables.

 

1 hour ago, dsr-burnley said:

So like has been said, the club experts will have to study it in detail to get full value.

Yes, exactly. Really good post. Those examples you give are really helpful. Cheers