Owning our own analytical data

[hogesar 9,766]

A pretty big deal to become our own primary data collectors.

xG haters look away now...

Although of course, the data we collect would be far more tailored and relevant than that.

It would explain largely the reasonings around Knapper's appointment.

It's also a big step for a Championship Club. Courtesy of Bethnal:

 

[Ulfotto 644]

Yeah I don’t like the sound of that. Nearly all it software now is software as a service it seems a massive gamble to setup our own in house collection system for our own purposes.

There is a probably good reason why no one does this currently. It is probably likely to be expensive and the source data is unlikely to be any better.

I’m not anti data let’s buy it all in as other clubs due process into our own systems and databases. The recruitment team and the data team can use that to try and strike gold. I’m not sure collecting the data ourselves on any sort of scale will create a point of difference.

[Indy 3,323]

I’m for it! If it’s used by the right people and used to build the right team for the right system, you can’t use eight players aged 30+ to utilise a high press style! It’s ridiculous so I really hope Knapper can build a team in the analytical department to utilise it to set the right scouting, a lot like the A’s did back in the day for baseball, here’s hoping!

[Thirsty Lizard 3,183]

4 minutes ago, Ulfotto said:

Yeah I don’t like the sound of that. Nearly all it software now is software as a service it seems a massive gamble to setup our own in house collection system for our own purposes.

There is a probably good reason why no one does this currently. It is probably likely to be expensive and the source data is unlikely to be any better.

I’m not anti data let’s buy it all in as other clubs due process into our own systems and databases. The recruitment team and the data team can use that to try and strike gold. I’m not sure collecting the data ourselves on any sort of scale will create a point of difference.

I've told this story on here more than once. My brother used to work for Opta compiling their match statistics - watching videos of the full 90 minutes. He did a Man Utd v Fulham match. Man Utd broke away 2 on 1 -  a great chance to score. The two players were Andy Cole and Paul Scholes. Cole just had to play a simple 10 yard pass and Scholes would be right through one on one with the goalie. Instead Cole massively overhit the pass and Scholes had to chase the ball out to near the corner flag and a great chance was totally lost. My brother was almost weeping as he looked through the categories and had to log it as "Andy Cole - completed pass". (Opta had no category for "glorious opportunity totally fu**ed up"). 

Now, I hope that the way they compile data has improved since then, but any data is only as good as the people collating it and the system they have in place to analyse it. I strongly suspect that we think there is still an edge to be gained by doing it ourselves and doing it properly. 

[TheGunnShow 6,023]

2 minutes ago, Thirsty Lizard said:

I've told this story on here more than once. My brother used to work for Opta compiling their match statistics - watching videos of the full 90 minutes. He did a Man Utd v Fulham match. Man Utd broke away 2 on 1 -  a great chance to score. The two players were Andy Cole and Paul Scholes. Cole just had to play a simple 10 yard pass and Scholes would be right through one on one with the goalie. Instead Cole massively overhit the pass and Scholes had to chase the ball out to near the corner flag and a great chance was totally lost. My brother was almost weeping as he looked through the categories and had to log it as "Andy Cole - completed pass". (Opta had no category for "glorious opportunity totally fu**ed up"). 

Now, I hope that the way they compile data has improved since then, but any data is only as good as the people collating it and the system they have in place to analyse it. I strongly suspect that we think there is still an edge to be gained by doing it ourselves and doing it properly. 

I think you've just hit on what they aim to do. Come up with what's really needed to suit a certain style of football, then get compiling with only that in mind. In short, I think Knapper's going to build that system - possibly in cahoots with Wagner, possibly not.

[hogesar 9,766]

4 minutes ago, Thirsty Lizard said:

I've told this story on here more than once. My brother used to work for Opta compiling their match statistics - watching videos of the full 90 minutes. He did a Man Utd v Fulham match. Man Utd broke away 2 on 1 -  a great chance to score. The two players were Andy Cole and Paul Scholes. Cole just had to play a simple 10 yard pass and Scholes would be right through one on one with the goalie. Instead Cole massively overhit the pass and Scholes had to chase the ball out to near the corner flag and a great chance was totally lost. My brother was almost weeping as he looked through the categories and had to log it as "Andy Cole - completed pass". (Opta had no category for "glorious opportunity totally fu**ed up"). 

Now, I hope that the way they compile data has improved since then, but any data is only as good as the people collating it and the system they have in place to analyse it. I strongly suspect that we think there is still an edge to be gained by doing it ourselves and doing it properly. 

The good news is yes, the data collecting has changed considerably and the world of "expected assists" etc has helped in this regard.

[lake district canary 4,551]

"Becoming their own primary data collectors"

Sounds expensive to me.

Just now, hogesar said:

The good news is yes, the data collecting has changed considerably and the world of "expected assists" etc has helped in this regard.

So the data collected is much more detailed - and this has to be the way forwards.....expcept that like most things when you dissect them more and more intensely - the more you find out the further you get away from the whole.

The more science finds out about the universe, the more they see that it cannot be pinned down - the same is true of football, the more you try and pin it down, the more you take away from what it's supposed to be - a beautiful flowing game of skills and thrills.

Data may be useful up to a point - but too much reliance on data is always going to be a bad thing. It's about balance - and I do hope this "becoming their own data collectors" is not counter productive in the long term. Only so much is achieved with data - at the end of the day football is about people running around a field playing with a ball - and data is subservient to that - not the other way round.....

[Rolf Harris 31]

3 minutes ago, lake district canary said:

"Becoming their own primary data collectors"

Sounds expensive to me.

So the data collected is much more detailed - and this has to be the way forwards.....expcept that like most things when you dissect them more and more intensely - the more you find out the further you get away from the whole.

The more science finds out about the universe, the more they see that it cannot be pinned down - the same is true of football, the more you try and pin it down, the more you take away from what it's supposed to be - a beautiful flowing game of skills and thrills.

Data may be useful up to a point - but too much reliance on data is always going to be a bad thing. It's about balance - and I do hope this "becoming their own data collectors" is not counter productive in the long term. Only so much is achieved with data - at the end of the day football is about people running around a field playing with a ball - and data is subservient to that - not the other way round.....

Better to spend it on that, than tzolis. 

[Robert N. LiM 4,543]

14 minutes ago, TheGunnShow said:

I think you've just hit on what they aim to do. Come up with what's really needed to suit a certain style of football, then get compiling with only that in mind. In short, I think Knapper's going to build that system - possibly in cahoots with Wagner, possibly not.

That would be great, and is probably the only way of a club of our size being competitive in the current climate. The danger is that you need to pick the right style for five years' time, not for now. I don't have the expertise to judge, but I do have a suspicion that in the Webber years we were fighting last year's war. By the time our Pep-lite team got to the PL it got outmuscled by teams prioritising size and strength. Now we've switched to Klopp-lite high pressing (at least in theory!) just at a time when teams are beginning to abandon it...

[Creedence Clearwater Couto 1,297]

Brentford have been doing this for years, and clearly works for them.

I suspect there are big hurdles to overcome, given most clubs look to one or two specialist companies that dominate the market currently. 

[Ulfotto 644]

So the plan is that we can compile stats better than say the DAZN group who have revenues of over 2 billion pounds. Just setting this up etc.. will cost millions and running millions more. Yes get the data use the data as part of the decision making process but come on let’s be realistic.

And people think Soccerbot was a waste of money.

[hogesar 9,766]

4 minutes ago, Creedence Clearwater Couto said:

Brentford have been doing this for years, and clearly works for them.

I suspect there are big hurdles to overcome, given most clubs look to one or two specialist companies that dominate the market currently. 

Yes, the concern is if the club has really considered the financial implications. Thankfully with MA having gone down a similar path with Brewers you'd hope he's dived in eyes-open.

[norfolkngood 1,092]

14 minutes ago, hogesar said:

Yes, the concern is if the club has really considered the financial implications. Thankfully with MA having gone down a similar path with Brewers you'd hope he's dived in eyes-open.

Sounds fantastic we are looking at every area ,

But with this maybe a company that deals with 99% of the other teams can invest a lot more than just one club on its own to build the database etc ?

MA is no fool he stated he has a team over there looking after his interests in NCFC ,

 

[Wacky Waving Inflatable Arm Flailing Tube Man 3,865]

I certainly see why they're doing this. 

If all other clubs are getting their data from one of those two main providers (I'm guessing one is WyScout?) then everyone will end up with similar shortlists and it becomes a financial arms race. If we create our own, we can find gaps in the market. 

In Attanasio and Knapper, we clearly have two specialists in this area so we should be able to do this effectively. 

However, as a more general point, with the huge reliance on data nowadays, there'll surely be a point in a few years when traditional physical scouting will start to be what gains an edge. When teams started using data, with Brentford being the innovators, it gave them an advantage as nobody else was doing this and it allowed them to target players who the eye test may miss. With every club now using data, there'll probably be a time when the club(s) which revert to traditional scouting with have the edge as this may pick up things which the data might not.

[Canary Wundaboy 1,360]

28 minutes ago, Ulfotto said:

So the plan is that we can compile stats better than say the DAZN group who have revenues of over 2 billion pounds. Just setting this up etc.. will cost millions and running millions more. Yes get the data use the data as part of the decision making process but come on let’s be realistic.

And people think Soccerbot was a waste of money.

I'm with you here, I worry that the club feel they can do this better than established, mainstream providers who have massive advantages in terms of investment as well as technological and analytics expertise for whom this is their only job and product. This feels like the same sort of technological vanity project as the Soccerbot.

[repman 635]

41 minutes ago, Wacky Waving Inflatable Arm Flailing Tube Man said:

I certainly see why they're doing this. 

If all other clubs are getting their data from one of those two main providers (I'm guessing one is WyScout?) then everyone will end up with similar shortlists and it becomes a financial arms race. If we create our own, we can find gaps in the market. 

In Attanasio and Knapper, we clearly have two specialists in this area so we should be able to do this effectively. 

However, as a more general point, with the huge reliance on data nowadays, there'll surely be a point in a few years when traditional physical scouting will start to be what gains an edge. When teams started using data, with Brentford being the innovators, it gave them an advantage as nobody else was doing this and it allowed them to target players who the eye test may miss. With every club now using data, there'll probably be a time when the club(s) which revert to traditional scouting with have the edge as this may pick up things which the data might not.

I'm not sure how true this is though. I think all the clubs who use data providers will still have people inside the club creating models to understand what is best suited to their needs.

Statsbomb is probably the leader in the space right now and since starting up they've moved from a data consulting firm to becoming a provider themselves, and their platform allows for analysis but I would think clubs would go deeper. I wonder if us going this route of collecting our own data is us trying to get ahead in case the prices of these companies go up, or they even get bought out by a big club. Arsenal bought out a stats company well over 10 years ago but it wouldn't shock me if a club thought having all that information to themselves would be worth the investment long term. Even if not buying outright they could pay to have some sort of exclusivity I imagine.

Your point about physical scouting has some truth to it, but the way it will come about is a result of a new generation of scouts who are immersed in the data side and then also develop the eye for scouting. The big problem with a lot of traditional scouting is that it's based so much off opinion and biases.

[Ulfotto 644]

1 hour ago, Creedence Clearwater Couto said:

Brentford have been doing this for years, and clearly works for them.

I suspect there are big hurdles to overcome, given most clubs look to one or two specialist companies that dominate the market currently. 

No what Brentford do will be pay to top dollar to DAZN or Opta or potentially both. Load that data into their own system and run their own algorithms over the top of it. Which is what all clubs do some at a larger scale and priority than others.

What is seemingly being suggested is to collect, process and analyse the data ourselves rather than just analyse it.

Edited December 1, 2023 by Ulfotto

[ged in the onion bag 857]

We have seriously flawed staff on board at present based on the recruitment, the coaching and the football team but also running the club.   Surely that's the priority.

No point in having data if your staff are so ineffective that they can't translate it to good use. 

What analytics are they actually on about anyway.... is this to see patterns in ours and opponents play and adapt accordingly?   Is it to identify players and sign them or is it both?     

Can they use data to identify good scouts and employ a new recruitment team.     

Based on what's happened this past 3 seasons, the massive decline and finally last night, I am of the view that those in charge of our club are completely out of road now.    Nothing constructive to offer about how to address the problems, no mention of how we might address the decline and I am sure they couldn't explain exactly what 'analytics' will do to help us prosper. 

We are in freefall now.  No way those in charge have the capability to stop it. 

[Google Bot 3,329]

2 hours ago, lake district canary said:

"Becoming their own primary data collectors"

Sounds expensive to me.

They'll just get TNT, Sky sports and DAZN subscriptions for Soccerbot, flick the magical AI switch on his back, and he'll sit and watch every match possible and process all the player data for us.

[Ulfotto 644]

48 minutes ago, Google Bot said:

They'll just get TNT, Sky sports and DAZN subscriptions for Soccerbot, flick the magical AI switch on his back, and he'll sit and watch every match possible and process all the player data for us.

Right now I would have soccerbot playing centre back.

[RobJames 892]

Meaningless nonsense. There is nothing that cannot be seen by watching the game - other than how 'inaccurate' it is. 30 yards out and a screamer misses the bar by inches. Hitting row z from inside the six yard box, both recorded as misses. Tip tapping at the back clocks up umpteen 'completed passes', yet do not register that the keeper finally in desperation had to hoof it clear. Hwangs goal v Watford's registers the same as Idahs tap in v Cardiff. Does in matter if 12 or 15 passes went astray v Watford. Or is it those at the game could not see those missed passes ? The most obvious failing is the measurement of 'possession' - which all too often does not equate to how the game panned out.

However, we are told this mumbo jumbo helps with recruitment. So explain then, the number of failing European players we have signed in the past few years. Were those failed signings due to this ridiculous nonsense, or human error ?  If as I suspect it is the latter, then therein lies the problem and all this idiocy does is allow those (Webber) not up to the job to hide behind 'the 'stats'. Has Rowe made it to the first team because of his 'stats' or because those responsible judged him to be ready ?

We were told previously that having Soccerbot360 would give the squad an advantage. It would improve the passing rate of our players. No other English club had this. "It will give players the ability to recognise cues, to make quick decisions and execute key technical skills in a controlled environment.", but fell down due to the fact, that as much as you can control this gadget's output, you have NO control over the player's reaction, or ability to respond. .And I doubt if Hwang would have scored his cracker against Watford had he been programmed by this absurdity. It is that spontaneity, flash of genius/skill that makes watching football so enjoyable. I suspect that is in fact actually the cause of our inability to mark players and the increasing number of wayward passes.

 

[king canary 7,680]

42 minutes ago, RobJames said:

Meaningless nonsense. There is nothing that cannot be seen by watching the game - other than how 'inaccurate' it is. 30 yards out and a screamer misses the bar by inches. Hitting row z from inside the six yard box, both recorded as misses. Tip tapping at the back clocks up umpteen 'completed passes', yet do not register that the keeper finally in desperation had to hoof it clear. Hwangs goal v Watford's registers the same as Idahs tap in v Cardiff. Does in matter if 12 or 15 passes went astray v Watford. Or is it those at the game could not see those missed passes ? The most obvious failing is the measurement of 'possession' - which all too often does not equate to how the game panned out.

However, we are told this mumbo jumbo helps with recruitment. So explain then, the number of failing European players we have signed in the past few years. Were those failed signings due to this ridiculous nonsense, or human error ?  If as I suspect it is the latter, then therein lies the problem and all this idiocy does is allow those (Webber) not up to the job to hide behind 'the 'stats'. Has Rowe made it to the first team because of his 'stats' or because those responsible judged him to be ready ?

We were told previously that having Soccerbot360 would give the squad an advantage. It would improve the passing rate of our players. No other English club had this. "It will give players the ability to recognise cues, to make quick decisions and execute key technical skills in a controlled environment.", but fell down due to the fact, that as much as you can control this gadget's output, you have NO control over the player's reaction, or ability to respond. .And I doubt if Hwang would have scored his cracker against Watford had he been programmed by this absurdity. It is that spontaneity, flash of genius/skill that makes watching football so enjoyable. I suspect that is in fact actually the cause of our inability to mark players and the increasing number of wayward passes.

 

Tell me you have no idea how football data works without telling me you have no idea how football data works.

[Capt. Pants 4,297]

Feels a bit like reinventing the wheel to me. I'd rather they spent the time interpreting and analysing the data than creating some database from scratch. I was assuming we'd be following Arsenal's approach given that's what Knapper's expertise and learnings have been based on. Is this also their approach?

 

[Mello Yello 2,303]

Me? I just watch the game and form an opinion on what I witness and then look at where we are points wise in the league table....and that includes the goals for and against column....

 

[hogesar 9,766]

41 minutes ago, king canary said:

Tell me you have no idea how football data works without telling me you have no idea how football data works.

There's odd occasions where I really appreciate you on here. This is one of those occasions. Not least because now it won't be me having what will inevitably be a really illogical and annoying argument on the subject 😄 

[nutty nigel 7,646]

So I think the point of this would be to have different data. Not necessarily better data. The advantage being we would exclusively have it. Not sure how realistic keeping it exclusive would be? 

 

[hogesar 9,766]

31 minutes ago, nutty nigel said:

So I think the point of this would be to have different data. Not necessarily better data. The advantage being we would exclusively have it. Not sure how realistic keeping it exclusive would be? 

 

Not sure. Obviously, the club will want to keep their cards close on something like this.

I agree it's likely to be different data that they can compile alongside what's also already available to hopefully throw up some players with beyond-the-norm numbers or combinations.

I've mentioned it before, but people initially scoffed at the expected assists stat in its early form. Apparently plenty of clubs did too.

We didn't. We used it to identify and sign Buendia.

[BicesterCanary 5]

With the numbers involved in football this is a very astute move. The cost is nothing compared to the wages, transfer fees we are paying the players. Owning the whole process unlocks so much more, and ultimately gives us the edge. We can't do it like everyone else because we need all the edges we can get if we want to continue punching above our financial weight - which by this interview is the best we can hope for. 

 

This is actually huge it shouldn't be underestimated. The game already has and will continue to change to be data and machine learning focused as pretty much every single industry is doing. 

[Terminally Yellow 2,485]

6 hours ago, Ulfotto said:

So the plan is that we can compile stats better than say the DAZN group who have revenues of over 2 billion pounds. Just setting this up etc.. will cost millions and running millions more. Yes get the data use the data as part of the decision making process but come on let’s be realistic.

And people think Soccerbot was a waste of money.

On what basis do you make that estimate of cost? What history, experience or qualification do you have to make that sort of statement? 

[Chichcan 148]

Genuine question 

Can anyone who knows about football data tell me what sort of data they are likely to be talking about? 

Is it looking at data from other clubs and leagues to identify potential transfer targets? 

Or is it looking at our own performance and using the data to improve how we play? 

Or a combination? 

If it's looking for potential transfers it would presumably be super expensive to try to analyse just about every match in the world. I guess if they are targeting a specific region, eg South America, they could look in detail at that specific league