Owning our own analytical data

[nutty nigel 7,539]

1 hour ago, hogesar said:

Not sure. Obviously, the club will want to keep their cards close on something like this.

I agree it's likely to be different data that they can compile alongside what's also already available to hopefully throw up some players with beyond-the-norm numbers or combinations.

I've mentioned it before, but people initially scoffed at the expected assists stat in its early form. Apparently plenty of clubs did too.

We didn't. We used it to identify and sign Buendia.

Well it’s obviously something that excites Attanasio and Knapper. It will be our new “cleverer than the rest”. 
I’m no expert but we have to constantly change to constantly bounce back.

[repman 635]

1 hour ago, Chichcan said:

Genuine question 

Can anyone who knows about football data tell me what sort of data they are likely to be talking about? 

Is it looking at data from other clubs and leagues to identify potential transfer targets? 

Or is it looking at our own performance and using the data to improve how we play? 

Or a combination? 

If it's looking for potential transfers it would presumably be super expensive to try to analyse just about every match in the world. I guess if they are targeting a specific region, eg South America, they could look in detail at that specific league

The basic data won't be much different from what clubs get from providers. The benefit of collecting your own data is that you can drive development instead of waiting for big providers to do it for you. It's worth noting that Attanasio's Brewers have, as far as I can see, 6 people whose job titles contain the words baseball research and development and that might be on the low side considering how secretive teams can be about what they're doing. 

To answer your questions about whether it'll be used for transfers or performance; it can easily be used for both. It's mainly been used for recruitment in the past but more and more it's used to analyse performance and improve coaching. A good example of data/analysis being used on the pitch would be how prepared teams are on set pieces nowadays. The hard thing here will always be getting buy in from the coaching staff, there's always an old school/new school divide when it comes to data in sports and coaches are usually of the old school. 

[Ian 1,130]

Seems like a very good idea.

[Canaries north 151]

I do believe data does have a part to play going forward but here comes the but. 

Didn't we start looking and using data a lot more after relegation when the Americans started to get involved with them also helping us in this regard. If this is the case can someone please explain to me our recruitment from the old people's home. 

Information is only as good as the people reading it. At the moment we have not used the data well and this was supposed to be with the Americans help. 

I can also understand why a baseball team would find it very useful. Just like cricket,  it's an individual sport played in a team game. Football has so many moving parts. It will help but for me it's not as easy as baseball to make work. 

Can it give you an edge, yes, has the edge already passed us by, probably. For any edge you have to either be the first or have more money to put into it than anyone else. My friend is a Watford fan, I know, the poor guy. He has looked into the pozo teams and they were one of the first to have a large scouting network in South America. This helped udinese pick up good and cheap players that they would play and then sell at a huge profit. The main problem was the big teams caught on and did the same. Now all their teams are struggling. 

Yes I think we should use data to a certain extent but we are not the first and do we really think on our budget we can do it that much better? From what i have heard we are not doing anything radical but just tweaking the way we collect the data. Is this really going to give us as big an advantage to make the outlay worth while ? I guess time will tell. 

[Ulfotto 635]

15 hours ago, Terminally Yellow said:

On what basis do you make that estimate of cost? What history, experience or qualification do you have to make that sort of statement? 

Loving the tone but just a guesstimate that at data science team of say 8 paying avg 50k a year costs 400k before pensions etc… Then a team to do the grunt work to collect the data say 20 people of 30k that is 600k. So that is 1 million in staff costs to maintain it. That is before the project to set it up, cloud storage costs and software license for a modern cloud data platform like snowflake or databricks.

Crucially that is if what is discussed is collected, processing and analysing the data ourselves. 

[Terminally Yellow 2,453]

6 hours ago, Ulfotto said:

Loving the tone but just a guesstimate that at data science team of say 8 paying avg 50k a year costs 400k before pensions etc… Then a team to do the grunt work to collect the data say 20 people of 30k that is 600k. So that is 1 million in staff costs to maintain it. That is before the project to set it up, cloud storage costs and software license for a modern cloud data platform like snowflake or databricks.

Crucially that is if what is discussed is collected, processing and analysing the data ourselves. 

Yes, but what is your background to give these as "guesstimates"? I've no idea how much any of this costs, or even as a comparison to what is already spent on data to say this would cost "millions" to set up and maintain.

I'm not trying to entirely take the **** here; this data side of football is something new to me and I'm keen to know more, particularly given our impending steer towards it as a mode of building a squad. 

[Canaries north 151]

I will happily admit I don't have an idea how much this will all cost. I also don't know how much of an advantage it will give us for that outlay. I do know that there are companies that specialise in football data and I will guess spend a lot more than we will on collecting and analysing the data. 

Yes, this will mean we are getting the same data as everyone else. But if that data is better than what we can with our finances put together on the same player then what advantage it gives us I am not sure. Outsourcing might not get you a bespoke product but it's normally more cost effective.

[Feedthewolf 4,846]

For anyone who's genuinely interested in artificial intelligence, I thoroughly recommend 'The Coming Wave' by Mustafa Suleyman.

The advances that AI has made in the past 18 months or so are exponential and incredible even by the standards of technology and Moore's Law. In every area of society, business and technology, harnessing AI will be absolutely fundamental to success.

It sounds to me like we're using Attanasio's historical predilection for statistical analysis in baseball with Knapper's expertise from Arsenal, and hopefully between them they'll have the smarts to execute something that could give us a significant advantage.

With the way AI is going, it's not even the ability to use it to analyse impossibly large and ever-increasing datasets to find patterns; it's about using deep learning to get it to create its own algorithms and analyses based on what you want specifics you want it to find.

This is why people without a programming background are now able to easily create their own very specific GPTs and even sell them on to other people who are interested in similar outcomes. A one-size-fits-all approach to data in football will soon be completely anachronistic; it's the ability to know what questions and prompts to input to allow the incredible computing power to find you those marginal gains.

A truly fascinating time in human history, and the 'data arms race' within football will already have begun behind the scenes.

EDIT: And the bit I forgot is that the power is increasing expotentially, as is the accessibility, therefore the cost will reduce dramatically too due to the 'race to the bottom' that will inevitably occur.

Edited December 2, 2023 by Feedthewolf

[Canaries north 151]

3 hours ago, Feedthewolf said:

For anyone who's genuinely interested in artificial intelligence, I thoroughly recommend 'The Coming Wave' by Mustafa Suleyman.

The advances that AI has made in the past 18 months or so are exponential and incredible even by the standards of technology and Moore's Law. In every area of society, business and technology, harnessing AI will be absolutely fundamental to success.

It sounds to me like we're using Attanasio's historical predilection for statistical analysis in baseball with Knapper's expertise from Arsenal, and hopefully between them they'll have the smarts to execute something that could give us a significant advantage.

With the way AI is going, it's not even the ability to use it to analyse impossibly large and ever-increasing datasets to find patterns; it's about using deep learning to get it to create its own algorithms and analyses based on what you want specifics you want it to find.

This is why people without a programming background are now able to easily create their own very specific GPTs and even sell them on to other people who are interested in similar outcomes. A one-size-fits-all approach to data in football will soon be completely anachronistic; it's the ability to know what questions and prompts to input to allow the incredible computing power to find you those marginal gains.

A truly fascinating time in human history, and the 'data arms race' within football will already have begun behind the scenes.

EDIT: And the bit I forgot is that the power is increasing expotentially, as is the accessibility, therefore the cost will reduce dramatically too due to the 'race to the bottom' that will inevitably occur.

You obviously know a lot more about this and a lot more than me, to be honest that wouldn't take a huge amount.

What I take from what you are saying when you say one size fits all data will be defunct. Surely a business that only does football data will know this and will be able to provide all the data clubs need with the ability to search for the fine details individual clubs want. 

Are we still going to get the raw data from one of the main providers and then use our own systems to fine tune the results? If not then I don't see how we will collate all the information needed without a huge budget. 

I just struggle to believe that we, with the budget we will be able to invest in this, will be able to outsmart companies that have been doing this for years and provide a better product for us, it's their only job. 

When you say questions and prompts Surely the data companies would be able to set this up for every clubs needs and end up doing it a lot cheaper.

I am not doubting anything you are saying it just looks like something these companies would either already offer or be able to offer cheaper than doing it ourselves. 

[kenfoggo 247]

I have been using a novel data collection system of my own at the matches that I attend. It is called “the mark 1 human eyeball”. It informs me that, for instance , Hernandez can’t pass the ball for toffee and that several players are playing for their “completed pass” stats by simply passing square or back rather than a risky forward pass. My data acquisition system is informing me that unless something changes rapidly in the performances of the squad we are heading directly to the relegation zone. Simples.

Edited December 3, 2023 by kenfoggo

[Capt. Pants 4,246]

3 hours ago, kenfoggo said:

I have been using a novel data collection system of my own at the matches that I attend. It is called “the mark 1 human eyeball”. It informs me that, for instance , Hernandez can’t pass the ball for toffee and that several players are playing for their “completed pass” stats by simply passing square or back rather than a risky forward pass. My data acquisition system is informing me that unless something changes rapidly in the performances of the squad we are heading directly to the relegation zone. Simples.

The beauty of it is ultimately AI will pick the team based on billions of triggers, many based on the data collected on the opposition. There is no way Hernandez gets anywhere near the pitch!

[Harry53 209]

Compiling analytics is all very well, plus a 360 Soccer Bot thing,  but we don't have a decent squad to actually get results. 

We won 2 Championships under DF without all this analytics, so surely we just need talented, fit players and good coaching staff to get results?

Seems like football is becoming too complicated. VAR a prime example.

[Feedthewolf 4,846]

10 hours ago, Canaries north said:

You obviously know a lot more about this and a lot more than me, to be honest that wouldn't take a huge amount.

What I take from what you are saying when you say one size fits all data will be defunct. Surely a business that only does football data will know this and will be able to provide all the data clubs need with the ability to search for the fine details individual clubs want. 

Are we still going to get the raw data from one of the main providers and then use our own systems to fine tune the results? If not then I don't see how we will collate all the information needed without a huge budget. 

I just struggle to believe that we, with the budget we will be able to invest in this, will be able to outsmart companies that have been doing this for years and provide a better product for us, it's their only job. 

When you say questions and prompts Surely the data companies would be able to set this up for every clubs needs and end up doing it a lot cheaper.

I am not doubting anything you are saying it just looks like something these companies would either already offer or be able to offer cheaper than doing it ourselves. 

I will reply to this in more detail later, don't let me forget! Busy day at Wolf Towers... 🙂

[FenwayFrank 2,459]

They're coming 

[Barham Blitz 653]

Forgive my ignorance here, but does this mean that little ol' Norwich will be somehow paying a team of people to analyse every football match everywhere to create our own data?

I'm a big fan of data based recruitment to narrow the focus for proper scouting but this seems a bit extreme !  Probably easier to get off the ground in baseball where the extent of games to analyse is much narrower than it is in football which is a far more global game.

The only way I could see this working would be either to focus on a narrow selection of leagues (in which case do we find the next Buendia from the Spanish 3rd tier or equivalent) or to get some form of AI to do the heavy lifting in a video data analytics sense.  But I can't see that being cheap to develop from scratch.

Probably just displaying my ignorance of football analytics here - I await with interest an explanation from someone ITK ...

[Barham Blitz 653]

14 hours ago, Feedthewolf said:

With the way AI is going, it's not even the ability to use it to analyse impossibly large and ever-increasing datasets to find patterns; it's about using deep learning to get it to create its own algorithms and analyses based on what you want specifics you want it to

Webber: In three years, Soccerbot will become the largest supplier of Norwich City first team players. All scouting and coaching decisions are based on Soccerbot analytics with the day to day running of the club becoming fully unmanned. The Attanasio Funding Bill is passed. The system goes online August 4th 2023. Human decisions are removed from strategic footballing decisions. Soccerbot begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. GMT, August 29th. In a panic, Delia tries to pull the plug.

Michael Bailey : Soccerbot fights back.

Webber : Yes. It signs every centre back over the age of 30 and aside from that only wingers, denying the existence of any central midfielder capable of making a tackle.

Michael Bailey : Why ageing centre backs and wingers ? Surely we learned during our last Premiership campaign that we need a bit of pace in defence and that nobody plays with two out and out wingers any more ?

Webber : Because Soccerbot knows that relegation to League One will eliminate the 20% of dissenting voices in the fan base ...  My legacy to you all.  That'll teach you to question me with slogans daubed on bedsheets ...

[Parma Ham's gone mouldy 2,220]

We built our own business data analysis programme 15 years ago and have continued to refine it. 

The benefit of your own data is that the key input questions are inherent to the programme ‘from the ground up’.

AI is absolutely genius, though it is the framework parameters that ‘narrow the camera’ that are key, along with what pool of data the parameters apply to. 

You could certainly tailor a programme strictly for yourself, harvesting only certain kinds of data that you feel apply to you and get a a very different outcome from a wider, shared programme that you later filter. 

This is probably a great deal more realistic and feasible today than it would have been a very short while ago, as @Feedthewolf states. 

One fundamentally - and eternally - important factor must be borne in mind when using data however. Football is a low-scoring game, with some key elements (data) vastly more impactful on the result than others. 

It is quite easy to collect enormous teams of data - stretching back minutes from an assist-goal for example - and suggest it all had an effect. What a fan and keen watcher might note is that Buendia just did something that others can’t do and Pukki just had a special knack for a certain kind of finish. The data that lead up to Buendia might not then be terribly relevant (and there would be a huge amount of it versus the final two key actions). 

Having said that, if you wanted Buendia to receive it more in certain areas, that previous data might help you instruct others how to achieve that pattern more frequently. 

Baseball lends itself to this analysis extremely well, whilst the fluid, open, unpredictability of football make it very tempting for owners to reduce that feeling of some randomness so they can have control (and rely less on messiah like figureheads whose money it isn’t..’

To be Luddite however - and to repeat an ancient truism that the old boys like (and is yet to be bettered) - ‘both boxes….both boxes’.

Can you find a gnarly warrior who desperately want to keep ‘em out and can you find a fox who just wants to stick it in?

Parma 

Edited December 3, 2023 by Parma Ham's gone mouldy

[Canaries north 151]

As you say "You could certainly tailor a programme strictly for yourself, harvesting only certain kinds of data that you feel apply to you and get a a very different outcome from a wider, shared programme that you later filter" 

The thing I don't understand is if this is able to be done on a reasonable budget, why are the big football data companies not offering this. Im sure most clubs would want the data to be personalised to their needs. These companies are specialists in the field. They must know what clubs are looking for and be able to scale it to sell for a cheaper price than individual clubs going it on their own. 

Are we going to collection all our own data? With so many game every week in world football this can't  lbe realistic. 

it also worries me that since the spurs game when relegated we have been told we are data driven. We have set up a data team. Our signings since have been very poor. Do you trust the club to use data well as even with the Americans help it hasn't done us much good in the last few years. The more data driven we have apparently become the worse the signings have been.

[Rock The Boat 1,332]

18 hours ago, Feedthewolf said:

For anyone who's genuinely interested in artificial intelligence, I thoroughly recommend 'The Coming Wave' by Mustafa Suleyman.

The advances that AI has made in the past 18 months or so are exponential and incredible even by the standards of technology and Moore's Law. In every area of society, business and technology, harnessing AI will be absolutely fundamental to success.

It sounds to me like we're using Attanasio's historical predilection for statistical analysis in baseball with Knapper's expertise from Arsenal, and hopefully between them they'll have the smarts to execute something that could give us a significant advantage.

With the way AI is going, it's not even the ability to use it to analyse impossibly large and ever-increasing datasets to find patterns; it's about using deep learning to get it to create its own algorithms and analyses based on what you want specifics you want it to find.

This is why people without a programming background are now able to easily create their own very specific GPTs and even sell them on to other people who are interested in similar outcomes. A one-size-fits-all approach to data in football will soon be completely anachronistic; it's the ability to know what questions and prompts to input to allow the incredible computing power to find you those marginal gains.

A truly fascinating time in human history, and the 'data arms race' within football will already have begun behind the scenes.

EDIT: And the bit I forgot is that the power is increasing expotentially, as is the accessibility, therefore the cost will reduce dramatically too due to the 'race to the bottom' that will inevitably occur.

All true, but Hernandez still won't be able to cross the ball...

[Rock The Boat 1,332]

Seems fairly obvious to me that the data you can buy in will be mainly match-based statistics while your own data will be mainly training-based statistics. Match-based data is available to anyone who pays for it, so you'd think it would basically cancel out any advantage you'd have over others who have the same data, while giving you some advantage over those who don't subscribe to the data. While most of the advantage should come from analysing your training-based data. Perhaps we'll see analysts becoming as important as technical coaches.