Amazing study claims spending public money on sports is good because big cities are big

One of the basic tenets of sports economics at this point is that there is zero evidence that whether a city plays host to sports teams has any measurable effect on its local economic health. That was the finding back in the 1990s when Joanna Cagan and I first started researching sports stadiums, and it’s continued to be the case in more recent studies — though as academic researchers have told me, it’s hard to get new studies approved when studying the economic impact of sports teams is like studying whether gravity makes things fall down.

So it was more than a little surprising when I was alerted (thanks, David!) to a new study out of Ball State University that claims that investing in luring and retaining pro sports teams has a long-term payoff for cities. According to the university’s press release, Census Bureau data show that if you look at certain long-term metrics, hosting a sports team “has a positive effect on the GDP and the population of the metropolitan area.”

There were some red flags, certainly — for starters, nobody on the study team appears to have any background in economics, rather being an assemblage of IT professors and business students — but still I figured this would be worth checking into. And then I got a look at the actual study itself, and I knew that I had to head straight here to share it with you, because this is one of the most hilariously incompetent pieces of academic work you could ever expect to see. Let’s skip straight to the “methodology” section, which is really where you want to start in any economic research paper:

We argue that not all economic benefits are measurable with typical metrics. Nor are they tangible in terms of their visible impacts on the appearance of the city and its residents. Rather, the main impact of investing in large-scale sports facilities and also hosting professional sports teams appears in the long term.

Okay, so not “typical metrics,” but rather something “in the long term.” What’s your actual data, guys?

In order to see the effect of hosting professional sports teams on the local economy, the correlation between the sum of the number of teams and the GDP of the Metropolitan area is calculated which is given below.

GDP SUM(Teams)
Linear Correlation Table Data Set #1 Data Set #1
GDP 1.000 0.881
SUM(Teams) 0.881 1.000

Table1. Correlation between Team Numbers and GDP

And … that’s it. This entire paper is based on the observation that if you look at which cities had the most economic growth from 2001 to 2018, and which cities have the most sports teams, they’re the same cities.

But, you know, of course they are. The whole reason you choose put a sports team in a city is because it’s growing in relative size and economic activity — it’s why Rochester used to have an NBA team, but doesn’t anymore, and won’t anytime soon. Also, the last two decades have seen a historic rebound of the largest cities in particular, with well-off residents and companies alike recolonizing the metropolitan areas that they largely abandoned in the 1960s and ’70s. So if big cities are doing well over all, and big cities have the most sports teams, obviously there’s going to be a correlation there — it’s like observing that rich people tend to own the most yachts, and concluding that buying a yacht is a great way to get rich.

This is one of the most famous logical fallacies of all time, and is usually summed up as correlation does not imply causation. It has its own Wikipedia page, with some terrific examples, the best of which may be this one:

Example 1

Sleeping with one’s shoes on is strongly correlated with waking up with a headache.

Therefore, sleeping with one’s shoes on causes headache.

The above example commits the correlation-implies-causation fallacy, as it prematurely concludes that sleeping with one’s shoes on causes headache. A more plausible explanation is that both are caused by a third factor, in this case going to bed drunk, which thereby gives rise to a correlation. So the conclusion is false.

I reached out to Ball State’s PR spokesperson to see if the authors would be available to answer some questions about their work, but I haven’t heard back. In the meantime, I talked to College of the Holy Cross economics professor Victor Matheson, who agreed with me on the correlation vs. causation error, and added:

It is unfortunate that they didn’t understand the huge statistical error they were making here, but that’s what being a student is all about – learning by doing and recognizing mistakes so you don’t make them again in the future. What is concerning here, however, is that two Ball State professors also put their name on a paper that is clearly wrong and allowed it to be issued in such a way that the general public could read it despite its obvious flaws. Perhaps they were just trying to do something out of their area of expertise and didn’t understand why the analysis was wrong, but I
certainly wouldn’t put my name on a paper about computer technology or information systems unless I was very sure that I wasn’t doing something that would instantly make me look very foolish.

If there’s a silver lining here, it’s that at least this paper doesn’t seem to have resulted in a flurry of media coverage that grabs at the man-bites-dog nature of the headline without looking into whether the paper itself makes any damn sense. In fact, I thought twice about whether writing about this would be giving the paper more attention than it deserved — but it is still a fascinating case study in how bad ideas can take on a momentum of their own, eventually enlisting an entire public relations apparatus to put them out into the world and defend them. Thank goodness we have professional journalists out there to tell legitimate research from utter gibberish — oh wait

Share this post:

8 comments on “Amazing study claims spending public money on sports is good because big cities are big

  1. I am officially using one of my limited trump cards to preemptively remove Ball State from my rising Senior’s college application list.

    Just awful work.

    PS: Thanks Neil for the links to the COVID charts in the last blog post. Data on positive infections is probably the most telling stat.

    1. None of the Covid rate stats are perfect: New cases is too dependent on testing rates, hospitalization and death rates lag way behind (and can be undercounted if people get sick and/or die at home), and positivity rates can depend on testing rates too — if you suddenly start testing more asymptomatic people, you’re going to see the recorded positivity rate go down even if the actual positivity rate is unchanged or rising.

      If all of those are going up at once, though — which is largely what we’re seeing in the Covid Belt — then you got trouble with a capital T.

    2. The funny thing to me is that these authors are professors and students in computer science and analytics, but are doing a paper on economic impact. Meanwhile, Ball State has a “Center for Business and Economic Research” that is designed and staffed by actual economists to do this sort of work. (And their researchers have gone on record before against stadium subsidies in Indianapolis, so there’s that.)

  2. Possibly one of these students is the recipient of the David Letterman “C” student full ride scholarship?

    1. Letterman’s scholarship is for telecommunications students, so not likely. Though I guess they wouldn’t be any more unqualified to write about this than business majors:

      https://www.bsu.edu/web/letterman/scholarship

  3. This reminds me of 1990s era NFL pregame shows… I’m sure they all did it, but I specifically remember Dierdorf doing his pregame checklist of which team had the advantage on offense, defense, special teams… and intangibles. They actually rated one team higher than the other based on intangibles.

    Nice to see academic work finally ‘catching up’…

  4. When you hear the word “study” you think that its been published somewhere, and that the study was peer reviewed prior to publication. But of course that’s not necessarily the case. Anyone can produce a study about anything. It doesn’t have to be peer reviewed, and it doesn’t have to be published anywhere.

    The really surprising thing about this paper is that two professors put their names on it. I would like to know why they did that. But we’re not going to get an answer from the Ball State PR department. Someone should contact those two professors directly. Start with a simple question: did you even read the paper you just put your name on?

Comments are closed.