One of the basic tenets of sports economics at this point is that there is zero evidence that whether a city plays host to sports teams has any measurable effect on its local economic health. That was the finding back in the 1990s when Joanna Cagan and I first started researching sports stadiums, and it’s continued to be the case in more recent studies — though as academic researchers have told me, it’s hard to get new studies approved when studying the economic impact of sports teams is like studying whether gravity makes things fall down.

So it was more than a little surprising when I was alerted (thanks, David!) to a new study out of Ball State University that claims that investing in luring and retaining pro sports teams has a long-term payoff for cities. According to the university’s press release, Census Bureau data show that if you look at certain long-term metrics, hosting a sports team “has a positive effect on the GDP and the population of the metropolitan area.”

There were some red flags, certainly — for starters, nobody on the study team appears to have any background in economics, rather being an assemblage of IT professors and business students — but still I figured this would be worth checking into. And then I got a look at the actual study itself, and I knew that I had to head straight here to share it with you, because this is one of the most hilariously incompetent pieces of academic work you could ever expect to see. Let’s skip straight to the “methodology” section, which is really where you want to start in any economic research paper:

We argue that not all economic benefits are measurable with typical metrics. Nor are they tangible in terms of their visible impacts on the appearance of the city and its residents. Rather, the main impact of investing in large-scale sports facilities and also hosting professional sports teams appears in the long term.

Okay, so not “typical metrics,” but rather something “in the long term.” What’s your actual data, guys?

In order to see the effect of hosting professional sports teams on the local economy, the correlation between the sum of the number of teams and the GDP of the Metropolitan area is calculated which is given below.

	GDP	SUM(Teams)
Linear Correlation Table	Data Set #1	Data Set #1
GDP	1.000	0.881
SUM(Teams)	0.881	1.000

Table1. Correlation between Team Numbers and GDP

And … that’s it. This entire paper is based on the observation that if you look at which cities had the most economic growth from 2001 to 2018, and which cities have the most sports teams, they’re the same cities.

But, you know, of course they are. The whole reason you choose put a sports team in a city is because it’s growing in relative size and economic activity — it’s why Rochester used to have an NBA team, but doesn’t anymore, and won’t anytime soon. Also, the last two decades have seen a historic rebound of the largest cities in particular, with well-off residents and companies alike recolonizing the metropolitan areas that they largely abandoned in the 1960s and ’70s. So if big cities are doing well over all, and big cities have the most sports teams, obviously there’s going to be a correlation there — it’s like observing that rich people tend to own the most yachts, and concluding that buying a yacht is a great way to get rich.

This is one of the most famous logical fallacies of all time, and is usually summed up as correlation does not imply causation. It has its own Wikipedia page, with some terrific examples, the best of which may be this one:

Example 1

Sleeping with one’s shoes on is strongly correlated with waking up with a headache.

Therefore, sleeping with one’s shoes on causes headache.

The above example commits the correlation-implies-causation fallacy, as it prematurely concludes that sleeping with one’s shoes on causes headache. A more plausible explanation is that both are caused by a third factor, in this case going to bed drunk, which thereby gives rise to a correlation. So the conclusion is false.

I reached out to Ball State’s PR spokesperson to see if the authors would be available to answer some questions about their work, but I haven’t heard back. In the meantime, I talked to College of the Holy Cross economics professor Victor Matheson, who agreed with me on the correlation vs. causation error, and added:

It is unfortunate that they didn’t understand the huge statistical error they were making here, but that’s what being a student is all about – learning by doing and recognizing mistakes so you don’t make them again in the future. What is concerning here, however, is that two Ball State professors also put their name on a paper that is clearly wrong and allowed it to be issued in such a way that the general public could read it despite its obvious flaws. Perhaps they were just trying to do something out of their area of expertise and didn’t understand why the analysis was wrong, but I
certainly wouldn’t put my name on a paper about computer technology or information systems unless I was very sure that I wasn’t doing something that would instantly make me look very foolish.

If there’s a silver lining here, it’s that at least this paper doesn’t seem to have resulted in a flurry of media coverage that grabs at the man-bites-dog nature of the headline without looking into whether the paper itself makes any damn sense. In fact, I thought twice about whether writing about this would be giving the paper more attention than it deserved — but it is still a fascinating case study in how bad ideas can take on a momentum of their own, eventually enlisting an entire public relations apparatus to put them out into the world and defend them. Thank goodness we have professional journalists out there to tell legitimate research from utter gibberish — oh wait…

Thanks to everyone who responded to my latest funding appeal by blowing past the 10-new-supporters goal in just a little over 24 hours! As promised, you’ve unlocked an unpublished article that fell victim to the assassination of Deadspin, on a subject that’s long deserved a deeper dive: the most bumbling sports stadium consultant of all time, and how they still manage to keep clients happy because in the stadium-lobbying game, it’s not about what the Acme Corporation puts in the box, it’s all about the packaging.

For Montreal baseball fans, last year’s World Series win by the Washington Nationals had to be aggravating: The club that took home its first title in its 50-year history, after all, spent 34 of those years plying its trade in French Canada, before it was rudely ripped from its home by an unholy alliance between a league seeking to threaten its players union with lost jobs and one of the most hated owners in pro sports.

As is the case in most cities left outside MLB looking in, Montreal has a group of local owners pressing for a Nouveaux-Expos, and last winter they tried to boost their chances with a study of their city’s viability as a big-league market. The resulting report did the trick nicely, placing Montreal 12th among the 27 existing baseball municipalities in TV market size, 15th in metro population, and 18th in median household income, comfortably in range as a feasible MLB expansion or relocation candidate.

Unfortunately, as the Montreal newspaper La Presse discovered when it asked some actual economists for comment, the study’s authors, Convention, Sports & Leisure, had committed une erreur méthodologique grave. They had neglected to adjust for U.S.-Canadian exchange rates, which when taken into account bumped Montreal all the way down to second-worst in MLB, ahead of only Cleveland. CSL replied that it had intentionally ignored exchange rates in order to measure local “purchasing power”; La Presse replied in turn that in that case, MLB should really expand to Japan, where the average income is 5.6 million a year — so long as you don’t bother with whether that’s in dollars or yen.

If you’ve even casually followed the world of sports stadiums and similar development projects, you’ve probably come across CSL’s name, likely under similarly embarrassing circumstances. CSL’s greatest hits include: releasing an economic impact study of a new D.C. United soccer stadium that massively overstated new revenues from the project; issuing a report on the impact of the San Diego Padres’ stadium that credited the new building with spending by attendees of an unrelated convention center; and releasing a paper on a possible MLS stadium in Louisville that admitted it would lose money for the public, but argued that if taxpayers won’t fund money-losing projects, who will? With this kind of track record, it’s all too appropriate that the “Learn more about CSL” link on the company’s website leads to a page reading: “Sorry. This video does not exist.”

Yet despite its track record of flubs, CSL continues to pick up high-profile accounts: It’s conducted studies of nearly every football and baseball stadium and basketball and hockey arena in the U.S. (Its financial analyses of a new baseball stadium in D.C. helped grease the skids for the Expos’ relocation as the Nationals, something that Montreal baseball backers apparently didn’t hold a grudge over.) The company has likely been aided in its efforts by some friends in high places: Though founded in 1988 by a pair of refugees from Coopers and Lybrand, another economic consulting firm, since 2011 the company has been owned by Legends Entertainment, the concessions-and-marketing behemoth launched by none other than the owners of the Dallas Cowboys and New York Yankees. It’s a cozy association that has led some to wonder if CSL’s reports should be branded with asterisk, especially when the company conducted a glowing report on the economic impact of the Los Angeles Angels at the same time Legends was bidding on the team’s concessions contract.

This is a problem for pretty much all consultants, actually—even those that aren’t wholly owned subsidiaries depend on sports team owners and their development partners to underwrite their fees. But CSL has taken the handwaving to another level. Heywood Sanders, a public administration professor at the University of Texas at San Antonio and author of Convention Center Follies, says that in the world of convention centers—an industry that like sports venues relies heavily on public cash and puffed-up claims of economic benefits—CSL “has a track record of overly optimistic and inaccurate forecasts.”

None of these clients appear to be bothered by CSL boasting a record of screwups so laughably oblivious that they seem designed by a hapless cartoon supervillain. (CSL itself didn’t dignify my request for an interview with a response.) Take the D.C. United soccer stadium, for example: When the Washington city council was preparing to vote on the project in 2014, it hired CSL to prepare a report to help make the $181.5 million public cost go down easier. According to the 406-page study, though D.C. would be committing to the largest MLS stadium subsidy in history, it would still turn a profit of up to $109.4 million thanks to the resulting new revenues.

These numbers turned out to be wrong—spectacularly so, when the consultants who issued the report were forced to walk it back just one week later. “We want to be clear that the $71.4 million in land exchange proceeds is not intended to convey net new benefits to the District,” wrote Convention, Sports & Leisure International in a letter to the D.C. council chair. This meant that nearly two-thirds of the projected benefits were, in fact, just the city selling land to help pay for the stadium; and if the rest of CSL’s projections were off by even a sliver, D.C. could end up with zero return on its investment, or worse.

Other errors, though, may become apparent only over time, and then only if you know where to look. Sanders is intimately familiar with CSL’s reports for convention centers, if only because they issue so damn many of them. “They do this all the time,” he says, noting that CSL issued studies advocating for convention center expansions in Seattle, Los Angeles, and San Diego within a few months of each other in 2014 and 2015.

Convention centers, like sports stadiums, typically receive public money on the grounds that bringing more out-of-towners will pay off by bringing fresh spending money to the local economy. And while there may be a better case for convention centers for that in the abstract—though hard numbers are hard to come by, the percentage of conventioneers who travel from other cities is at least likely higher than that of sports fans—building a new or expanded convention center only makes sense if it will result in expanded attendance.

That’s what CSL promised to Philadelphia and Washington, D.C. in a pair of studies in 2003. According to Sanders, the firm forecast that an expansion of Philadelphia’s convention center would boost annual hotel stays by 56%; for D.C., it projected that a brand-new center would result in a whopping 103% jump. Instead, after hundreds of millions of dollars in expense by each city, hotel stays went down, by 26% in Philly and 23% in D.C.

This isn’t unusual for the convention center industry, where an increasing number of cities are chasing a smaller and smaller number of conventions, with the unsurprising results that unless you’re Las Vegas or Orlando, bigger buildings are only likely to result in more unused space. Still, says Sanders, CSL continues to pump out reports showing positive projections for “almost every proposed convention center building project.”

Of course, sometimes the numbers won’t do what you want no matter how much you waterboard them, which can require more extraordinary measures. In 2016, CSL issued a “feasibility study” on a new $30–50 million stadium for the Louisville City F.C. soccer franchise that included some extremely dire economic projections:

In other words, by investing a mere $30–50 million in a new soccer stadium, Louisville could reap the benefits of … $2.7 million in new tax revenues. Spread over the next 20 years.

You might be tempted to think that this would be one scenario where CSL would finally dust off the “INFEASIBLE” stamp. But no, CSL helpfully explained, the sea of red ink that would accompany a Louisville stadium was a feature, not a bug:

The net income from operations will not be able to fund a material amount of stadium project costs, which is typical of most soccer-specific stadiums that have been built for teams in USL, NASL and other similar leagues. Historically, the development of soccer-specific stadiums has generally involved varying degrees of public-private partnerships.

And really, that’s all that studies like those issued by CSL are designed to do: not so much present a disinterested evaluation of a potential project as to provide cover for team owners and elected officials who want a 400-page rationale for what they were hoping to do in the first place. A friend of mine who once worked on economic impact assessments likes to compare them to the old Calvin and Hobbes strip where Calvin was assigned a science presentation about bats, and, disdaining anything that might be considered research, began with the premise that bats are giant bugs. When Hobbes objected that Calvin was just making up facts, Calvin replied that he had a “secret weapon” that would guarantee a good grade: a clear plastic binder to put his report in. Economic consulting reports are the clear plastic binders of the development-subsidy world.

But what failed for Calvin with his teacher has gone over way better with elected officials and much of the media that cover stadium deals. In the case of CSL’s overblown D.C. United study, for example, the Washington Post endorsed the project and cited the study’s figures, even as its own business reporter was reporting on its flaws. And while Washington’s city council balked briefly at the expense, they then went ahead and approved the spending anyway.

And that’s the true litmus test of the work done by the CSLs of the world: However bad they are at the job of producing valid economics numbers, they are very good at generating numbers on government checks. If Wile E. Coyote could only land the roadrunner in the end—or enable his billionaire clients to do so—nobody would be questioning his methods.