Blog post

See-Saws, Circles, and Narrative Fallacies

A minor word of caution on statistical inference and the stories it can tell
Norman Wright | February 3, 2014, 4am PST
Share Tweet LinkedIn Email Comments

I once tried to seduce a girl by telling her that if she went out with me, she’d have a lot of laughs. As evidence, I demonstrated that I was a funny guy. I made her laugh. Nonetheless, she didn’t go out with me.

Obviously, the reason she turned me down was because of the sample size. She didn’t have a large enough statistical sample of my humor to determine whether or not I was truly funny. One good joke wasn’t enough to derive a conclusion. And she didn’t want to test the theory any more. She had enough of me after one simple exposure.

But I still like to think I’m funny. And to prove it, I offer you the latest data-driven conclusion that I have developed from recent analysis inspired by Shane Phillips’ blog post on road expansion and per capita income. I’ve taken his same data on road growth and applied a few more bits of data to come up with something rather interesting.

First, road growth has many impacts on many things. Correlating road growth and 10-year growth in nationwide renter-occupied housing units delivers a 0.34 correlation coefficient, meaning that as roads expand, rental properties expand, too.

But to further prove Mr. Phillips’ original point, rental expansion shares a -0.25 correlation with per capita income growth, meaning as rentals expand, per capita income goes down.

Ah, but per capita income shares a 0.24 correlation with violent crime, meaning that as per capita income goes down, so does crime.

Therefore, road growth increases rental units which decrease per capita income which in turn decreases crime.

Therefore road growth lowers violent crime.

To some degree.

This begs the question: do you want to live in an America that keeps families safe from violent criminals? Do you want to lower the prevalence of murder, assault, and battery in your community? If you answered YES then I have one clear and obvious solution: build more roads.

Oh, but don’t build too many roads. If more roads lowers crime, there is another relationship that can be even worse: violent crime has a -0.22 correlation with obesity. That’s another negative relationship, meaning that as crime lowers, obesity rises. Lower the crime too much and we’ll make our country fat.

We want a safe America but not a fat one.



Narratives and Narrative Fallacies

I have weaved narratives into every single thing I’ve ever written. It’s inescapable. But what makes one narrative better than another? I don’t know. But I do know that narratives can be dangerous.

Let’s be clear on something: Shane Phillips is an smart, ethical person who stated many times throughout his blog post that he’s searching for truth; he didn’t say he had found any yet. He said he was searching and will continue to search. That’s awesome. And this article is not a critique of him or his work. He was very upfront.

But sometimes, readers see text and numbers and charts all compiled together and they can’t help but jump to a conclusion. This is where a narrative becomes narrative fallacy. Blogs and Powerpoints are notorious breeding grounds for this sort of thing.

As Mr. Phillips pointed out, correlation does not indicate causation. But I’d like to expand on something that runs even deeper: causation is a wildly overrated concept. Just because one variable can cause another variable to behave a certain way (which is very difficult to prove outside of physics), does not mean that it will have any true direct impact to the rest of the interconnected operating system that we call “Life”. Or worse, it can have an impact, but not nearly the impact that we usually foresee. This phenomena refers to the Law of Unintended Consequences. And it is so prevalent in society because we spend far too much time looking and thinking of see-saws instead of circles.

See-Saws of Inference

Here’s the see-saw: variable A goes down, causing variable B to go up. And vice versa.

We discover relationships like this all the time. But when we manipulate these simple see-saw relationships in an actual system, things get crazy. Suddenly, what was a see-saw of Variable A and Variable B becomes a circle, a continuum, of many more things.

Circles of Complexity

Consider roads and income. If road expansion really does cause incomes to drop, we could eliminate road expansions. This would cause incomes to rise. People would earn more money and be able to afford new lifestyles. One such lifestyle people may desire would be to live the country life. Which means living further from town. Which is great since they can afford it! But the lack of road expansion would lead to sprawl along existing corridors. Suddenly, it’s 2003 all over again. Without road expansion, there would be no way to install new streets along the corridors where people now want to live. McMansions would return and large lots would be all the rage but the extra travelers on those non-expanded road systems would result in congestion, delayed travel times, smog, and dry, itchy scalp.

Yes! Dry, itchy scalp! Oh the humanity!

Wait, wait, wait … I’m throwing all kinds of crazy assumptions into this, aren’t I?

Well, that's okay. I just described one scenario. In truth, there are many other scenarios—millions in fact—because we are not dealing with just two variables (road expansion and per capita income). We are actually dealing with massive, nation-wide systems where two shifting variables are really just small parts of an unimaginably complex machine.

So my assumptions are likely wrong. But so are yours. And if we argue over which of our scenarios might be better, we will create some seriously inane conversation that will look great on some newspaper's op-ed page. 

Good People Lead To Great Work: A Perfect Correlation

Believe me. I want Mr. Phillips’ linkage between roads and income to be true. I want such a link to convince policy makers to spend less on roads and spend more on neighborhood schools and public safety and libraries and parks. Maybe you feel the same. If so, let’s help Mr. Phillips to that end. I'm with you, Shane! You’re clearly a good person looking to make a difference and you will succeed. We should all be so lucky.

But when we readers examine his reports, let’s bear in mind that we cannot justifiably develop narratives without sound, absolute evidence. It takes much more than good intentions and a sub-0.50 correlation coefficient to make a point.

We must remain ever-mindful of the narrative fallacy and our tendency to seek patterns that reflect our preferences. I want the story expressed by Mr. Phillips’ data to be true. It’s a good, seductive story. Very seductive. But seduction isn’t enough for rational decision-making. It’s not enough for dating, either. I know from experience.

Sources for my data:

Income, 2000 - 2010

Road Growth, 2000 - 2010

Obesity Trend by State, 2000

Obesity Trend by State, 2010

Rental Occupancy Trend by State, 2000 - 2010

Violent Crime by State, 2000

Violent Crime by State, 2010

Share Tweet LinkedIn Email