What 311 Calls (Cannot) Tell Us about Political Engagement
Ariel White (MIT), and Kris-Stella Trump (Office of Evaluation Sciences)
Let’s say that the last heavy rain left a pothole on your street significantly deeper, or that a streetlight went out right in front of your house.
What do you do? Some of us stick to airing a few choice curse words when the car bottoms out or when we can’t see where we dropped those keys… but some call “311”, the non-emergency service request line that now operates in many cities across the US.Is calling 311 a form of political participation, or perhaps an indicator of being a generally civically engaged person? It seems intuitively reasonable to think of 311 calls as a relatively cheap and generally accessible form of civil or political participation. In this article, we empirically examine this intuition. We compare 311 calls to other forms of civil and political participation (voting, making political donations, and returning the census). Based on our results, we give a few reasons for why researchers should proceed with caution when using 311 data to study political engagement.
Residents can call 311 lines to report potholes, ask about public benefits programs, point out hazardous construction sites, or even to report “squeegees”, “unsanitary pigeon conditions”, or “trans fats” (those are all actual call categories from the dataset of 311 calls placed in New York City). And many people make use of this service: New York City alone received an average of 4,500 calls per day during 2010-2014.
In several cities, the contents of the calls (but not details about the individuals who place the calls) are recorded in databases that are made available to the public. These datasets are rich in detail, listing the time, location, and content of the report; this in turn makes the datasets attractive to researchers who seek to understand public engagement with the authorities. But is 311 data a good proxy for measuring political engagement?
To figure out whether 311 data is a good resource for measuring politically motivated behaviors, we compare it to other forms of civil and political participation: returning the census form, turning out to vote, and making political donations. We work with 311 data from New York City in 2010-2014, and analyze the data at the precinct and census tract levels. We also recognize that 311 calls might vary in part because some neighborhoods have more problems to call the city about. In the paper, we show that controlling for some reasonable proxies for the prevalence of such problems (median household incomes and poverty rates) does not make a difference to the findings we discuss below.
When we compare neighborhoods’ 311 call rates to their voter turnout rates, we see very little correlation: neighborhoods that place a lot of 311 calls aren’t especially likely to have many voters. Similarly, having high 311 call rates doesn’t predict a neighborhood’s rate of mailing back census forms. This is surprising, as these are both fairly low-cost civic behaviors that might seem comparable to calling 311.
Plot comparing neighborhoods’ voting rates to their per-capita 311 call rates, showing almost no relationship. The three panels present three subsets of call data: all calls, calls focused on “public” (rather than persona/individual) issues, and calls narrowly focused on street maintenance concerns. The non-relationship holds across all subsets.
One behavior that is correlated with 311 calls? Political donations. Neighborhoods where more people give money to politicians are also likely to place more 311 calls. This starts to give us some insight into the nature of 311 calls. One way to interpret this relationship is to say that the act of placing a 311 call must be much more resource-consuming than one might think, and that this explains why it looks similar to making political donations.Another way to think about this finding is to observe that political donations share a key feature with 311 calls (but not with voting or census returns): they are not limited to one act of participation per person per time period. Perhaps the key similarity, then, is not that 311 calls are particularly expensive to make, but rather that only a small minority of residents have the motivation to place many 311 calls. We do not have the individual level data to definitively test this possibility, but we think it is a plausible explanation of these results.
One implication of our findings is that neighborhood rates of 311 calls might reflect the presence of a few really politically engaged individuals, rather than a broadly shared, community-level spirit of political participation. If that is the case, then researchers who are interested in studying community-level attitudes toward political participation should think carefully about whether 311 data is the best fit for their research question.Finally, we want to emphasize that our findings do not mean that 311 data should not be used for research. For example, we expect that this data can be very useful as a tool for studying how cities respond to citizen requests or how fast specific problems get reported to the authorities. Our note of caution applies specifically to the uses of 311 data as an indicator of broadly shared, community-level political engagement.
Ariel White is an assistant professor of political science at MIT. She studies political participation, race, and the criminal justice system. Her recent work uses large administrative datasets to explore people's everyday interactions with government.
Kris-Stella Trump is a research scientist at the Social and Behavioral Sciences Team in the federal government. She studies political attitudes and behaviors, especially attitudes toward redistribution and perceptions of the poor.