I watched a city council meeting recently where they debated whether to remove the gender question from their annual community survey. Staff relayed that the research firm needed the question to ensure survey respondents match Census data – knowing the gender of each respondent should not be optional.

One council member advocated for removal, citing concerns about the safety of residents who identify as non-binary in the current political climate. Several colleagues agreed, with one asking bluntly: “Does it matter if men and women answer differently about the library? They’re still two people with opinions.”

If I had been there, I would have interjected, “yes, two very different people” – with different lived experiences.

The council voted 4-3 to remove the question.

This was good intentions producing bad results.

Government Managers Should Stand Firm

Removing demographic questions like gender from a community survey makes it impossible to achieve the accuracy that makes a statistically validity community survey worth conducting in the first place. Without demographic data, researchers cannot ensure that responses match the population – staff and policymakers are left with data that over-represents some groups and minimizes others. The unintended consequences can be serious – much more damaging than the slight chance of making a few people uncomfortable answering a question about themselves.

Key Facts to Keep in Mind

• Demographic questions are asked so researchers can ensure every group is proportionally represented.
• Without gender, age, and ethnicity, own/rent, etc. data, a community survey is about as good as posting a SurveyMonkey link on your website and pretending those who respond represent all of their neighbors.
• Men and women consistently differ on policy priorities including community development, infrastructure, public safety, taxes, and water quality. They also consume different media – important to understand for outreach purposes. Ignoring those differences is nonsensical.

Why Personal Characteristics Really Matter

When a research firm conducts a community survey, the goal is to deliver results that mirror the population. For example, if a city is 51% female and 49% male, but 58% of survey respondents are women, the data grossly over represents the female perspectives.

 

When Results Match Demographics	When Results Don’t Match Demographics
Respondents closely match gender, age, ethnicity, geography, etc.	Some groups are left behind and not properly represented
Enables cross-tabulations, to see key differences in concerns, knowledge, and aspirations – data can be used to make meaningful change	No ability to understand how communities differ, no chance of using the data to make meaningful change
Gives anyone reading the results confidence that they are statistically valid and can be relied upon for decision-making	Gives anyone reading the results pause that they are not statistically valid and should not be relied upon for decision-making

Do Men and Women Actually Differ on Local Government Services?

Yes!

The differences are consistent and profound. We have the data from 1,000+ surveys over decades – across hundreds of cities, counties, and special districts.

We know that women express higher concern about environmental quality and public health. Men express significantly higher concern about taxes and housing development. Here are some data points:

In a wealthy Northeast town, education is the top issue for 27% of women, but just 14% for men

In a Central Coast California beach town, 52% of men say homelessness is the top issue of concern, and it’s just 39% for women

In a tiny Alaskan town, men have a dimmer view of code enforcement, with 45% saying they are unsatisfied, but a majority of women (54%) are happy

In a suburban Michigan County, a majority of women cite mental health as being very important, for men it’s only 39%

In a rare community services district that maintains its own police force, just 14% of men use Nextdoor, but it’s more than double (30%) for women

In an Ohio countywide park district, 62% of women say they are familiar with nature education programming, for men it’s 47%

In a large suburb of Denver, Colorado, 61% of women use Facebook regularly, but the same is true for just 42% of men

In a wealthy and well-educated bedroom community near Atlanta, 49% of men say newspapers are their top source for news, that number is just 35% for women

The women living in a large suburban water district in New York are 50% more likely to express concern over contamination in drinking water

People’s opinions also differ by all other factors like ethnicity, what country they were born in, and how old they are. Failing to acknowledge these differences is short-sighted. You will absolutely get bad data if you ignore reality.

The fluctuations in results based upon demographics are the kind of insights that help an agency improve outreach and communication, allocate limited resources, and be more responsive to their community. Your residents are not a singular being. They are unique souls with totally different experiences in life. Giving everyone their own voice is how you respect those differences.

To the council members who worry about harming respondents, I explain that any research firm that works for local government – ETC Institute, Polco, Probolsky Research, or ZenCity keeps respondents anonymous.

Questions and Answers

Does removing one demographic question really ruin a survey?

It certainly can. It definitely weakens your ability to know if the data is accurate. Gender is one of the primary balancing variables alongside age, race, and location within the community. Removing it means there is no way of knowing if the final data is truly reflective of residents.

Can’t researchers just use other demographics to create proportions?

No. Your residents are not a singular being. People differ in how they look, talk, and believe. They have had different experiences. There is no way to make up for losing a central demographic like gender with other demographics.

Is there any risk to respondents from answering a gender question on a city survey?

No. Not if the agency is contracting with one of the four national research companies that do work for local government – ETC Institute, Polco, Probolsky Research, or ZenCity. Community surveys conducted by professional research firms are anonymous – no names are attached to responses.

If You Do Nothing Else

If you are a management analyst one week on the job, a department director, a public information officer, or a city manager with 20 years of tenure – if you are leading your agency’s effort to conduct a community survey with a research company, you should have the confidence that your community survey is methodologically sound. If you need back-up with policymakers, require a senior representative of the survey firm you hire to be present for a meeting to handle council questions.

About Dr. Adam Probolsky

Dr. Adam Probolsky is President of Probolsky Research where he has conducted over 1,000 polls and surveys for local governments across the U.S. Adam has also served in government roles at the city, county, and state levels where he made and oversaw policy related to finance, land use, parks, planning, public safety, transportation, and waste & recycling. He was also a sheriff’s department public information officer. He is a Senior Research Fellow for the Drucker School of Management at Claremont Graduate University.

About Probolsky Research

Probolsky Research conducts public opinion research for corporate, election, government, and nonprofit clients. The firm works for public agencies in 29 states from offices in Dallas, Denver, Newport Beach, Pasadena, San Francisco, and Washington DC.