The clerk reports it at the end of the hearing: forty-one comments opposed, nine in favor.
Everyone in the room hears the same thing. The community has spoken, four to one.
Except that isn’t what the number means, and treating it as though it were is one of the most consequential mistakes local officials make in these decisions. Not because the commenters are wrong. Because public comment is not a sample, and it has never been one.
The study that settled it
In Neighborhood Defenders, Katherine Levine Einstein, David Glick and Maxwell Palmer did something tedious and clever. They collected zoning meeting minutes across ninety-seven Massachusetts municipalities, identified every commenter by name, and matched them to the voter file.
That last step is what makes the work unusual. Most research on public participation asks who says they participate. This asked who actually stood at the microphone, then looked up who they were.
The gaps are not subtle:
- Age 50+: 75.0% of commenters, against 52.6% of voters in the same towns.
- Homeowners: 73.4% of commenters, against 45.6% of voters.
- Women: 43.3% of commenters, against 51.3% of voters.
- Opposition: across the proposals studied, 14.6% of comments supported the project and roughly 63% opposed it — about four to one.
Read that last number carefully. The four-to-one ratio isn’t a finding about any particular project. It’s the baseline. It’s what public comment looks like almost everywhere, on almost everything. Which means a four-to-one hearing tells you approximately nothing about your project specifically.
The finding I didn’t expect
The obvious explanation for the skew is access. Meetings happen on Tuesday nights. You need transportation, childcare, and three free hours. Naturally that filters toward older homeowners with time.
If that’s the mechanism, then removing the barriers should fix it.
In 2020 that experiment ran itself. Meetings moved to Zoom. No travel, no parking, no babysitter, attend from your kitchen while the kids do homework.
The gaps held or narrowed slightly. Homeowner over-representation went from 27.8 points to 25.3. The age gap moved from 22.4 points to 22.3.
Essentially nothing changed. Which means the bias is structural, not logistical. It’s driven by stakes and time, not access. People who own property near a proposed change have concentrated, immediate, financially legible interests. People who might benefit diffusely — a renter who’d like more housing, a resident who’d like a lower tax bill in six years — have no comparable reason to spend an evening on it.
That has an uncomfortable implication for how we usually respond to unrepresentative participation. More outreach does not fix a sampling problem. You cannot advertise your way out of a structural asymmetry in who has a reason to show up.
What comment actually measures
None of this means comment is worthless. It means it measures something different from what it gets reported as.
Public comment is an excellent measure of which arguments the organized will make, and how intensely. That is genuinely valuable information. It tells you where the fight will be, what the dominant frame is, which specific harms people are most afraid of, and how much energy sits behind each one.
It is a poor measure of what your community thinks. For that you would need something built to sample: a survey, a randomly recruited deliberative panel, or targeted outreach to populations you know are systematically absent — and even then you’d report it separately, with its own limitations attached.
So report both, labeled honestly. “We received forty-one comments in opposition, concentrated on noise and water” is true and useful. “The community opposes this four to one” is a claim your data cannot support, and if you make it from the dais, you have put your own credibility behind a number that will not survive contact with the next election.
The trap, in both directions
This finding is dangerous in the hands of both sides, which is exactly why a neutral should be the one holding it.
For a developer, it looks like permission. The opposition is unrepresentative, therefore we can disregard it. No. Volume being unrepresentative doesn’t make the substance wrong, and the organized minority at the microphone is precisely the group that files appeals, funds referenda, and recruits candidates against the officials who voted yes. Intensity is not opinion, but intensity is what determines whether your approval survives its first year.
For an official looking for cover, it looks like an excuse. The research says I can ignore the room. Also no. What the research says is that you should stop treating a hearing as a poll — not that you should stop treating it as a source of information about real harms that real people will experience.
What to do instead
Three things, none expensive.
Stop reporting comment as opinion. Change the language your staff uses in summaries. It takes one memo.
Find the chosen messengers. In every town there are people residents actually trust to tell them the truth — the extension agent, the volunteer fire chief, the retired engineer who reads everything. Ask three unconnected residents who they’d believe on this. You’ll get overlapping names quickly. Those people are worth more than any amount of outreach, and identifying them costs a week of coffees.
Design a second channel. Small-group sessions, a structured survey, a deliberative panel — anything that reaches people whose stake is real but diffuse. Then publish what it found alongside the comment record, without pretending either one is the whole picture.
Why this matters more than it sounds
Here’s the thing about a category error: it feels like rigor. Counting is what careful people do. Numbers look objective.
But a number that answers the wrong question is worse than no number, because it launders a guess into evidence. And when the decision later turns out badly — when a project that “the community opposed four to one” is approved anyway and nobody riots, or when a project everyone assumed had support gets ambushed — the people who reported the count are the ones who look like they weren’t listening.
Count the comments. Read every one. Just don’t call the total a measurement of anything but itself.