Still "His Girl Friday"? (Nicole Underhay stars as Hildy Johnson in an adaptation of the classic film comedy last year in Ontario, Canada.) Photographer: Emily Cooper/Shaw Festival via Bloomberg
Still "His Girl Friday"? (Nicole Underhay stars as Hildy Johnson in an adaptation of the classic film comedy last year in Ontario, Canada.) Photographer: Emily Cooper/Shaw Festival via Bloomberg

Slate has an article up today with a provocative headline: “Most Female Journalists Have Been Threatened, Assaulted, or Harassed at Work. Here's Why We Don't Talk About It.” However, the evidence provided by the article doesn’t say any such thing. Amanda Hess reports the result of a Web survey performed by International Women’s Media Foundation and the International News Safety Institute. Says Hess:

Sixty-four percent of the 875 respondents said they had experienced “intimidation, threats, or abuse” in the office or in the field. Most of the abuse was perpetrated by the journalists’ bosses, superiors, and co-workers. Forty-six percent of female journalists said they had experienced sexual harassment at work, including “unwanted comments on dress and appearance.” That harassment was also overwhelmingly perpetrated by colleagues. Twenty-one percent said they had experienced physical violence -- including being pushed, pinned down, or threatened and assaulted with weapons -- in the course of their work. Thirteen percent had been sexually assaulted on the job -- again, mostly at the hands of co-workers.

If you’re a female journalist, these numbers are unsurprising. Pervasive sexual harassment and violence against female reporters, editors, and writers is rarely aired publicly, but it is an open secret in the field. The majority of incidents of sexual harassment and physical assault detailed in the IMWF survey were not reported to employers; 76 percent of women who met physical violence on the job did not report the assault to police. That’s partly because bosses and cops are the ones responsible for threatening and assaulting us. A small portion of abuse and intimidation reported in the study came from government officials (7 percent of reported incidents) and police officers (3 percent); 23 percent of women who said they had experienced physical violence on the job were assaulted by cops. The majority of these cases involved men we work with. When we do hear about harassment and assault against female journalists, it’s generally in the context of incidents perpetrated by people outside of the business (see: Lara Logan and Erin Andrews).

That doesn’t mean that female journalists are not forthcoming about the issue. We talk among ourselves, naming names in private email threads, drinks outings, and anonymous blogs. This is our “sad coping mechanism,” as Ann Friedman put it this year. Female journalists keep these discussions at a whisper because we know the men responsible are “too professionally powerful, too entrenched to really be held accountable for their behavior.”

Well, I’m a female journalist, and I found these numbers surprising. Don’t get me wrong: I believe that sexual harassment happens, because I’ve experienced it, though not in my years as a journalist. I certainly believe that many female journalists have been made uncomfortable by bosses and coworkers, subjected to unwanted propositions and so forth.

But these numbers, especially the ones on assault, are high enough to surprise this female journalist. And so I went looking at the survey.

The first thing to note is that the survey questions do not seem very well designed. They’re somewhat vague, and multiple offenses are lumped together. Here are the results on "the most commonly experienced types of intimidation threats or abuse”:

1. abuse of power or authority (185 / 22.5%);
2. verbal, written and/or physical intimidation including personal threats (173 / 21.04%) and
3. attempts or threats to damage reputation or honour (154 / 18.73%).
    Like anyone who’s had a bunch of jobs, I’ve witnessed a lot of petty abuses of power. Some of them even happened to me. But sticking me on a terrible job assignment to avenge an inadvertent personal slight, blaming subordinates for the boss’s bad decisions, or effectively stealing my rewards points from corporate travel don’t really belong in the same verbal basket as physical intimidation and personal threats. Nor is there any clue as to what constitutes an “abuse of power.” Are we talking “swore a blue streak at employees over a mistake” or “made the marketing analysts pass hors d’oeuvres at his daughter’s wedding”?

There’s an even bigger problem, however: The way the data were collected makes it hard to generate any useful data from it. I asked the woman who analyzed the data how the sample was created. Here’s her answer: “The sampling for the survey was a mixture of opportunity sampling (INSI and IWMF asking women in their networks to complete the survey) and snowball sampling (where women who had already completed the survey forwarded details to other women in their network to complete it).”

Now, I’m not saying that opportunity samples are completely useless; I’ve used them myself from time to time. But you have to be very careful about what they can tell you. If you’re looking for qualitative insights about an experience -- for people to tell you how they felt about getting divorced, or winning the lottery, or having kids -- then an opportunity sample can be very useful. Companies use opportunity samples all the time to ask people how they feel about their products or their websites, and they get valuable feedback.

But companies are well aware that they can’t generalize from those results to the whole population; a lot of people out there may love or hate your stuff, but they didn’t bother to respond to your survey. An opportunity sample is great for identifying things that people feel strongly about. But it cannot tell you how many people feel that way.

The problem with the INSI/IWMF survey is that the people who decided to respond probably aren’t very like the whole population of “female journalists.”

For one thing, they are connected to the folks at INSI/IWMF, and because social networks have a strong impact on one’s beliefs about various issues, this probably means that they care more about these issues than your average female writer. And because there is inevitably some disagreement about what crosses the line into “abuse” or “harassment,” friends of INSI/IWMF may be somewhat more likely to identify incidents that way than a random sample of female journalists would be.

But there’s an even bigger problem: Who answers which questions? This is an issue that repeatedly plagues online surveys. Say you want to know how many of your readers have kids with autism. If you put up a survey asking people about their special-needs kids, with maybe some follow-up questions about the challenges of dealing with the special-needs bureaucracy, what do you think will happen? Parents of special-needs kids will forward the survey to each other, tweet about it, discuss it over coffee. And when you get your results back, you’ll find out that 50 percent of your readership apparently has kids with special needs.

And indeed, at the end of her article, Hess says: “Some of the journalists who responded to the survey were assaulted by journalists with whom they did not directly work, but the news business is an erratic one. We could be working with them soon. And then we could be working beneath them.

“So we keep whispering. The survey is still live; female journalists can report their experiences anonymously here.”

Did I click through the survey to report that I have been working as a journalist for 10 years and have never once been pushed, shoved, pinned down, punched, kicked, torn, folded, mutilated or spindled? Of course not; I already have a very full day. But I bet that if I had been assaulted, I’d have clicked through to share my story.

All of which is to say that people who feel strongly about something, or have had something unusual happen to them, are much more likely to respond to your “opportunity survey” than, say, folks who work at Bloomberg with brilliant, courteous and respectful editors and a swell boss. That’s why professional opinion researchers discourage their use to represent the broad public experience.

This survey undoubtedly tells us one thing: Some female journalists have been assaulted and harassed, which is appalling. But contra the Slate headline, we have no idea how common this experience actually is.