Sokal’s Children

Background: This Areo piece and the ensuing flurry of social media hot takes. See also the Sokal Affair.

1.The Biggest Problem with the Hoax is the Hoax 

A major strength of Alan Sokal’s paper for Social Text was that it didn’t present fake data. It was a nonsensical and error-ridden argument that would be just as nonsensical and error-ridden if it had been genuine. He wasn’t trying to fool the journal about anything other than his intentions, which shouldn’t be relevant to the review process anyway. Even a basic amount of common sense, like running the paper by a physics student, or saying “we don’t understand physics, submit somewhere else,” could have undid his plan. Thus he showed the editors either lacked sense or didn’t care about accuracy.

“I fooled you” actually wasn’t the damning part. We don’t expect editors and reviewers to investigate whether stuff is submitted in good faith. We expect them to investigate if its correct or incompetent.

A lot of the discourse around Sokal 2.0 focuses on the “I fooled you” part. But the damning thing shouldn’t be that you can succeed in misrepresenting yourself to editors, it should be they publish garbage that would still be garbage even if it were genuine and that could have been easily uncovered with a modicrum of effort. The hoax aspect of the hoax is overemphasized, both in design and in people’s reactions to it.

A major weakness of Sokal 2.0 is that some of the papers, like Dog Park Rape Culture, used fake data. The papers were still terrible,  and anyone who actually cared about accuracy would have asked more questions about the vague method and questionable data than the reviewers apparently did. Still, aside from being more ethically shady, fraudulent data distracts from the aims.

Even competent scholars can be fooled (though one has to tell better lies). That’s the point a lot of defenders of the hoaxed journals have zoomed in on. Which helps them ignore the other papers were reworked bits from Mein Kampf. Like Sokal’s paper, these are what they are, regardless of the author’s intentions. Ideological rant is still ideological rant, gobbledygook still gobbledygook.

2. The Biggest Problem with Peer Review is the Papers 

I skimmed through feminist Mein Kampf. It’s not as overtly Nazi-ish as one might think, and if you haven’t read Mein Kampf, which I haven’t, it’s not obvious it’s loosely based on it.

(“My Struggle with Whiteness,” written in the first person and talking about the insidious influence of whiteness in media, business, and scholarship, is much more obvious.)

Were I editor, I too likely would not have been able to tell the difference between Feminist Mein Kampf and a standard feminist paper advancing the need to unite in the struggle against the oppressor via [insert latest buzzword here].

But why send such a thing out for peer review in the first place? Why even submit it to a peer-reviewed scholarly journal?

I always thought the main purpose of peer review was to weed out errors — logical errors, methodological errors, errors of fact. How do you do that with a moral position?  How can a political call-to-arms be factually wrong? How do you review ideology?

You can make the peer review process as rigorous and onerous as you want, but if its dealing with political advocacy it’ll never get much past “I agree with this position!” being the standard. Which, judging from the reviewer comments, is the standard they used.

You’ll never “fix” peer review if people are applying it to ideological manifestos. It’ll be nothing more than an exercise in ideological purity carrying the patina of scientific authority. Which is probably the idea.

3. The Biggest Problem with the Journals is the Fields

To extent the hoax draws attention to a problem with the journals, it isn’t some error in the peer review process that keeps people from noticing that My Struggle with Whiteness was plagiarized from Hitler, or Dog Park Rape Culture had fake data. The problem is there are reviewers and editors out there who think such things are good and important work. And they’re not randomly distributed.

Several commentators have pointed out that we haven’t learned much from this stunt. Some nutty and sloppy papers got published in journals that we already know publish nutty and sloppy papers. That was the whole impetus: write something that mimics the crazy stuff getting posted on Real Peer Review. The hoax papers are no worse than the real ones.

No surprise that if you mimic the kind of stuff published in an outlet, you stand a good chance of getting published yourself. “Model your paper on what they’ve already published” is generally good publishing advice.

Also no surprise, really, that the sociology journals didn’t take the bait. Whatever their shortcomings, they don’t usually publish purely ideological rants or studies with N= myself. As several have pointed out, the sociology journals look pretty good by comparison. Though we knew that already too.

Taking the hoax as an investigation, it didn’t teach us much about the quality of the journals and their respective fields that we couldn’t learn from just reading the real stuff.

If we learned anything, it was from the reviewer comments. Before this, you might have convinced me that reviewers just didn’t engage with the crazier pieces, or that editors just shuffled them through to fill journal space. Which seems the case at Feminist Theory, which just told them to edit for length, but there were others where papers got serious  feedback and even praise. The comments generally raise my priors that many of these folks are just looney tunes.

4. The Moral Value of the Hoax

If there is any other value in the hoax, it is probably in drawing attention to the kind of goofy stuff that’s routinely published and upon which one can base at least a middling academic career as a tenured professor.

While the “I fooled you” part isn’t important for showing this, it seems to be the core attention-getting device (and also core source of criticism for the hoaxers). It has another function as well: it undermines attempts to defend nonsense with appeals to authority. “You uninitiated knaves just don’t get it because you lack our expertise in the theory and methods.” Expertise looks a lot less impressive when an outsider can easily fake it with some fashionable jargon and bell hooks quotations.  You can certainly fake your way into Critical Victim Studies a lot easier than faking your way into Physics.

I doubt the hoax was worthwhile, even as a form of social control. I don’t imagine the editors and reviewers are going to stop liking fashionable word-salad, N=me, everything-is-violence, and overtly ideological rants, or that anyone will inflict any serious obstacles to business as usual. There will be some momentary embarrassment, but what is that against tenure, travel money, professional status, and the ability to spread your politics to the young? Tain’t nothing at all.