SurveyFail redax: Downey adds to Lende
Posted by gregdowney on September 7, 2009
Daniel did a posting earlier today on Sex, Lies and IRB Tape: Netporn to SurveyFail that explores a research project that self-immolated through bad design, horrible conflict management, and a number of other character flaws. I’m really glad Daniel did this because he’s the more tech-savvy half of this duo. I just saw this yesterday and started to read up on the commentary but quickly realized that I was over my head, having pretty much exhausted my ability to navigate communication technology and resulting subcultural movements with a Twitter-related post a while back.
But I did want to add a couple of points because I’m particularly interested in research design and ethics and because I like kicking researchers when they’re down. No, no, just kidding — because I find the focus of ‘evolutionary’ theorists on the supposed ‘hard wiring’ of sexuality to be one of the more irritating and, well, hard-wired theoretical assumptions, even in the face of OVERWHELMING evidence to the malleability of human sexuality.
I apologize for not putting up some clever graphic, but I spent most of today helping friends build their mud-brick house and then went to a Showground Association meeting, where I was elected president (that’s kind of like the County Fairground in my town). My brain’s fried, but I don’t want to let this post sit for too long or it’s moment will have well and truly passed.
In my brief and incomplete survey of the discussions of this research, it became obvious that slash fans were particularly irritated, not just by the initial bad research design, but also by the seeming inability to apologize, learn from criticism or even simply back off on the part of the researchers.
This tenaciousness is interesting in a number of ways, not just that it layers on PR FAIL on top of initial Research Design FAIL. I serve on a Human Ethics Research Review board and have for a number of years, and it’s intriguing to see which researchers simply don’t get it, and then which ones also CAN’T get it when it’s pointed out to them. That is, even when their proposals get rejected or held up (which is surprisingly rarely), some researchers simply cannot think flexibly enough to come up with a new method for getting at something like the data that they initially expected. Moreover, the truly obdurate researchers often cannot see glaring ethical issues, even when they are pointed out and explained to them.
I often wonder if the ethics application process itself could be a sort of psychometric for testing what sort of ethnographic researcher a person might be (although I realize that these particular researchers were not ethnographers and never submitted a proposal to the ethics review panel or Institutional Review Board, IRB, at the university that they were ostensibly connected to, which I will not name as it does not deserve the guilt by implication). If a researcher can’t think of several ways to set up a survey or interview question to get at a particular bit of information, he or she should simply not be devising interviews.
In this particular case, for example, this subject population (the slash fans) seems extraordinarily capable of doing all sorts of things that a researcher would just love to have in any project: they’re great at doing online communication (oh man, this would uncomplicate some stages of data collection SO much), they’re opinionated, they’re eloquent, they seem to be self-reflexive and subject themselves to some pretty sophisticated introspection even without the researcher prodding with questions, they’re savvy enough to do things like create online identities (which makes the whole use of pseudonyms less problematic), some of them even seem to seek recognition so they might be interested in appearing in a book. Might. (Perhaps less likely after this whole debacle.)
In other words, this research should be a breeze compared to some of the projects that ethnographers do. For example, no offense to my rugby subjects, but YOU try getting long, in-depth, self-critical interviews out of some of them and then see how you feel about working with people who write — voluntarily — as a hobby!
So research should be a walk over. Done correctly, this kind of project, conducted by anyone with a modicum of genuine curiosity and sympathy for the people involved, should quickly be swimming in enough data to choke a hard drive. The only thing that could possibly go wrong… (suspense)… would be to steadfastly screw up, insult the people involved, not respond when someone points out several times that you were being insulting, refuse to apologize, still don’t ‘get’ it when people en masse start to come back at you, hold the line on the refusal to apologize, then imply you might do the project anyway, possibly by collecting data unethically from bulletin boards or chat rooms or whatever forums are being used…
In other words, you’d have to pretty monumentally stuff up and then tenaciously stay the course as everything around you screamed, ‘this is not working!’ Yes, I know that the bow has crunched into a reef, but if we just accelerate, we can push on through!
So, kudos to the researchers if they have managed to get any data that can possibly be used in a book because they may have burned an ideal subject population so that no follow-up research can be conducted without serious hurdles. What a great strategy: piss off your subjects so much so that no one can ever do a follow up study and show that you were completely wrong because of residual hostility.
But to get back to my original point, ethnographic research always requires a certain degree of inventiveness and flexibility. One organization turns out to be defunct or unapproachable, but you improvise in the field. A certain question turns off your respondents or sends them off on the wrong line of thinking, but you figure that out as you go and you stop asking it.
I know that these researchers are not ethnographers (thank God!), but these researchers, like the applicants to our ethics review board who can’t think of any other ethically-sound way to get the information they need, demonstrate a kind of inflexibility and lack of imagination which I think is anathema to pioneering research. It’s one thing to run an established and well-tested survey on a new population — even that can require some finesse — but to try to enter a fundamentally new area of research with a terribly blunt instrument shows another level of insensitivity (in addition to the ones many commentators have identified).
Having read through the commentary and bits and pieces of the original communications, I’m persuaded that the researchers were never really that serious about getting good data from this particular survey. Anyone with half a brain and basic knowledge of research methods knew from the get go that the data would be corrupted, inadequate to front the most basic peer review, inappropriate for the research ‘questions.’
Instead, the survey was a bit of a fishing trip, a chance to get some people to write some things on their survey that could get used in the book. Then, the authors could trumpet the sheer number of responses as a kind of proof (over 2000 people surveyed!), cherry-pick through the answers, and slap together a popular press book that wouldn’t hold up to any sort of serious scrutiny. But with a little right wing outrage (as Ogas anticipated), a book that argued ‘deep down, those kinky girls with their slash fantasies are just doing what their primitive brains want them to do’ would sell a stack of copies and get them on a few talk shows.
I write this as an author working on a ‘popular’ book (I put it in quotes because I would sort of have to sell some copies before it was technically ‘popular.’). I can’t really know what’s in the mind of the researchers, but I suspect it was not to do any sort of real research, but instead to suture together some researchy looking survey with some legitimate findings by other scientists, throw in plenty of titilating online erotica, and finish off with a big dollop of whipped evolutionary psychology to explain it all. Which brings me to my other issue…
Who is attracted to an evolutionary explanation for sexuality?
Sweet Jesus, can we officially amend the saying, ‘Patriotism is the last refuge of a scoundrel,’ to point out that renegade scientists often seek cover instead in evolutionary psychology? The red flag that goes up so often that it’s tattered around the edges and turning pink from sun bleaching is the assumption that human sexuality is a kind of drive rooted in some deep and primitive part of the brain.
Yes, I know, I teach evolution, so I’m aware that reproduction is necessary for our species survival… yup, got that. So please don’t write a response to this which treats me as if I haven’t spent WAAAAAY too much time reading heaps of evolutionary theory. I know the physiology of sexual arousal and am aware of the species preserving effects of heterosexual sex.
Of course, I also know that, with a few quick and dirty calculations, anyone can see that sex isn’t just about reproduction — average age of female sexual maturation in foraging groups (18), four year birth spacing in same groups, approximate chance of getting pregnant having sex with no birth control, child mortality rates and only slightly greater than replacement population growth rates over most of human history… A lot of human sexual contact is way over and above any sort of evolutionary imperative. We’re a heavy parental investment reproducer with heaps of sexuality unaccounted for in the absolutist account of evolutionary fitness.
But do the people making these arguments really even stop to think about what they’re saying half the time? The logical inconsistencies are pretty mind-blowing. I mean, every time I try to figure out what Ogas and Gaddam could have been arguing, I’m just struck by the most patently obvious fact: ‘Rule 34,’ that every type of imaginable porn already exists or will soon exist, seems to inherently undermine any argument about the ‘primitiveness’ or universality of human sexuality. That’s the very point of ‘Rule 34′ (and the implication, that porn variation is growing, would seem to necessarily include that human creativity was expanding the realm of visual erotica or porn).
By the way, Rule 34 is new to me, but I suspect that it has heaps of exceptions; most of the examples in the Urban Dictionary, for example, focused on food and cartoon characters, which actually seem to me to be particular areas of cultural elaboration for the sorts of ambivalent associations with which sexuality plays. Contrast the prevalence of these things with inanimate objects, for example, especially those of a character or scale that make anthropomorphism more difficult (the Urban Dictionary brings up Rule 34 itself and abstractions as counter-examples, also). But then again, human creativity is a magnificent beast…
If you like, I’m happy to be proven wrong about the Rule 34 suspicions, but we’re a family blog, so please do so with a modicum of restraint.
The very fact that so much of this sexual diversity is expressed through writing and reading, two activities that are both vicarious (in terms of physical interaction) and evolutionarily quite young, show precisely how flexible sexuality in humans can be. Dogs don’t dig porn. Nor do they read or write erotica. We breed horses at our farm and believe me, there’s no creativity or much flexibility involved (actually, it’s a pretty scary mix of chasing, kicking, biting, and a few moments of mounting followed by general disinterest all around).
In addition, slash writing always seems to focus on fictional characters: score one again for imaginative construction rather than innate compulsion. That is, writing erotica about imagined characters, better yet, someone else’s imagined characters, is gaining sexual stimulation or gratification from layers of imagination, creativity, and collective creation, your own and someone else’s, sometimes several someone elses. Some of the commenters make this point, but I think it bears repeating: slash literature is especially good for demonstrating three traits of human sexuality — innovation, flexibility (the ability to get satisfaction from a range of activities), and collective creativity (aka, culture).
Assumptions about the brain and sexuality
As Daniel’s original post points out, we’ve made hay here at Neuroanthropology.net with slack ‘evolutionary’ descriptions of human sexuality before, so much so that I hate to retrace some of these arguments. But the assumptions in this study seem to be particularly egregious by the standards of this literature.
However, I agree with some of the points that seem to be made in the argument for reasons opposite to the intentions of Olgas and Gaddam (to the best of my ability to figure out their intentions). Daniel quotes one of the pair’s communications with eruthros:
The structure and activity of our subcortical circuits are shaped by neurohormones such as testosterone, estrogen, oxytocin, progesterone, and vasopressin; these circuits function differently in men and women.
I agree whole-heartedly. These ‘circuits’ (c’mon though, get over the computer metaphor!) do function differently in men and women. And among men and among women. And in the same person over the course of a lifetime, perhaps even over the course of a day or a sexual encounter; ‘dude, that was so sexy… now, for God’s sake, stop it!’ And over time, the actions of some of these neurohormones can change the underlying functional links among different brain areas and subsequent rounds of productions of hormones.
Lots of researchers have pointed out that, if you really want to focus on differences, you can do so, but that there’s a huge number of similarities between men and women, even in terms of sexuality and sexual arousal. After all, it’s not like men (or women) were dropped here from a different planet (Mars or Venus) with some fundamentally different biochemical composition or radically alien anatomical structure.
Daniel highlights how, in their responses to some of their critics, Gaddam offers the blanket explanation that, ‘When we talk about the ‘oldest parts of the brain’ [the subcortical regions], it is in the context of the tectonic tussle between these and the prefrontal cortices that give rise to the peaks of our culture and the terrain of our behavior.’ Daniel points out that Gaddam describes an opposition in the brain between the ‘oldest’ pre-cultural, primitive elements and these newer cortices that produce culture; nature v. culture played out in brain layers.
I agree with Daniel’s critique, but I also would add that the example of slash fan fiction clearly demonstrates that, like so many other human brain functions, sexual desire stretches through these layers, triggering processes that link together ‘oldest’ and ‘newest’ parts of the brain. Like I said, this is erotica: written, visually-processed, imagined, arousing, sexually stimulating… a cascade of stimuli and effects (with plenty of loops and doubling backs) that combines different brain functions, spanning the neural equivalent of virtually the entirety of chordate evolution as it appears in human neural architecture. As Marc Hauser (2009) recently argued in Nature, it’s the human brain’s ‘promiscuous interface,’ the ability to link together functions and inputs from parts of the brain that are specialized and separated in other animals’ brains, that is one of the crucial characteristics of our species.
The ‘tectonic struggle’ is a very old model of the internal conflict between dangerous desire and self-restraint, id and superego, passion and reason, one that stretches back to at least Classical Greek thinkers, and hardly takes its starting point in neurological data. For example, the metaphor could easily have been lifted out of Freud’s Civilization and Its Discontents, albeit without the snazzy neurospeak that makes it seem so timely and up-to-date.
Last point: do we really need ‘discourse’ to get at this?
eruthros wrote a great response to the researchers when approached about the survey (see, I told you they’d be an amazing subject pool — if one of my interviewees gave me this sort of discussion in an email correspondence, I’d jump the next plane to see them, rush to their workplace, and hug them if I could see through the tears of joy). The irony is that I don’t fully agree with eruthros, although I do agree with the problems identified in the researchers’ proposal.
I don’t want to be churlish, but part of the problem with ‘research’ like that conducted by Ogas and Gaddam is that it tends to polarize the science/anti-science battle lines that proved such a brutal theatre in the Great Post-Structuralist Wars. As a scientist (of sorts — social scientist), I feel like we need to be surgical and precise in our attacks on Ogas and Gaddam because they do NOT represent the current, quite interesting moment that we have in the neurosciences, the opportunity to really forge a new kind of trans-disciplinary study of the brain.
Here’s eruthos’s discussion:
Your field [Yikes!], and others like it, rely on “biological adaptation” and the evolution of an efficient, adaptive brain to produce concepts of universality and universal maps of human behavior, unmediated or minimally mediated by cultural practice [okay, so it's hard to be surgical when you're really, really pissed]…
Biological adaptation means that evolution has shaped our brains, and that culture does not – no matter how much neurologists talk about experience continually re-shaping our brains. We are not denying biological differences and biological realities; we are asking why some biological differences become differences that matter. And it is fairly clear that they become defined as differences that matter as the result of cultural discourses – and that you are reinforcing those definitions. (See, for example, Butler on pregnancy and the definition of sex.) And these biological adaptations of sexual behavior, biological differences that you have defined as important, lend support to the “naturalness” of certain categories of sex practice. They mean that heterosexuality is normal and understandable and biologically necessary – after all, evolutionary success can only mean genetic success can only mean only having heterosexual intercourse in the missionary position. They mean that homosexuality and kink practices are not only deviant, queer practices in American culture, but that there are underlying biological reasons for the perception of homosexuality as deviant. They reinforce our position as objects of fascination; they reinforce our political status as secondary citizens; they reinforce violence and certain kinds of violent response against the sexually deviant (“gay panic”). When we are in the DSM, when we are objects of fascination, when we are biologically determined as deviant and queer and perverted, you have taken popular discourses of sexuality and made them concrete and real. And we want to make trouble in those discourses, to point out the problems and the flaws, to stand outside categorization and to make you work to fit us in. We’re not going to do that work for you.
Although I agree with the general anger and outrage of eruthros, I have lots of objections to this statement, such as PLEASE don’t think everyone who talks about or thinks about evolution engages in this kind of sloppy thinking — unfortunately, the ones who like to write newspaper columns that actually get printed (no, I’m not bitter) tend to be prime offenders. For example, I just lectured on brain evolution last week (ironically, I’ll do sexuality on Wednesday), and one of the things I point out is not just the inefficiency, but the quirks in brains and thinking that demonstrate how evolution works. This is a well worn strategy for demonstrating the relevance and evidence for evolution.
The thumbnail sketch eruthros draws of evolutionary discussions of homosexuality apply to some, but certainly not to all scientists in the field. The problem with Ogas and Gaddam is not that they are scientists; it’s that they are really BAD scientists (at least in respect to this project).
But some of eruthros’s points, while understandable in terms of outrage, misrepresent this battle anachronistically, assuming that the ‘enemy’ is an old familiar one called out by Judith Butler. The battle lines have changed, in my opinion, although eruthros is probably justified in attacking Ogas and Gaddam on this point because their research design was itself pretty anachronistic. In the past, the question was about how categorization and discourse fixed upon certain biological differences, making them salient over others, imbuing them with social meanings. Butler sort of won that fight, albeit with a bit of unnecessary hyperbole that may have gunked up the attempt to build a successor synthesis.
The new battle, to me, is not to stand up to biology while acknowledging that ‘We are not denying biological differences and biological realities,’ but to truly understand that the ‘biological realities’ themselves are not being accurately represented by people like Ogas and Gaddam. It’s not eruthros v. biology, but eruthros and good biologists v. bad biologists like Ogas and Gaddam. Cognitive science is shifting and large fragments of the field are calving away in directions that would place eruthros’s critique on solid neurological footing.
eruthros goes on to say, ‘We are operating in discourses with tremendous institutional and institutionalized power – your power, your medicalizing discourses, your determinations of what is natural and normal and what is deviant and unusual and fascinating.’ Collapsing of ‘science’ into a second person ‘discourse’ alongside various oppressive institutions — government, media, church, etc. — is a hallmark of some kinds of post-structuralist critique in cultural studies, but it is certainly not terribly helpful from within the sciences, trying to convert and remake them. This ‘discourses’ discourse, which is also very very theoretically strong in anthropology, can be corrosive of crucial distinctions for understanding how science might leverage social change. A discourse is like a stiff current; step in, and the assumption is that you become part of it, floating downstream toward certain powerful social conclusions. In fact, there are scientists trying to swim in different directions, although Ogas and Gaddam may be water-skiing downstream at high velocity.
This is not really a critique of eruthros, who’s writing from a very different position, feeling face-to-face with prejudice and exoticization targeted at the fanfiction community. I’m glad that slash fans have called out this research project: there seems to be no way it could have ever ‘succeeded’ by any scientific standard, but the community resistance may have scuttled the planned book. But as I prepare for my lecture on human sexuality, reproduction, child rearing and evolution this Wednesday, I appreciate the reminder of how important these topics are for affecting human prejudices and our sense of self and belonging.
Hauser, Mark D. 2009. The possibility of impossible cultures. Nature 460: 190-196. doi:10.1038/460190a (abstract)
Also see the other posts in the Slash and SurveyFail Series:
Sex, Lies and IRB Tape: Netporn to SurveyFail
“Ogas has already drawn the wrong conclusion from this reaction, ‘Personally, I’ve been the recipient of massive flaming on a larger scale than this…’ Ogi, one suggestion. If community created this ‘massive flaming,’ then community certainly played a role in why people get involved in fanfiction in the first place.”
Nature/Nurture: Slash to the Rescue
“Quite simply, nature vs. nurture is an oppressive division. Slash reworks the relationship between nature/nurture in ways that help us in our thinking and that are closer to the actual reality of how nature/nurture works.”