01 July 2010

How Not to Track Intellectual Trends

Anyone who knows me, and certainly anyone who follows this blog at all closely, will know that I am an enthusiastic advocate of anything that undermines the misrule of "Theory" in the humanities and social sciences.

Here uppercase "Theory" stands for the vast array of assertions made in these fields without any systematic or critical attempt to support the assertions with evidence, and generally without any attempt to examine what might disconfirm them.

"Theory" of this sort is intellectually largely vacuous.

This is not at all the same thing as saying that any individual instance of an assertion made under this intellectual regime is wrong; it's just that the presentation of the assertion is insufficient—often radically insufficient—to establish it. Nor is this the same as saying that the questions broached by "Theory" are uninteresting or useless. It is just that the methods employed are insufficient to answer them

To me, this is the crucial difference between "science" and other types of intellectual endeavor ("not science"): it is the difference between how assertions are made, tested, and supported or disconfirmed.  Science is not a particular set of subject matters, but rather a way of thinking (and it is, by no means, a way of thinking that is always adhered to in what are commonly called the "sciences").

And thus science is a way of thinking that is just as applicable to the topics generally considered the province of the humanities as it is of those traditionally considered the province of the sciences.

So I'm generally happy to see any evidence of the decline of vacuous Theory.

And thus  I'm disappointed to report on two blog posts that purport to show such a decline, and don't, at least not reliably.

On 30 June, Neuroskeptic (a blog I subscribe to and generally enjoy) published a post entitled "The Fall of Freud". The author used a bash script to investigate the occurrence of various terms in the PubMed database. Neuroskeptic summarizes the results in the following graph:


Neuroskeptic summarizes the trends in these graphs as follows:
As you can see, the number of published scientific papers related to Freud-y search terms like psychoanalytic has flat-lined for the past 50 years. That represents a serious collapse of influence, given the enormous expansion in the amount of research being published over this time.

Since 1960 the number of papers on schizophrenia has risen by a factor of 10 and anxiety by a factor of 80 (sic). The peak of Freud's fame was 1968, when almost as many papers referenced psychoanalytic (721) as did schizophrenia (989), and it was more than half as popular as antidepressants (1372). Today it's just 10% of either. Proportionally speaking, psychoanalysis has gone out with a whimper, though not a bang.
Yet it seems obvious that this is not at all what the graph shows (or at least, this description is misleading). If the term "psychoanalytic" has "flat-lined," then its use in the articles tracked here has not decreased.  Nor does the graph necessarily show a "serious collapse of influence."  What it may show is that the rate of occurrence of the term "psychoanalytic" in the literature that is tracked by PubMed has remained stable. It seems to be true that the proportion of occurrences of this term relative to the tremendous increase in the number of words in all articles on all topics tracked by PubMed has decreased markedly, whereas the proportion of various terms related to other types of psychological research, such as "anxiety" or "antidepressants," has increased very rapidly.  It seems very likely that much of this increase is accounted for by journals that did not exist 50 years ago or (in many cases) even 10 years ago. Most of these new journals will probably have an explicit "medical model" approach to psychological research, and it is highly unlikely that anyone writing an article using the term "psychoanalytic" is going to submit to such a journal.

But surely the graph shows at least a relative decline in the influence of Freudian ideas?  Well, that depends on what what one means by "influence."  Certainly within the realm of the kinds of journals indexed by PubMed, the relative influence (judged by the ratio of psychoanalytic terms to the number of words published and indexed) has decreased. But this doesn't really address the question fully:  for what about the many journals (for example, journals in the humanities) that are not tracked by PubMed?

Thinking about this question led me to investigate the possibility of doing a similar sort of lexical analysis of the appearance of psychoanalytic terms in the humanities and social sciences.  So far as I can see, though, there is no publicly available index like PubMed for these fields.  (Please let me know if I've missed something.)  The Arts and Humanities Citation Index is a commercial product of Thomson Reuters, and is not (so far as I can see) subscribed to by either of the public library systems for which I have cards (the Boston Public Library and the Minuteman system). I do have access to JSTOR, but because it generally lags behind the date of publication by 3 to 5 years, it isn't suitable for doing this kind of analysis.  So far as I have been able to determine, no electronic resource available to me through BPL or Minuteman will do the job.

Neuroskeptic points to what is described as a "classic" post at Gene Expression (gnxp), "Graphs on the death of Marxism, postmodernism, and other stupid academic fads," from 22 September 2008 (the post was contributed by user "agnostic"). This post uses JSTOR (with an appropriate caveat on the time lag) to investigate the rates of occurrence over time in that body of articles of the terms "social construction," "psychoanalysis or psychoanalytic," "postmodern or postmodernism," "postcolonialism," "narratolog*," "markist or marxism," "hegemony," "feminist or feminism," and "deconstruction*."  (The spans of time covered by the graphs vary depending on the term; all graphs end with 2005.)

These graphs do seem to show declines in the use of all these terms, but the declines are mostly modest, with the exception of "psychoanalytic or psychoanalytic" (the latter term is misspelled in the graph heading, so it's possible that the search results are inaccurate in any case), "deconstruction," and "marxist or marxism," which perhaps unsurprisingly show a marked fall off after the mid 1980s (and one wouldn't be surprised to see a resurgence after 2008). Again, the analysis by the blogger fails to take into account the extreme expansion of scholarly publishing overall, which certainly accounts, at least in part, for the steep increase in the use of all of these terms beginning in the period between around 1970 and 1980. And again, the blogger fails to address the question of how representative the JSTOR sample is of scholarly publishing as a whole. (JSTOR is by no means comprehensive, and is rather weak on non-English-language publications.)

Thus, while one can perhaps sympathize to some extent with the jubilant triumphalism of the blogger's conclusions ("We are living in very exciting times -- at long last, we've broken the stranglehold that a variety of silly Blank Slate theories have held on the arts, humanities, and social sciences."), it seems clear that the methodology here is flawed, and that any such triumphalism would be premature.

One of the comments to the Neuroskeptic post pointed to GoPubMed, a fascinating and potentially very useful (if still somewhat rough-and-ready) site that allows sophisticated searching and analysis of the PubMed database.  I am currently using it for research into the history of the "Mozart Effect," and I hope to report on this topic here by this weekend.
Bookmark and Share

2 comments:

  1. It occurred to me after I finished this post that it may well be that the apparent declines in many of the graphs from the 2008 post on Gene Expression may be due to the staggered rate at which journals release articles to JSTOR. In other words, if the graphs all end in 2005, but some journals allow incorporation into and indexing by JSTOR after three years, but others only after four or five, then the statistics for 2005 and 2004 will not take into account all of the journals that were indexed for 2003. Any statistical analysis would need, at the very least, to take the possibility of this effect into account. At worst, it might actually account for most of the alleged "decline."

    ReplyDelete
  2. Thanks for the link and the interesting comments.

    My view on this is that the PubMed hits do show a definite decline in the influence of Freudian ideas within medicine. As you point out, what's probably happened is that there are lots of new journals nowadays with an explicitly biological approach; although I don't think this can account for all of the difference as if you look at, say, the American Journal of Psychiatry or the British Journal of Psychiatry, 30 years ago they publishing Freudian articles which they would never print today.

    But the fact that there are so many biological journals nowadays means that psychiatry has become more biological. 50 years ago, the idea of a purely biological psychiatry journal would have seemed bizarre to most people, because "everyone knew" that Freudian concepts like unconscious conflicts and family dynamics were important.

    Nowadays, while some people still write about those things, most people don't even bother to dismiss them - they're just off the radar for biological psychiatrists. Which is a decline in influence.

    I agree that outside of PubMed the picture may be different, but I think the fact that medicine used to talk about Freud a lot and now never mentions him most of the time, is in itself very striking.

    On a technical note, GoPubMed does look very interesting but I'm going to keep using my bash script as the latest version (here) allows you to simply input a list of terms, wait 10 minutes, and copy the output straight into a spreadsheet... GoPubMed is clearly more powerful, but for my limited purposes i.e. making graphs for blog posts, I think my script is still more convenient.

    ReplyDelete