The "Experimental Philosophy Meets Conceptual Analysis" last week was a lot of fun -- one of the most stimulating conferences I've been to for some time. I've posted photos, and the Powerpoint for my wrap-up talk "X-Phi Meets A-Phi" (some of which is summarized below). Here the A stands for "armchair" or "a priori", as you please. See the experimental philosophy page and the experimental philosophy blog for some background on the issues, and see also Alex Plakias's conference recap on the Go Grue blog.
The conference had something of a tag-team wrestling format, alternating X-Phi speakers (in the 'red corner") with A-Phi speakers (in the blue corner). The X-Phi speakers were Steve Stich (with a nice overview of his work on disagreement over the intuitions that analytic philosophers often appeal to, in epistemology, the philosophy of language, and ethics), Josh Knobe (who outlined experimental work on people's intuitions about consciousness, suggesting that they're willing to ascribe nonphenomenal states much more freely than phenomenal states), John Doris (who used empirical work on the role of social processes in moral thinking to support a socially-extended view of cognition), Alex Plakias (on empirical work on moral disagreement) and Adina Roskies (on the implications of acquired sociopathy for moral internalism). The A-Phi speakers were Frank Jackson (on conceptual analysis as a kind of experimental philosophy), Michael Smith (on pure and applied conceptual analysis, responding to various aspects of the X-Phi critique), Farid Masrour (on the relevance of the distinction between prima facie and ideal intuitions), Jeanette Kennett (who responded to Adina on empirical arguments against internalism), and, I suppose, me.
In the end there was a lot more agreement than disagreement, though there were certainly some contentious issues along the way. Given the emphasis on conceptual analysis, it's not surprising that different concepts of experimental philosophy were distinguished. For a start, one needs to distinguish experimental philosophy from empirical philosophy simpliciter, where the key distinction is the focus on data about philosophically relevant intuitions and judgments. Farid also usefully distinguished "positive" from "negative" experimental philosophy. The former, typified by Josh Knobe's work on intentional action, tries to find interesting patterns in people's application of concepts to cases, drawing out conclusions about the way those concepts work. The latter, typified by the work of Steve Stich and colleagues on Getter and Kripke intuitions, tries to find cross-group or cross-cultural differences in philosophically relevant intuitions, with a view to potentially undermining the appeal to these intuitions in traditional philosophy.
We also brought some experimental philosophy to bear on the relationship between experimental philosophy and conceptual analysis. In my talk I presented a series of vignettes ranging from (a) someone asking a number of other people for judgments about intentional action to (b) someone asking one other person for such judgments to (c) someone asking themselves for such judgments, and polled the audience on whether each counts as experimental philosophy, or as conceptual analysis. The numbers slid gradually from (a) to (c), suggesting a pretty strong continuity. The moral is that positive experimental philosophy, at least, seems fairly continuous with conceptual analysis, though with more than one subject and performed in the third-person mode.
Perhaps the most substantive issue concerning positive experimental philosophy was whether it can be used only to generate conclusions in the "formal mode" (e.g. concerning people's use of the expression "intentional action", or their concepts of intentional action), or whether it can be used to generate conclusions in the material mode (concerning intentional action itself). Josh stuck resolutely to the formal mode, distinguishing the "New Jersey plan" of reaching conclusions about cognition from the "Canberra plan" of drawing conclusions about the world. For my part, I argued that positive experimental philosophy can indeed be used to draw conclusions in the material mode, as long as one can make a case that subject's judgments (a) are not in error, and (b) do not involve concepts different from our own. Someone like Josh himself has already done relevant work supporting (a) and (b) in the case of intentional action, so it seems to me that he shouldn't be shy about drawing material mode conclusions. As soon as one accepts (as Josh does) that the CEO really did harm the environment intentionally but didn't help the environment intentionally, one has the material to draw conclusions about intentional action itself. That has the advantage of making positive experimental philosophy interesting not just as psychology, but as a contribution to other areas of first-order philosophy -- e.g., the theory of action.
As for negative experimental philosophy, the most contentious issue was whether disagreements in the sort of prima facie judgments made by experimental subjects do much to undermine the sort of idealized judgments made by philosophers in reflective circumstances. Farid argued that they do not, and Michael argued that that they don't do much to undermine moral realism, where the sort of convergence required is highly idealized ("philosophy is hard!"). In response, Steve argued that many philosophers (e.g. Gettier and Kripke) at least appear to be appealing to prima facie rather than idealized judgments. It's at least not obvious that crosscultural differences in these prima facie judgments won't ramify into differences in ideal judgments.
The other main response to the negative critique is to argue that the differences in judgments correspond to a difference in concepts, so that both groups get to be right about their own concepts. I think this move is pretty plausible in some cases and less plausible in others, with normative cases (e.g. judgments about what one ought to do) being among the less plausible cases. Frank Jackson was willing to make this move quite broadly, even embracing it in normative cases, though recognizing that it is counterintuitive there.
I suggested that perhaps we might be able to get some sort of experimental purchase on both of these responses, if only we can devise experimental tests for (i) prima facie vs ideal judgment, and (ii) conceptual difference vs conceptual similarity. There was some skepticism that one could do this in general (at least not without a big contribution from prior armchair philosophy), but even a limited and imperfect test might be useful here. E.g. if it turns out that more ideal judgments tend to produce greater convergence, that's at least an interesting data point. Though if it turns out that philosophy is so hard that we are a long way down the scale of idealization, this data point might not be more interesting than the observation that smart earthworms tend to converge!
In any case, the overall sense of the conference was that X-Phi and A-Phi can work usefully together. X-Phi needs A-Phi at many points, e.g. in interpreting its results. A-Phi can benefit from the contribution of X-Phi, both in extending its methods (more systematic conceptual analysis) and in providing a check on its methods (e.g. a check on intuitive judgments). So in the words of Josh Knobe, kum ba yah!

Hi Dave,
Re: experimental re: prima facie vs ideal judgment, coming up with an operationalization of ideal judgment strikes me as quite a bit harder than one for prima facie judgments. Any suggestions?
Posted by: Pete Mandik | July 27, 2007 at 06:29 AM
Sounds like a great conference Dave! I wish I could have made it. And I really like the powerpoint summary of the issues.
People interested in these debates about experimental philosophy may be interested in a debate between critic Antti Kauppinen and responses by Thomas Nadelhoffer and me and by Joshua Knobe in the most recent Philosophical Explorations. The info is at the exp phil blog:
http://experimentalphilosophy.typepad.com/experimental_philosophy/2007/07/the-past-and-fu.html
Posted by: Eddy Nahmias | July 27, 2007 at 09:39 AM
ZOMG, I have ur name!!! LAWL
Posted by: David Scott Chalmers | July 29, 2007 at 08:14 AM
It may help to add some CAPTCHA to this blog, see here: http://experimentalphilosophy.typepad.com/experimental_philosophy/2007/08/sorry-for-the-i.html
Posted by: Wes Anderson | August 04, 2007 at 03:29 AM
Pete: Well, a perfect operationalization of ideal judgment is something of a pipedream. But there are presumably all sorts of operationalizable measures that at least correlate with relative ideality of judgments. For example, length of reflection, articulation of reasons (and more controversially, factors such as education, IQ, and the like). Michael Smith suggested that a useful operationalization would be: build institutions for educating people and developing their judgment (call them universities), develop outlets whereby the results of well-considered judgment can be circulated and reviewed (call them journal), and look at the results.
Wes: Thanks. I've done this now. Hopefully it will help with the spam.
Posted by: djc | August 20, 2007 at 08:05 AM