Caveats concerning possible pitfalls

5. Caveats concerning possible pitfalls

In my position paper, I made several specific suggestions about how researchers should change the ways they use quantitative methods. For example, I argued for using relational codes instead of always coding discrete behaviors. It should now be clear that my comments along those lines were misleading if they suggested that I believe certain quantitative methods (e.g., discrete behavior codes) are always problematic—or, if they suggested that I wanted to rule out quite generally what others might call ‘‘strong’’ quantitative methods. According to my approach, ‘‘good’’ quantitative research includes many examples of what others consider ‘‘strong’’ methods in addition to many examples of ‘‘soft’’ methods.

Notwithstanding possible confusion, at this juncture, I also want to state that this does not amount to wholesale approval of all quantitative research. I agree with Stam that there are real dangers in what he calls ‘‘Pythagoreanism.’’ Quantitative methods frequently are employed in a problematic manner. In my opinion, this occurs when they are used in such a way that they cannot serve their interpretive function. For example, measures of decibel levels are likely to fail at assessing angry behavior if the vocalizations in question do not occur in a structured situation in which loudness serves as a concrete example of such behavior (this is circular, and that is the point). In general, quantitative methods are unhelpful in a particular case insofar as they are actually used in a way that conforms to traditional positivist conceptualizations about ‘‘real’’ measures and the like. Hence, I hope that my position paper and this rejoinder serve to mark out a view about how to employ quantitative methods and how not to employ them, rather than a position about which quantitative methods we should use.

Some brief comments are in order about techniques for statistical analysis. In large measure, I agree with the cautionary remarks Stiles offered in his commentary about ‘‘high end statistics.’’ How researchers use these techniques very often reflects what I view as a misguided understanding of quantitative research. As Stiles noted, these sophisticated techniques are often applied to decontextualized variables. One of the examples I mentioned in my position paper is relevant here. I argued that investigators studying parent–child interaction from a social learning theory vantage point attempt to explain interactions by breaking them down into isolable, elemental behaviors (e.g., prosocial child behavior, parental praise) instead of taking as their starting point what parent and child are doing together. These researchers then try to put together an account of the exchanges by statistically examining sequential dependencies between these putative building block behaviors. In my opinion, there is a great deal that cannot be recovered about the interactions when we proceed in this way, no matter how sophisticated we may get at looking at dependencies across multiple lags. At the same time, I also agree with Stiles when he urges us not to throw out inferential statistics because of its historical association with misguided notions about methodology. Notwithstanding Stam’s interesting points about longstanding problems that remain unresolved in the logic of hypothesis testing, I think these techniques can be useful. But I do wonder whether something could be gained by reexamining the assumptions of the statistical procedures we employ and considering whether some other analytic techniques are called for in light of the approach to quantitative research I have offered.

I would like to underscore one point from my position paper that is highly relevant when it comes to pitfalls associated with quantitative research. I believe that theory plays an extremely important role in whether quantitative researchers proceed in ways that are truly useful. In particular, I believe that researchers are likely to conduct ‘‘good’’ quantitative studies if they are guided by theories that are based on the idea that people are always already involved in practical activities in the world. In my position paper, I briefly discussed my participatory approach, which is an attempt to mark out a general framework for theories of this sort (also see Westerman & Steen, in press). I also gave the example of scaffolding research and pointed out that investigators in that area—in contrast to social learning theory researchers—often use relational codes rather than discrete behavior coding. In this rejoinder, I noted that when Wood did code discrete behaviors in his investigations of scaffolding (e.g., Wood & Middleton, 1975), he did so in a way (his specificity scale) that still made it possible to examine what parent and child were doing (i.e., the parent was attempting to teach the child how to build the puzzle) instead of breaking down what they were doing into isolable behaviors (e.g., prosocial behaviors, praise). He even examined sequential dependencies in a simple statistical way to study maternal homing in and out (also see Westerman, 1990), which suggests that statistical analyses, too, are likely to be useful so long as a study is based on helpful theory.