Smagorinsky on Methods

In this article, I’d like to argue for greater attention to accounts of research method, both for the reader’s sake and the writer’s. As a reader, I simply need to know how data become results in order to trust the author’s claims. But for me as a writer, the Method section plays a pivotal role in the production of a research article. It serves as the core from which radiate the content and orga- nization of each of the other sections of an APA-style research report. (394)

Time after time, I run across something such as the following:

I read the data several times. In the first reading, I generated provisional cat- egories to guide my subsequent readings. Then, I read the data and refined these categories, looking for themes and patterns. Upon further reading I cre- ated the ultimate categories used for the analysis. And now, my Results.

I’m all for multiple readings of data and continual refinement of categories but I need to know specifically what the categories are and how they evolved through the process of data reduction and analysis, how they are applied as codes, how they work in relation to the author’s framing theory, how they are reduced from a “raw” state to “cooked,” and so on. (393)

I have the same feeling all too often when reviewing manuscripts for journals: I have only the vaguest sense of what the author is doing with the data in order to render it into results. If I don’t know pretty clearly how the researcher is conducting the study, then it doesn’t matter much to me what the results are because I have no idea of how they were produced. To me, that’s reason enough to recommend that the article not be published. (393)

Indeed, many have argued that particularity, rather than generalizability and replicability in this sense, is a worthy aim of research (e.g., Bloome & Bailey, 1992; Valsiner, 1998). (394)

Smagorinsky's model:

• collection of data
• reduction of data
• analysis of data

Data Collection

As Chin (1994) has argued, simply announcing that data are composed of “interviews” overlooks the fact that interviews may be conducted in many ways, obligating the researcher to be explicit about who conducted the interviews, whether or not multiple interviewers were involved and if so, how consistency across interviewers was achieved (e.g., relying on a uni- form interview protocol or set of prompts and providing the text of such scripts), and other factors that help to reveal the specific nature of the data collection. I use interviews here for illustrative purposes; virtually any qual- itative research method benefits from explication of this sort. (395)

Limitations and cautions about the data collection procedures also merit attention. (395)

Further, the report of method should be tied to the study’s motivating theory in terms of data collection, reduction, and analysis. (395)

Data Reduction

All researchers reduce their data: some to numbers, some to words, some to both. (397)

The researcher’s task is to take this amorphous mass of data and reduce it to something comprehensible and useful. (397)

The familiar pattern I’ve seen so often in manuscripts I’m asked to review—that of providing a general account of data reduction involving reading, finding provisional themes, and developing and refining codes, yet without explicitly naming them—does little to illuminate for readers how a researcher has reduced data from an inchoate corpus to a systematically organized set from which a subset can document representative trends. As a reader, I want to know the principles by which an author has either elim- inated data or selected something representative. Simply announcing that something is representative of the larger corpus is not convincing. (397)

Related to the reduction of data so that representative samples are available is attention to disconfirming or discrepant data, that is, data that are unrepresentative of the corpus as a whole and that raise questions about available generalizations. (397)

Disconfirming data...

Data Analysis

...the means by which a reduced data set gets rendered into a persuasive, evidentiary set of results. (398)

Coding as the manifestation of theory. (399)

coding makes evident the theoretical approach used to analyze the data by applying code names to segments of text (typically, in my work, field notes, interview transcripts, and transcripts of people speak- ing as they work). In this conception, coding manifests what theory would say about data and makes the researcher’s theoretical perspective on the data corpus explicit, without precluding other ways of looking at it. (399)

coding establishes the researcher’s subjectivity in relation to the data and the framework through which data are interpreted. From this perspec- tive, the codes are not static or hegemonic but rather serve to explicate the stance and interpretive approach that the researcher brings to the data. (399)

The Context of the Investigation

...it has become de rigueur for authors to include a section on the context of the investigation. In many manuscripts, the Context section is included in the Method section but I’ve begun breaking it out as a separate entity in spite of the Publication Manual of the American Psychological Association’s (2001) guideline to the contrary. I believe that while context and method are related, context merits its own attention (403)

As a school-based researcher, I often provide skeletal statistics about a school’s size and demographics and profiles of the primary participants in the research. (404)

the ultimate shape of the context section is a function of what I’m able to gather and include and what an editor believes to be relevant to the report, within the page limits for both individual articles and the overall annual page maximum that a journal’s sponsor allows its editors to produce. (404)

In my experience, studies work best when an author poses a limited set of answerable questions and then aligns the paper around them, making sure that they are the- orized, that the method produces data that serve as evidence for claims, that the results are presented in clear relation to the questions, and that the discussion follows from the analysis. (405)

I always write the Method section very early in the process because most of what I need to consider when writing it affects other parts of the paper. (405)

...the Results need to invoke the codes so that it’s clear how the analysis has produced the findings. (407)

But even when a method is clearly explained, an author often proceeds to report results with no further mention of how the analysis produced them. Referring back to the analytic method while reporting results ought to be, I believe, a standard move in published research. (407)

A research method requires a theoretical perspective (407-408)