Ethnographic Interviews and Writing Research: A Critical Examination of the Methodology (Chin)

We may not be able to see how interviewing is problematic because there is still a tendency to read interview data as true reflections of people's thoughts and practices. (248)

There are two types of interviews typically used in social scientific research: structured and semistructured or unstructured interviews.

• Structured Interviews: used most often in survey research, may involve f2f interaction or communication that occurs at a distance. The questions are fully formulated in advance and interviewers are trained to follow the exact wording and sequencing of the questions.
• Semistructured/Unstructured Interviews: sometimes called open-ended or nondirective may be guided by a set of questions developed in advance, but topics discussed are generally negotiated between the interviewer and interviewee.
• Discourse/Text-based Interview: for an example, see Doheny-Farina, 1989. (270)

Analyzing interview data is more akin to the interpretation required in analyzing a literary text; we imbue statements with a special significance, determined in large part by the particular theoretical frame we impose upon our projects. (252)

...the common thread in ethnographic interviews is that the questions asked reflect the researcher's growing understanding of how the culture under study is organized and how people of that culture make sense of their experiences (253-54)

Smagorinsky on Methods

In this article, I’d like to argue for greater attention to accounts of research method, both for the reader’s sake and the writer’s. As a reader, I simply need to know how data become results in order to trust the author’s claims. But for me as a writer, the Method section plays a pivotal role in the production of a research article. It serves as the core from which radiate the content and orga- nization of each of the other sections of an APA-style research report. (394)

Time after time, I run across something such as the following:

I read the data several times. In the first reading, I generated provisional cat- egories to guide my subsequent readings. Then, I read the data and refined these categories, looking for themes and patterns. Upon further reading I cre- ated the ultimate categories used for the analysis. And now, my Results.

I’m all for multiple readings of data and continual refinement of categories but I need to know specifically what the categories are and how they evolved through the process of data reduction and analysis, how they are applied as codes, how they work in relation to the author’s framing theory, how they are reduced from a “raw” state to “cooked,” and so on. (393)

I have the same feeling all too often when reviewing manuscripts for journals: I have only the vaguest sense of what the author is doing with the data in order to render it into results. If I don’t know pretty clearly how the researcher is conducting the study, then it doesn’t matter much to me what the results are because I have no idea of how they were produced. To me, that’s reason enough to recommend that the article not be published. (393)

Indeed, many have argued that particularity, rather than generalizability and replicability in this sense, is a worthy aim of research (e.g., Bloome & Bailey, 1992; Valsiner, 1998). (394)

Smagorinsky's model:

• collection of data
• reduction of data
• analysis of data

Data Collection

As Chin (1994) has argued, simply announcing that data are composed of “interviews” overlooks the fact that interviews may be conducted in many ways, obligating the researcher to be explicit about who conducted the interviews, whether or not multiple interviewers were involved and if so, how consistency across interviewers was achieved (e.g., relying on a uni- form interview protocol or set of prompts and providing the text of such scripts), and other factors that help to reveal the specific nature of the data collection. I use interviews here for illustrative purposes; virtually any qual- itative research method benefits from explication of this sort. (395)

Limitations and cautions about the data collection procedures also merit attention. (395)

Further, the report of method should be tied to the study’s motivating theory in terms of data collection, reduction, and analysis. (395)

Data Reduction

All researchers reduce their data: some to numbers, some to words, some to both. (397)

The researcher’s task is to take this amorphous mass of data and reduce it to something comprehensible and useful. (397)

The familiar pattern I’ve seen so often in manuscripts I’m asked to review—that of providing a general account of data reduction involving reading, finding provisional themes, and developing and refining codes, yet without explicitly naming them—does little to illuminate for readers how a researcher has reduced data from an inchoate corpus to a systematically organized set from which a subset can document representative trends. As a reader, I want to know the principles by which an author has either elim- inated data or selected something representative. Simply announcing that something is representative of the larger corpus is not convincing. (397)

Related to the reduction of data so that representative samples are available is attention to disconfirming or discrepant data, that is, data that are unrepresentative of the corpus as a whole and that raise questions about available generalizations. (397)

Disconfirming data...

Data Analysis

...the means by which a reduced data set gets rendered into a persuasive, evidentiary set of results. (398)

Coding as the manifestation of theory. (399)

coding makes evident the theoretical approach used to analyze the data by applying code names to segments of text (typically, in my work, field notes, interview transcripts, and transcripts of people speak- ing as they work). In this conception, coding manifests what theory would say about data and makes the researcher’s theoretical perspective on the data corpus explicit, without precluding other ways of looking at it. (399)

coding establishes the researcher’s subjectivity in relation to the data and the framework through which data are interpreted. From this perspec- tive, the codes are not static or hegemonic but rather serve to explicate the stance and interpretive approach that the researcher brings to the data. (399)

The Context of the Investigation

...it has become de rigueur for authors to include a section on the context of the investigation. In many manuscripts, the Context section is included in the Method section but I’ve begun breaking it out as a separate entity in spite of the Publication Manual of the American Psychological Association’s (2001) guideline to the contrary. I believe that while context and method are related, context merits its own attention (403)

As a school-based researcher, I often provide skeletal statistics about a school’s size and demographics and profiles of the primary participants in the research. (404)

the ultimate shape of the context section is a function of what I’m able to gather and include and what an editor believes to be relevant to the report, within the page limits for both individual articles and the overall annual page maximum that a journal’s sponsor allows its editors to produce. (404)

In my experience, studies work best when an author poses a limited set of answerable questions and then aligns the paper around them, making sure that they are the- orized, that the method produces data that serve as evidence for claims, that the results are presented in clear relation to the questions, and that the discussion follows from the analysis. (405)

I always write the Method section very early in the process because most of what I need to consider when writing it affects other parts of the paper. (405)

...the Results need to invoke the codes so that it’s clear how the analysis has produced the findings. (407)

But even when a method is clearly explained, an author often proceeds to report results with no further mention of how the analysis produced them. Referring back to the analytic method while reporting results ought to be, I believe, a standard move in published research. (407)

A research method requires a theoretical perspective (407-408)

“Deep Theorizing”: Closing the Gap Between Text and Context in Academic Writing Research (by Theresa Lillis)

Ethnography as Method, Methodology, and “Deep Theorizing”: Closing the Gap Between Text and Context in Academic Writing Research (by Theresa Lillis)

And a good deal of what she is saying is helping me to think through the problems with my study. For example, something I have wondered about for some time: what IS the difference between METHOD and METHODOLOGY??? Well, here is Lillis on this question:

Interestingly for the discussion in this article, they list ethnography as a method, alongside categories such as interviews, observation, and case studies, rather than as a methodology constituted by multiple methods such as those they mention separately. (355)

And here's a passage which really resonates with me for the way it exposes the limited nature of my own study:

The point here is not that their classification is incorrect (classifications and taxonomies always involve a range of compromises) but rather that their positioning of ethnography in this way says much about the way in which ethnography is currently (often) framed in academic writing research.First, it tends to be viewed as a method rather than a fully developed methodology. Second, this conceptualization of ethnography as method in academic writing research often gets operationalized as one specific method—notably, the interview, or more precisely, talk around texts—which while offering the researcher an additional lens from which to understand the text, leads to only a truncated engagement with “context.” Third, this reduction of ethnography to method means that the value of other levels of ethnography, that is, as fully fledged methodology and as a specific epistemology and ontology, are still often overlooked in academic writing research. (355)

So, yes, a confusion about method and methodology that lead me to focus not enough on context. I never visited a workplace, never sat and listened and observed. I never visited a class, never sat and observed. I have had to try to imagine those contexts. We talked about students' lives and writing and we looked at the documents in isolation and while that was incredibly useful for me, personally, and I learned a lot and I might even argue that I've been engaged in the inevitable steps of an inevitable process, it doesn't allow me to DO a good deal with the data.

A useful definition:

...ethnography as methodology: the weaving together of data in order to understand a particular phenomenon. (356)

Another moment that has really helped me with the Lillis article is seeing her "Table 1:

Overview of Two Projects: Student Writing and Professional Academic Writing" (357). This table is so useful--to see the outline of these two projects and the data sources. It is the data sources piece which I find most useful.

And this passage resonates for me (I've changed it around a bit):

...core characteristics of ethnography: the importance of lengthy or sustained engagement over a significant period of time and the value of multiple data sources. (358)

This passage comes at the conclusion of a long bulleted list in which Lillis attempts to articulate a "broad working definition of ethnography."

What she does from there I find really interesting--she talks about the different METHODS within the METHODOLOGY called ETHNOGRAPHY and describes the constraints and affordances of each method (limits and opportunities) so that we can see a) that research methods have limits in terms of what they can tell us, b) to make the case for the triangulation of methods or data sources. The picture emerges from the triangulation, not from single methods or sources.

Bibliography

Qualitative Research

Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for
developing grounded theory. Thousand Oaks, CA: Sage.

Bogdan, R., & Biklen, S. (1998). Qualitative research for education: An introduction to theory and methods (3rd ed.). Boston, MA: Allyn & Bacon.

Discourse Analysis:

Atkinson, D. (1991). Discourse analysis and written discourse conventions. Annual Review of Applied Linguistics, 11, 57-76.

Atkinson, D. & Coffey, A. (1997). Analyzing documentary realities. In D. Silverman (Ed.), Qualitative Research: Theory, method, and practice (pp. 45-62). London: Sage.

Fairclough, N. 1992a. Discourse and social change. Cambridge: Polity Press

Fairclough, N. 1992b. "Discourse and the text: linguistic and intertextual analysis within discourse analysis." Discourse and Society 3.2 (pp. 193-217)

Barton, Ellen. "Linguistics and Discourse Analysis." In English Studies: An Introduction to the Disciplines. (green book--I own it).

Ethnographic Research

Emerson, R.M., Fretz, R.I, & Shaw, L.L. (1995). Writing ethnographic fieldnotes. Chicago: University of Chicago Press.

Lincoln, Y., & Guba. (1985). Naturalistic Inquiry. New York: Sage.

Writing Social Science Articles

Smagorinsky, P. (2008). The method section as conceptual epicenter in constructing social science reports. Written Communication, 25 389-411.

Lillis, Theresa. Ethnography as Method, Methodology, and "Deep Theorizing: Closing the gap between text and context in academic writing research." Written Communication 25.3 (July 2008): 353-388.

The Interview in Ethnographic Research

Chin, E. (1994). Ethnographic interviews and writing research. In P. Smagorinsky (Ed.),
Speaking about writing: Reflections on research methodology (pp. 247-272). Thousand
Oaks, CA: Sage.

Although more research on writing now uses qualitative methods such as interviewing, little has been written about the ways in which the interview can be used in writing research, the specific questions that the use of such a method raises, or the difficulties researchers might encounter in using interviews as a means of gathering data. (247)

deMarrais, K. (2004). "Qualitative interview studies: Learning through experience." In K. deMarrais & S.D. Lapan (Eds.), Foundations for research: Methods of inquiry in education and the social sciences (pp. 51-68). Mahwah, NJ: Lawrence Erlbaum.

Discourse-Based Interviews

Odell, Lee, Dixie Goswami, Anne Herrington, and Doris Quick. "Studying Writing in Non-Academic Setings. New Essays in Technical and Scientific Communication: Research, Theory, Practice. Eds. P. Anderson, R. Brockman, and C. Miller. Farmingdale, N.Y.: Baywood, 1983, p. 17-40.

Research Involving Verbal Reports (Think-Aloud Protocols)

Smagorinsky, P. (1994). Speaking about Writing: Reflections on research methodology. Thousand Oaks, CA: SAGE.

The book serves two basic purposes: to continue the debate about the validity of research methods involving verbal reports, and to explicate the problems and potentials of such methodologies for the benefit of prospective researchers. (xiii)