In earlier work (Palmer and Neumann forthcoming [a]), two of the authors of this report learned how difficult it is to get rich, descriptive details on the scholarly work processes, particularly those that involve the use of technology. People do not talk readily about tools that they take for granted or activities that are intellectual in nature. Work done with information resources is difficult to describe, and scholars look at their own techniques as specific to them and uninteresting to others. We dealt with these roadblocks in this study by focusing on scholars’ recent projects and combining a strategic mix of data collection techniques.
We followed the tenets of theoretical sampling, which specify that data should be collected on the basis of their ability to inform the central research questions. We also used the literature to assist in making connections between research questions and data collection and analysis (Glaser and Strauss 1967; Burgess 1984; Lincoln and Guba 1985). This tactic is based on an interest in process rather than in distribution or generalizability (Becker 1998). Applying these guidelines, we assembled a purposeful sample of humanities scholars drawn largely from the full-time faculty at the University of Illinois at Urbana-Champaign. We diversified the sample by adding participants from the University of Chicago.
Interviews were conducted with each of the 33 participants on at least two different occasions. The multiple interviews made it possible to build a richer understanding of their work than would have been possible with a single interview across the sample. We added a longitudinal dimension to some of our discussions by asking, during a subsequent interview, how research problems described in an earlier interview had been addressed. In the initial interviews, conducted between January and May 1999, we asked scholars to describe their recent work and to discuss the information processes involved in a specific recent project: what materials were used; how the materials were gathered; and how the interviewee worked through the project, including the final stages of writing. During follow-up interviews, conducted six to nine months later, we examined how projects had progressed, sought clarification on questions asked during the initial interviews, and investigated further the use of digital resources. The multiple-interview approach allowed us to customize the subsequent interviews and to obtain a fuller picture of the scholars’ research practices. The project-oriented interview approach was successful in helping participants think about and describe their work in specific terms.
Case studies of five of the scholars provided more coverage of a longer period of time and allowed us to check on the fullness of the picture of research presented in the interviews. Case-study participants were selected with points of comparability in mind, yet each addressed distinct research problems. All observations were documented through field notes; when possible, we made audio recordings of scholars’ comments. Because of differences in availability of the participants, each case varied in the amount and kinds of data collected, but each case included data from interviews, observations, and document analysis.
Two kinds of observations were performed for the case studies. The first type was devoted to a search session. Scholars were asked to search for materials they needed at that particular time. We deliberately left the search methods, topics, and venues up to the scholars. These sessions included everything from watching a scholar perform a database or a Web search to observing participants using rare books. The sessions almost always included a visit to the library. At the library, scholars used a wide variety of materials and tools, including electronic databases, card catalogs, reference works, manuscripts, microfilm readers, and general book and journal collections. We were able to document the relationship between the systems and resources in the library, the ways in which scholars carried out their tasks, the factors that influenced the search process, and the difficulties they encountered.
The second type of observation was carried out in the scholars’ personal workspaces. We observed research work in progress and recorded how the scholars organized and used the materials they kept there. Scholars were asked to “think aloud” as they worked. Although we attempted to remain unobtrusive, interviewers did interrupt with questions when scholars’ activities became unclear. We made notes and sketches of the offices and the organization of books and other belongings on shelves, in files and piles around the room, and in files and bookmarks on computers.
As part of the case studies, we also developed a document-analysis procedure for identifying the variety of resources a scholar draws upon and the ways these materials are applied in the creation of new texts. Using selected texts, we compiled a description of each item referenced by the author. Figure 1 provides a few sample lines from one document-analysis table.
The document-analysis process was then followed by a “critical incident”-style interview on the production of the paper analyzed. These interviews helped us fill in explanations for patterns we saw in the papers and obtain stories about the intellectual and physical work that went into the text. The interviews were also a useful mechanism for learning how the scholars identified and located specific sources and the attributes and relative significance they attached to them, and for gaining a sense of why they were valued. Moreover, we were able to document how the scholars constructed the arguments in the written piece, how they envisioned their audiences, and how they viewed the importance of their work.
In summary, the observations, document analyses, and critical incident interviews involved in the case studies allowed intensive investigation of humanities scholarly work from multiple perspectives. By combining data from these case studies with information from the broader sets of interviews, our study provides a rich picture of scholarly work.
Fig. 1. Document-Analysis Table
|Role in Text||Location of Reference|
|Comedy||Focus of paper, cited throughout||Introduction|
|Collection of association records||Case files||Undated||Internal documents||Historical grounding||Footnote|
|Newspaper column||Book review||Evidence-
reception of book/author; quoted
|Journal article||Film studies||Background;
|Legal code||Reprinted in film encyclopedia (1980)||Example||Footnote|
|Memorandum||Internal corporate record; part of case file||Administrative; editorial oversight||Evidence; quoted||Footnote|