4. Reviews of the Literature and Methods
Although this report summarizes conclusions from recent research studies and highlights some conclusions about how people use electronic collections, there are individual and library-specific differences that make it beneficial for many libraries to collect their own data. The last section of the bibliography in this report lists selected resources to help with this process and to identify additional usage studies.
For several years, Charles Bailey at the University of Houston has maintained a comprehensive literature review of all types of articles about scholarly electronic publishing. This monumental piece of work is the first place to look to identify articles on any aspect of the topic, including research and user studies. Since the review is updated regularly, bibliographic information about new studies appears there frequently.
Literature reviews by Kling and Callahan (2003) and Giangrande (2002) supplement Bailey; also, the Annual Review of Information Science and Technology occasionally publishes review articles that focus on electronic publishing or research techniques. Several recent relevant chapters from ARIST are listed in the bibliography. ARIST typically is published every autumn, but the topics vary from year to year.
Beyond general textbooks of research methods, several recent publications focus on research methods for library and Web usage studies. The ARIST chapter by Wang (1999) formed the basis for the categorization of research methods in this report. McClure and Lopata (1996); Liu and Cox (2002); Hurd, Blecic, and Robinson (2001); Griffiths, Hartley, and Wilson (2002); and Tenopir (2003) are all recent papers that discuss methods for collecting data and measuring usage of electronic library collections. Macintyre (2001) and Luther (2000) discuss the importance of and use of vendor statistics. Connaway (1996) and Chase and Alvarez (2000) describe how to conduct focus group interviews in information contexts.
Urquhart et al. (2003) describe in detail critical incident technique as it relates to information behavior studies. This method provides a richness in interview or survey data beyond opinions or reports of estimated behavior by asking respondents to focus on details of a specific incident of research or reading. The Tenopir and King studies also use critical incident technique to draw conclusions about readings.
Experimental (or usability) tests are less often used in the library environment, perhaps because they are time-consuming and must use relatively small groups of participants. Veldof, Prasse, and Mills (1999) and Wang (1999) provide some guidelines on running usability tests. Gullikson et al. (1999); Park (2000); Chisman, Diller, and Walbridge 1999; and Battleson, Booth, and Weintrop (2001) are some examples of practical experimental testing. Usability tests allow specific system design features to be compared and measured and are particularly useful for testing library catalog and Web site design. Think-aloud or verbal protocols provide information on why subjects pursue certain courses of action and how they react to systems at the time of use (Morrison 1999).
Probably the most important source for libraries planning to conduct their own user studies is the January 2002 report from the Digital Library Federation and CLIR by Covey (2002). Covey explains in detail when to use and how to design studies that gather data from surveys (questionnaires), focus groups, user protocols (experiments or observations), transaction log analysis, and other research methods.
In addition to excellent advice on conducting user studies, Covey (2002) presents a selected bibliography covering general research methods and specific articles through 2001 on each of the research methods she describes. The other methodological articles listed in this report were published after Covey's report.
5. Conclusions
Although there are some contradictions in the findings of the many recent research studies on user behavior with electronic library collections, some clear messages emerge. By examining the wide variety of methods, participants, and workplaces in these 200-plus studies we do know some things that library users are telling us about their use of electronic resources in the past, present, and future.
Although there is no one typical user for whom a single system design or collection decisions can be made, users can be segmented into groups that display similar preferences and patterns of use. Behavior differs based on the following:
- Status. High school students and undergraduate students, for example, turn first to the Web for research but will change behaviors if they are given a specific assignment or are asked to use a particular resource. Graduate students are heavy and cyclical users of electronic journals, especially for research. Faculty members and professionals will use electronic journals if they are convenient and support their natural work patterns. Peer reviewed journals that are considered to be core to a researcher's work will be sought regardless of convenience.
- Subject discipline, for subject experts. Scientists and business faculty members were early adopters of electronic journals and read from a variety of full-text databases and e-journals; some fields of science use many sources to get articles, including e-print servers. Social scientists and humanists use both electronic resources and print and rely more on books than other fields.
- Task. Most high school and undergraduate students turn first to the Internet for class assignments and feel they are expert searchers. The heaviest use of electronic resources is for research, followed by preparing for teaching and gaining current awareness.
- Type of institution or workplace. Academic faculty and graduate students read the most, and they readily use electronic journals accessible from their office or home, but scientists in government laboratories and companies also rely on electronic and paper journals for research. Students prefer to access electronic resources through the library from home. Users in medical libraries read from fewer journal titles than do general university or college users.
- Age. There is some evidence that younger users are more enthusiastic adopters of electronic resources than are older users. Younger users rely on electronic resources more heavily and rate themselves more expert in using them than do older users.
- Gender. There is little evidence that gender in most cultures makes a difference in use of electronic resources, although in the DLF/CLIR/Outsell studies, women report more use of electronic journals and men use Web search engines more often to locate journals.
In terms of information seeking, today's researcher seems to be comfortable with using a wide variety of sources for information. Internet search engines, e-print servers, author Web sites, full-text databases, electronic journals, and print resources are all used to some degree by most users. The relative amounts of use and enthusiasm for use vary as described above, but today's users are mostly flexible and adaptable.
Both browsing and searching remain important information- seeking behaviors, but there is some evidence that the amount of searching is going up when users have access to multi-title, full-text databases. Browsing through journal issues is done in print issues or in electronic journals for core journal titles. Articles from non-core journals are most often located through searching.
Students are highly responsive to recommendations of specific resources by their teachers, friends, or a librarian. Educating both high school and college students in the best resources, how to evaluate Web resources, and search strategies is important. Convenience remains the single most important factor for information use—all types of users prefer electronic journals only if they make their work easier and give them the information they need. Desktop access, speed of access, and the ability to download, print, and send articles are top advantages of electronic journals for all groups.
Almost universally, users report that they print out relevant articles for detailed reading. This means that both viewer-friendly formats, such as HTML, and printer-friendly formats, such as PDF, are important features in electronic journals.
Some concerns remain, such as worries that electronic journal collections may not be complete or long-lived. Concerns over the quality of e-journals seem to be diminishing as most mainstream peer-reviewed journals are digitized. Still, concerns remain over the quality of Web resources, particularly among faculty and librarians who fear students use the Web indiscriminately. There is still confusion over the variety and relative quality of e-resources, in particular among novice users or students. Archiving has been expressed as a concern in some studies.
When high-quality electronic collections are made available, people use them. Use of electronic journals increases every year. Among faculty members, graduate students, and other professionals, higher use of electronic journals is accompanied by a decrease in visits to the physical library. Access to back files and many journal titles is important to many users, although the 80/20 (or thereabouts) rule has been shown to apply to electronic journal titles. Most readings will come from a relatively small percentage of the collection, but users will read from a greater variety of titles when they are made freely and easily accessible to them.
Both Tier 1 and Tier 2 studies show that library policies have intentional and unintentional effects on user behavior. Unfettered access to electronic collections will result in an increasing use and reliance on electronic resources, although a certain percentage of use in many disciplines will continue to come from print resources for some time to come. Virtual reference services are needed to accompany this shift, as are better ways to count and report virtual library use.