2. Tier 1 Studies

2.1 Overview

Eight groups of studies were identified as “Tier 1” or major recent research studies on how people use electronic library resources. Tier 1 studies are (in no particular order):

SuperJournal
Digital Library Federation/Council on Library and Information Resources/Outsell (DLF/CLIR/Outsell)
HighWire/eJUSt
Pew Internet and American Life (with comparison to OCLC/Harris and Urban Libraries Council)
OhioLINK
Tenopir and King studies
LibQUAL+^TM
JSTOR studies

A synopsis of each is given first, followed by an analysis of the methods used, participants included, levels of conclusions, and findings for each group. In the bibliography, all of the publications that report on each study are listed together by the study group name.

2.1.1 SuperJournal

The SuperJournal project is a group of studies of e-journal use that began in 1995 in the United Kingdom in response to the information explosion and limited budgets. The researchers use a variety of research methods, including log file analysis, surveys, interviews, and focus groups, to study how academic users interact with e-journals and what features they value. Academic scientists and social scientists were studied, including both faculty and students in British universities.

2.1.2 Digital Library Federation/Council on Library and Information Resources/Outsell (DLF/CLIR/Outsell)

Outsell, Inc., conducted a survey of information use for the Digital Library Federation and Council on Library and Information Resources in the fall of 2001 and early winter of 2002. Some 3,234 faculty, graduate students, and undergraduate students across seven subject disciplines at private and public doctoral research universities and leading liberal arts colleges were interviewed over the telephone. They were asked about their use and preferences for both print and electronic resources from the library.

2.1.3 HighWire/eJUSt

The Stanford E-Journal Users Study (e-JUSt), published by HighWire Press, used a variety of methods to gain insights into the use of electronic journals, including qualitative user surveys, transaction log analysis, and an ethnographic study of scholarly e-journal usage. The qualitative user surveys were done online with participants taken from subscribers to HighWire’s medical and scientific journal Table of Contents service. The participants included graduate students, faculty members, and clinicians from universities, hospitals, and government and academic research institutes from 99 countries. The studies were conducted between November 2000 and August 2002.

2.1.4 Pew Internet and American Life (also OCLC/Harris, and Urban Libraries Council)

The Pew Internet and American Life Project conducted two studies about how students use the Internet. In the “Internet Goes to College,” 2,054 college students at two- and four-year public and private colleges completed surveys. In addition, graduate student researchers observed the behavior of college students at Chicago area colleges and universities. In the other Pew Internet and American Life Project, “The Digital Disconnect: The Widening Gap between Internet Savvy Students and their Schools,” middle and high school students were studied between November 2001 and March 2002. About 200 students wrote essays in which they expressed how they and their friends used the Internet for school and how they might use it in the future. Both these studies included how the students view the library. OCLC/Harris and the Urban Libraries Council conducted similar surveys comparing library and Internet use by students and the public respectively. In the OCLC/Harris study, 1,050 participants were surveyed between December 11, 2001, and January 1, 2002. In the Urban Libraries Council study, 3,097 participants were surveyed by telephone between March and April 2000.

2.1.5 OhioLINK

The Ohio Library and Information Network is a consortium of Ohio’s college and university libraries and the State Library of Ohio. The consortium serves in excess of 500,000 students, faculty, and staff at more than 80 institutions of higher learning. OhioLINK’s Electronic Journal Center makes electronic articles and journals available to OhioLINK members. Transaction log analysis is used to measure the number of articles users download from the Electronic Journal Center. This program, begun in April 1998, is ongoing.

2.1.6 Tenopir and King Studies

The Tenopir and King research studies are a series of surveys of more than 16,000 scientists, engineers, medical professionals, and social scientists in university and non-university research settings. The surveys measure reading and authorship patterns of these subject experts through critical incident, demographic, and usage questions. Information-seeking behaviors, amount of reading, purposes of reading, and source of readings are all measured. Recent studies have focused on how reading patterns have changed over time with the adoption of e-journals and what role library-provided journals play in overall reading patterns. These ongoing experiments began in 1977.

2.1.7 LibQUAL+^TM

LibQUAL+^TM, conducted by the Association of Research Libraries (ARL) in conjunction with Texas A & M University, surveyed students, faculty, and staff at various community colleges, four-year colleges, and health science schools in the United States as well as the New York Public Library and Smithsonian Institution during the spring of 2002. More than 70,000 faculty, staff, and students related how often they used the physical and electronic libraries. Furthermore, they answered questions about their library’s level of service that they found minimally acceptable, the level they perceived, and the level they desired. The results are presented by status of respondent and type of institution. Only those few questions that focus on desired levels for print and electronic collections and services are relevant and reported here.

2.1.8 JSTOR

The JSTOR system provides electronic archives of back issues of scholarly journals. JSTOR uses log analysis of both viewed and printed articles to characterize use of its materials. In addition, some JSTOR subscribing libraries have analyzed their use of the JSTOR journals within their specific library environment. In the fall of 2000, JSTOR surveyed more than 4,000 academic users of the collection in humanities, social sciences, and economics to discover usage patterns and preferences of university faculty.

2.2 Participants

Each of the eight Tier 1 studies examined a variety of participants, with college and university students and faculty members the most often studied, followed by practitioners and other subject experts in science, engineering, health, and social sciences. Table 2.1 summarizes the main participants included in each study.

Study	Participants
SuperJournal	Students and faculty
DLF/CLIR/Outsell	Students and faculty
HighWire/eJUSt	Scholars and clinicians
Pew/OCLC-Harris/Urban Libraries Council	Middle, high, and college students/general public
OhioLINK	OhioLINK users
Tenopir and King	Scientists and social scientists (academic and non-academic)
LibQUAL+^TM	Library users at institutions of higher education (students and faculty)
JSTOR	JSTOR users (mostly faculty)

Table 2.1. Tier 1: Participants

2.3 Methods

The method or methods used in a research study determine what types of conclusions can be drawn about the sampled participants and what findings can be generalized to the population as a whole. Wang (1999) provides an overview of methods for user behavioral research. An extension of her categorization of methods is used here to describe Tier 1 studies. Tier 1 studies use one or more of the following methods:

surveying users
interviewing users (including focus groups)
observing users through experiments
observing users in natural settings (including keeping journals)
transaction log analysis (included under “observing users” in Wang 1999)

Covey (2002) also categorizes usage studies to help librarians design the most appropriate studies for the type of information they hope to gather. Covey’s categories of research studies are similar to Wang’s and include the following:

surveys (questionnaires)
focus groups
user protocols (experiments and observations are both included here)
other (heuristic evaluations, paper prototypes and scenarios, and card-sorting tests)
transaction log analysis

Table 2.2 summarizes the methods used by the Tier 1 studies. Several use multiple methods for different phases of their projects; others rely on a single method.

Study	Methods
SuperJournal	Logs/surveys/focus groups/ interviews
DLF/CLIR/Outsell	Interviews
HighWire/eJUSt	Surveys/interviews/logs
Pew/OCLC-Harris/Urban Libraries Council	Surveys/observation/focus groups/journal keeping
OhioLINK	Logs
Tenopir and King	Surveys/critical incident
LibQUAL+^TM	Surveys
JSTOR	Logs

Table 2.2. Tier 1: Methods Used

Surveys of users are typically done by sending a questionnaire by e-mail, the Web, or paper mail to a randomly selected percentage of the population under study. Tenopir and King, for example, survey samples of university faculty, members of professional organizations such as the American Astronomical Society, and scientists in companies and government laboratories. LibQUAL+^TM libraries survey students and faculty within their own university community for comparison with other LibQUAL+^TM libraries. Conclusions based on the responses are generalized to the whole using appropriate statistical tests. Care in selecting samples and a reasonable return rate are necessary to draw valid conclusions.

Almost all of the studies reported here that use surveys follow these basic precepts of sampling and analysis, but the types of conclusions that can be drawn vary by the types of questions that are asked. Among the Tier 1 studies that use surveys, the main distinctions in types of questions asked can be characterized as follows: 1. preference (focusing on what people want or think about a particular service; e.g., LibQUAL+^TM, Pew) 2. reported behavior (focusing on what people say they do in general; e.g., DLF/CLIR/Outsell, HighWire/eJUSt) 3. critical incident questions (focusing on what people say they do in regard to a specific instance or reading; e.g., Tenopir and King).¹

Table 2.3 shows methods in more depth by looking at what types of questions were asked.

Study	Type of Questions
SuperJournal	Preference and reported behavior
DLF/CLIR/Outsell	Preference and reported behavior
HighWire/eJUSt	Preference and reported behavior
Pew/OCLC-Harris/Urban Libraries Council	Preference and behavior-reported and observed
OhioLINK	Log analysis
Tenopir and King	Critical incident, preference and reported behavior
LibQUAL+^TM	Preference and reported behavior
JSTOR	Preference and reported behavior and log analysis

Table 2.3. Tier 1: Types of Questions

Together, the categories of participants and the methods used (as outlined in Tables 2.2 and 2.3) determine at which of three levels valid conclusions can be drawn:

the “user level,” that is, what do individuals or groups of individuals such as social science faculty say they do or prefer;
the “group level,” that is, what do groups of users at an institution do, without demographic differentiation; or
the “readings or incident level,” that is, what do specific users or groups of users do or prefer about a specific type of information or reading (see Table 2.4).

Study	Conclusions
SuperJournal	User level
DLF/CLIR/Outsell	User level
HighWire/eJUSt	User level
Pew/OCLC-Harris/Urban Libraries Council	User level
OhioLINK	Group level
Tenopir and King	User and reading levels
LibQUAL+^TM	User level
JSTOR	User and group level

Table 2.4. Tier 1: Conclusion Level

SuperJournal, for example, uses transaction logs, surveys with questions about preferences and behavior, focus groups, and interviews to study faculty and graduate students. Demographic information is known for each user. These multiple methods allow conclusions to be drawn at the user level for both behavior and preferences (what specific types of users do and what they prefer). JSTOR uses transaction logs separate from survey questions. DLF/CLIR/Outsell used interviews to gather information on what users say they prefer and say they do in general. Demographic information is known about each user. This allows conclusions to be drawn at the user level as well. Tenopir and King use critical incident questions in their surveys, which ask users to focus on the last article read. Together with demographic data, this allows conclusions to be made at the readings level (characteristics of the total amount of readings done by individuals and groups of individuals).

Each of these methods has advantages and disadvantages. According to Covey (2002), problems or concerns with surveys include the following:

General surveys are time-consuming and expensive to prepare, conduct, and interpret.
Unless follow-ups are sent so longitudinal analysis can track changing patterns of use, surveys provide no baseline data.
People receive many surveys, and it is difficult to motivate them to complete and return surveys.
The usage information gathered in general surveys might better be gathered by transactional logs.
Specific surveys are more beneficial, but must be repeated over time.
User satisfaction surveys may not provide enough information to solve the problem, and service “gap” surveys are more difficult to administer and analyze.
A survey is only as good as the wording of the questions and the response rate.

Problems or concerns with focus groups, according to Covey, include the following:

A skilled moderator needs to direct the groups to keep discussions on track.
An unskilled observer may fail to take adequate notes.
The qualitative data gathered in focus groups can be time consuming and difficult to interpret.

Problems or concerns with experiments or observations (called user protocols by Covey), include the following:

Librarians, if observing, have a difficult time not assisting the subjects.
Librarians may not be trained to interpret and analyze the data from user protocols.
Recruiting subjects, in particular subjects who are comfortable with the process of thinking aloud, is difficult.

Problems with transaction log analysis, according to Covey, include the following:

deciding on the right and most useful usage statistics
collecting the right usage statistics
getting the right and consistent usage statistics from vendors
analyzing and interpreting data (it can be time consuming and difficult)
presenting the data in a meaningful way

In summary, the conclusions that can be drawn from each of the Tier 1 and Tier 2 studies depends on the methods used, including the overall method(s), types of questions asked, level of questions, and participants studied. It may be tempting for a researcher to draw broader conclusions than his or her methods justify-a failing that was found in more than one study examined. Only those findings that are justified by each study’s methods are reported here. In general, the following types of conclusions can be drawn from each technique:

Transaction logs: what groups do in general; for example, what the college or university libraries in the OhioLINK system do in general. Transaction logs do not show preferences; rather, they show action from which preferences are often inferred. Demographic data from individuals are usually not gathered because of privacy concerns. Instead usage is identified by IP address, location, or library, so conclusions about differences in work fields or status of user cannot be drawn. Transaction logs that result from an experimental design allow more finely tuned conclusions (see Observation: experimental, below).

Interviews or surveys: preference questions. Preference questions, or questions about what people want, show what people say they prefer or value. Demographic information is almost always asked, allowing conclusions to be made about groups. (For example, in HighWire/eJUSt, most science faculty members say their favorite e-journal feature is linking.) Preferences may or may not predict actual or future behavior.

Interviews or surveys: behavioral questions. Questions about behavior in general (for example, “Do you use the library’s electronic journals?”) show what people do at least some of the time. Demographic information is almost always gathered as well, allowing conclusions to be drawn about individuals or groups of individuals. (For example, the DLF/CLIR/Outsell study shows that most humanities faculty members use the print collection for at least part of their work.)

Interviews or surveys: critical incident questions. Respondents are asked to focus on a specific incident; for example, the last article they read or the last article they authored. Specific questions about that incident are then asked. Demographic information is also collected. This allows conclusions to be made about the total amount of reading or for specific characteristics of users or readings. (For example, Tenopir et al. [2003] report that 80% of the articles astronomers read in a year are from electronic sources.)

Focus groups. Focus group participants are not randomly selected; instead, individuals who can express opinions about a service or issue are invited to participate. Therefore, preferences and behaviors observed in focus groups must not be overly generalized to the population as a whole, nor interpreted as the only possibilities. (For example, from SuperJournal, faculty members report a variety of uses for electronic journals, including keeping current in their areas of research, gathering background information, and preparing for a specific event.) Focus groups are helpful as a first step or in conjunction with other research methods.

Observation: experimental. Controlled experiments gather both quantitative and qualitative data on how users behave in a controlled environment, such as searching on a specific online system, and why they behave in a certain way. Demographic data are gathered, along with other data about individual differences, such as from controlled tests. Conclusions depend on the experimental treatment. (For example, log analysis in conjunction with experimental observation in the SuperJournal study shows that social scientists browse differently than scientists.)

Observation in natural setting: journal keeping. Participants are asked to record their interactions with information systems or their research process. Conclusions can be drawn about types of behavior, and models of behavior can be derived. (For example, in the Pew Studies, students recorded using electronic resources more than print resources and felt they knew more about the Internet than do their teachers.)

2.4 Tier 1: Analysis

2.4.1 SuperJournal

The SuperJournal studies use a rich variety of methods-including observations, interviews, focus groups, transaction logs, etc.-within the controlled environment of a test database of journals and journal articles. It is one of the best-designed controlled studies of how faculty, undergraduate, and graduate students interact with and use electronic journals. Conclusions are made within the boundaries of subjects, test settings, and resources under scrutiny.

SuperJournal found that users vary in their patterns of use, depending on their subject discipline and status (faculty, graduate students, or undergraduate students). It identified seven categories of e-journal users:

enthused (one or two sessions per month, wide use of journals and articles, mostly social scientists and graduate students)
journal-focused (many sessions but concentrated on 4 or 5 specific journal titles and 50% full text, mostly scientists and graduate -students)
topic-focused (searched less often and by subject rather than -specific journals, used many articles, mostly social scientists)
article-focused (searched less often, only on one journal, mostly scientists)
bingers (mostly social science students)
explorers (students across all disciplines, used tables of contents in multiple journals)
window-shoppers (students who viewed the journal system just once and did not use the full-text database)

A system must accommodate all of these variations in use by including features that enable browsing through the table of contents or journals, searching for topics or articles, creating topical subsets of journals or articles, and searching across the entire database.

Social scientists tend to retrieve recent articles of interest through vertical chaining (going from table of contents, to abstract, to full-text). Scientists often browse journal titles, retrieve known articles, and do vertical leaping (table of contents to full-text.) Social science students viewed multiple journal tables of contents while using e-journals to fulfill a specific class assignment. Both browsing through tables of contents of known journals and searching in full-text databases are important, but the relative importance of each varies by work field and status. Once a relevant article is identified, most users print it out.

Focus groups in the SuperJournal project were used as baseline studies. They identified a variety of reported behavior and uses of electronic journals, including the following:

keeping current with articles in the user’s area of research
keeping up to date with what is being published more broadly in related areas
gathering background information on a new area on which the user might be embarking, such as a new experiment
preparing for a specific event such as writing an essay or grant proposal
performing tasks associated with teaching, such as writing and updating lectures and reading lists

Differences between disciplines were evident in the focus groups, leading the SuperJournal researchers to conclude the following:

Social scientists seemed to be more task-driven than scientists.
Social scientists visited the library less often than scientists when new journals appeared (the former are more likely to visit quarterly). Social scientists used databases in the library without mentioning any particular database.
Social scientists expressed less anxiety about keeping up to date, while scientists expressed a feeling that there was not enough time to keep up to date.
Scientists didn’t think they were finding all the articles that they needed to find.
Scientists seek articles on a more regular basis.
Scientists combine online database searching and browsing.
Both scientists and social scientists value the library as the institution that provides them with journals.

At the end of the entire SuperJournal project, concluding focus groups revealed that as a result of their exposure to electronic journals:

They visit the library less because of desktop access.
They accomplish tasks more efficiently.
They felt more up to date.
Users do what works best for them. There appears to be no change through the project in individual preferences for searching and browsing.

2.4.2 DLF/CLIR/Outsell

Interviews in the DLF/CLIR/Outsell study also found that the use of electronic journals varies with the subject discipline, the use (teaching or research), and the status of the individual. According to the survey results, the percentage of faculty who use e-journals for research varies between a low of 62.1% (law) and a high of 83.3 % (biological sciences). The percentage of faculty who use e-journals for teaching ranges between a low of 27.7% (law) and a high of 55.5% (biological sciences). The percentage of students who use e-journals varies between a low of 35% (law) to a high of 61.7% (biological -sciences).

Although 80% of the respondents across all disciplines say that the Internet has changed the way they look for information, only about one quarter of the faculty and 38.3% of the students said they needed more online journals. More graduate students desire additional e-journals than do faculty or undergraduates.

About 80% of faculty and graduate students access e-journals online, and 75% prefer this mode of access. About 23% use e-journals in the library, but only 13.9% prefer to access them this way. Almost three-quarters of the faculty access electronic information from their office or home. About three-quarters of students access the journals online, and most prefer to do this but only 68.5% of arts and humanities students reported online use.

Respondents report a difference in how they trust information that comes from the library versus that from the Internet. Almost two-thirds of the faculty (62%) and graduate students (66%) say they use the library’s Web site. When information comes from the library almost all (98.2%) believe it is from a credible source. Less than half (45.9 %) reported using information from the Internet without verifying it.

DLF/CLIR/Outsell made some other interesting discoveries as follows: Respondents differ in their level of comfort with electronic information depending on discipline and status. Respondents in the arts and humanities do not feel as comfortable with electronic information as respondents in social sciences, engineering, and business.

Most people (72% of respondents) print out the information they find online. DLF/CLIR/Outsell found some differences between men’s and women’s reported use of online resources. More women than men say they use e-journals in their research (80.4% and 72%, respectively). In teaching, men report significantly more use of a search engine to access e-journals than women who access e-journals (23.8% and 5.9%, respectively). Women report that they use their institution’s Web site more often than did men (26.5% and 11.6%, respectively) to access e-journals for teaching. More women than men in research use electronic sources most of the time (37.4% and 31.7%, respectively) or all of the time (7.4% and 4.2%, respectively).

Not only did respondents differ in their use of e-journals based on status, discipline, and gender, they also differ in how they find information about e-journals. More than 90% of the faculty and graduate students who use e-journals for research in biological sciences, physical sciences/math, social sciences, and business find information about them online. More faculty members than graduate students (92.5% and 75.5% respectively) use online sources to discover information about e-journals. When it comes to course work, students learn about e-journals in a variety of ways. Most use online sources, but the percentage of the respondents varies according to discipline, with 88.4% of the biological sciences students using online sources compared with only 69.8% of the engineering students. More undergraduates said they preferred to use a search engine than did graduate students.

The DLF/CLIR/Outsell survey cuts across all sizes of colleges and universities and includes faculty members, graduate students, and undergraduates in all subject disciplines. The findings show reported behavior and preferences and how these respondents use resources at least some of the time.

2.4.3 HighWire eJUSt

The Stanford E-Journal User Study, published by HighWire Press, surveyed members of professional societies in life sciences that are affiliated with HighWire Press to find preferences and reported behavior. Although 92% of respondents reported they like online retrieval because it is convenient, they still prefer printed copies or paper journals for reading. Two-thirds of the respondents report that they print out selected e-articles for reading and for their own archives, and they do not like HTML for printing.

Like many other studies, eJUSt found differences in preferences and behavior between work fields and work roles. Biologists are more likely to read e-journals than are other life scientists, but the reason is unknown. Clinicians and biology researchers use e-journals differently: clinicians search online material (often abstracts) for educational and clinical purposes, while biology researchers use online material for research. Health professionals with an M.D. degree use abstracts rather than full-text articles to access treatment protocols and say they would go without an article rather than pay for online access. Those without the M.D. degree use e-journals even more often. Both life scientists and medical practitioners appear to have increased their e-journal usage somewhat from the first to the third eJUSt survey (May 22ÐJune 20, 2001, and MayÐAugust 2002). In addition, more than half (52%) of the respondents said that e-journals make them aware of the literature on the periphery of their discipline, and some said e-journals increase their ability to communicate with their peers.

Scientists say they like e-journals for retrieving full-text articles and for ease of browsing and searching. While searching, 77% of the scholars are more likely to begin their search at a multi-journal Web site with links to full text, such as PubMed, Ovid, Science Direct, HighWire, or Medline, than at a specific journal. The journal title as a known entity is less likely to be sought than a specific article found through a database or subject search.

By analyzing transaction logs, eJUSt found that readers of electronic journals often search journal tables of contents, then go to the full-text article to read it briefly online in HTML, and then request a PDF file for archiving or printing. Medical searchers often come to the journal article from a PubMed search, then view articles in HTML and print them out in PDF. Multi-journal databases such as PubMed create major traffic for journal Web sites (30-60% of the searches by life scientists and M.D.s are in PubMed).

Unlike some of the other studies, the eJUSt studies found differences in journal use by the age of readers. Younger scholars report they are more likely to be frequent e-journal users than are older readers, and older scholars believe e-journals decrease the quality and rigor of research literature searches. The older the respondent, the more likely he was to report that unfriendly interfaces waste users’ time and older scholars report more trouble with interfaces. Older scholars are less likely to think e-journal usage increases scholarly productivity, but they are more willing to pay for online access than to go without an article.

Online access may actually motivate personal subscriptions and society memberships. Scholars with very few or very many subscriptions used e-journals more frequently than those with an average number (found to be four per scholar in this study).

The eJUSt participants also commented on what they like about paper journals. Just over half reported that portability is an advantage of printed journals since 80% print out a copy to read or file, or both. They like the better readability of paper, including the ability to make notes on the paper and maneuver easily between articles. Finally, they say that paper journals help users move within and between documents while reading intensely. When they are scanning articles on the computer screen, they like hyperlinking.

Participants rated the value-added features of e-journals, with hyperlinking rated as the most useful value added feature (63% like linking to scientific databases; 61% like linking to an author’s e-mail address; 52% like linking to an author’s Web sites; and 45% like linking to video-animated graphics).

Participants made suggestions for improving e-journals by listing what they desired in e-journals, including deep archives, greater clarity from libraries and publishers about what they offer, and e-journal design that addresses the more comprehensive needs and practices of the user. Furthermore, users report they need tools and services that support seamless navigation across different landscapes; they need help knowing what is linked, and they need more choices in subscription and membership packages.

The bottom line for journal use by these subject experts is convenience and versatility. Scholars integrate paper and electronic journals in a way that makes the most sense to them. They develop multiple ways of using e-journals to support a range of information practices. They monitor and review content regularly to keep current. Because they report that they read less intensively than in the past, they search using abstracts and metadata to help them evaluate material. The eJUSt reports conclude that scholars’ research habits have not changed-they read to extract knowledge and prefer to do it on paper. They circulate and exchange content to build peer networks, organize content by context and relevance into mini-libraries, and document original content to establish ownership of ideas.

2.4.4 Pew Studies/OCLC-Harris/Urban Libraries Council

The Pew Internet and American Life Studies used focus groups, observation, and journal-keeping to study the use of the Internet by middle, high school, and college students. The comparison of Internet and library use is the focus of this report. In the focus groups held between November 2001 and March 2002, the middle and high school students explained why they used the Internet for library and reference resources. They reported that the Internet is easier and more convenient to access from home and it is closer to home than the physical library, plus it is open 24 hours per day, seven days a week. Although the quality of Internet materials may be dubious, users say Internet materials are more current than library resources. Respondents believe that the Internet material covers a large variety of topics. It can be cut and pasted virtually rather than physically. They can print out the material at home. Furthermore, they can do all these things while baby-sitting, and do them more comfortably than in the physical library.

Participants in the Pew studies realize there are problems with the Internet. They know the information is not always true or understandable. They retrieve too much irrelevant information because they don’t know how to conduct good searches. Sometimes, specific online material is unavailable or must be purchased. The respondents complained about too many advertisements at some Web sites and the lack of foreign-language material online. In general, the more Internet-savvy students believed that they were better than their teachers at using it.

Almost three-quarters of college students (73%) said they used the Internet more than the library. On the other hand, only 9% use the library more than the Internet. Most students doing research use commercial search engines because it is easier to find resources; few used the library or university-based Web sites. Most students were observed using electronic resources rather than print resources and reported that they use the computer because it is convenient.

The Pew studies were preceded by two major studies that compared Internet use with library use. OCLC commissioned Harris Interactive to survey 1,050 college students between December 11, 2001, and January 1, 2002. The Urban Libraries Council, in conjunction with the State University of New York at Buffalo, conducted a telephone survey of 3,079 adults in March and April 2000. The OCLC/Harris survey included 11% graduate students and 89% undergraduate students in a variety of majors. Students are self-confident in their use of the Web for course-related research and in their ability to judge the quality of the Web sites.

According to the OCLC/Harris survey, three-quarters of the students feel that that they are successful at finding the information they need for courses and assignments, but unless professors or teaching assistants direct them to specific course-related Web sites, they make their own decisions about which Web sites to use (OCLC 2002).

More than two-thirds strongly feel they know best what information to accept from the Web and only 4% think the quality of the information is not good enough on the Web. Respondents rated the importance of various Web attributes and how they believed the Web measures up. The most important attributes were as follows:

Accuracy of information was rated as the most important (9.0 on a 10-point scale), but they don’t believe the Web delivers accurate information (6.2 on a 10 point scale).
Web doesn’t cost too much (8.9), and they rate it as (8.0).
Information is up to date (8.8), and they rate it as (6.8).
Web is easy to use (8.3), and they rate it as (8.5).

About 80% of the students sometimes use the library for Web access, but only 20% prefer to do so. More than 90% of the students access the Web outside the library, from their home computer, and 78% prefer this type of access. When it comes to needing help, however, 80% of the students prefer face-to-face rather than online help. Nearly half say they are more likely to get help online or by telephone, but 62% say they would use online help from a librarian if there were no charge. When students do use librarians for help, they rate that help 7.8 on a scale of 10—which is similar to the ratings for help from friends (7.8) and faculty or teaching assistants (7.9).

The students use a variety of Web resources for assignments, including search engines, Web portals, course-specific Web sites, and the campus library Web site. They learn about the library Web site from the following sources: professors and teaching assistants (49%), look it up themselves (45%), classes about using the library (34%), and librarians (27%). Only 21% say they ask a librarian for help, while 61% ask their friends, and 36% ask their professors or teaching assistants for help.

Eighty-nine percent of the students use print resources from the campus library at least part of the time, including books (75%), journals and periodicals (70%), journal articles (64%), and encyclopedias (34%). When they locate information they need, two-thirds of the respondents prefer to print out a copy for reading, rather than read from the screen.

Based on their experiences, students say they would like the campus library to do the following to help them with their assignments:

make it easier to use and access library information
make both print and electronic journals available
offer interactive maps, study guides, and resource guides
provide links to other library and research sites—over half want some way to search other libraries

The students report the following as barriers to online use:

inability to access databases remotely because of password requirements or license restrictions, or both
difficulty searching and navigating within a library Web site
costs of printing and copying at the library
shortage of knowledgeable librarians
lack of customer orientation

The Urban Libraries Council report used focus groups to decide what questions to ask to find out about the interaction between library and Internet use. The focus groups included both users and non-users of the library, as well as those who did and did not use the Internet. The final questionnaire was administered over the telephone to 3,097 participants between March and April 2000.

The study found that three-quarters of Internet users say they are library users and 60% of library users are Internet users. Use of the library and of the Internet are both inversely related to age, with library users and Internet users both significantly younger than non-users. Both use of the library and use of the Internet are also positively related to educational attainment and annual household income. There is no relationship with race, but females used the library and the Internet more than males.

The study concluded that there is no evidence that “use of the Internet is changing the reasons why people use the library,” nor “that length, frequency or recency of use of the Internet is affecting the frequency with which people use the library” (D’Elia et al. 2002).

Library users who do not have Internet access at home or at work use the library more than others to attend literacy classes and for children’s schoolwork. Significantly more people who have access to both the Internet and the library use the Internet for the following:

to do research for school
to obtain information for children’s schoolwork
to obtain local history or genealogy information
to browse Web with children for fun
to participate in and communicate through chat rooms or listservs
to obtain ethnic heritage information

Library service ratings were significantly greater than the Internet’s service ratings for ease of use, low cost, availability of paper copy (versus digital copy), accuracy of information, helpfulness of librarian (versus net help lines), and privacy. The Internet’s service ratings were significantly greater than the library’s service ratings, however, for ease of getting there, time to get there, hours of access, range of resources, expectation of finding what is sought, ability to act immediately on the information obtained, currency of the information, fun, enjoyment of browsing, and the ability to work alone (versus being among people at the library).

The Urban Libraries Council concludes there is no evidence that use of the Internet is a reason people do not use their library. People who use neither the library nor the Internet get their information through newspapers and television.

2.4.5 OhioLINK

OhioLINK is a consortium of 84 college and university libraries in Ohio. The academic institutions represented by these libraries range from Ohio State, with almost 50,000 students, to two- and four-year colleges with 360 students (Mount Carmel School of Nursing). OhioLINK mounts electronic journals on its own system, providing it with consistent transaction log information. The Electronic Journal Center uses log analysis to study usage levels and usage patterns across all OhioLINK libraries.

Perhaps the most dramatic finding from several years of log analysis is that e-journal users are reading from a wider array of journal titles than anticipated—much wider than the titles to which libraries previously subscribed. From April 1999 to March 2000, 40% of the journal titles accounted for 85% of the downloads; 45% of the least-used titles accounted for just 10% of the downloads, and 1% of the titles accounted for 8 to 10% of the downloads for each publisher. Between April 2000 and March 2001, of the 1,306,000 articles downloaded, 58% were from journals not held in print at the downloading patron’s library. For small colleges, 90-95% of articles downloaded were from newly accessible electronic journals in 2000. For two-year schools (both technical and community colleges), 95% to 100% of the articles downloaded were from newly accessible journals. Furthermore, each title has at least one download somewhere in the consortium. This leads to the seemingly obvious conclusion that adding new journal titles increases use.

OhioLINK analysis found that all member libraries have experienced growth in the annual number of downloads. The total number of annual downloads from April 1998 to March 2001 increased from 280,000 articles to 1,306,000 articles. More than half (51%) of the articles downloaded were from titles not held in print except by the large universities of Ohio State University, Case Western Reserve University, and the University of Cincinnati. Furthermore, when the licensing agreements are analyzed with the OhioLINK downloads for 1999, Dierdrichs (2001) reports that 120,000 articles were downloaded from titles not previously held at the patron’s library. Eighty-six thousand nine hundred ten of the 120,000 would have been over the fair use limit of five. In addition, online access to new titles increased the use of journals at all schools. For instance, small colleges in Ohio owned between 3 and 54 print journals, but they downloaded between 126 and 6,284 articles from 45 to 410 titles during the first year. Thus, Dierdrichs (2001) concludes that the OhioLINK consortium both saves money and provides faculty and students with a much larger array of journals that help them keep current.

Providing electronic access to journals increases use in a way that would likely not be matched by merely increasing the size of the print journal collection. OhioLINK’s director, Thomas Sanville, concludes that “only through immediate desktop delivery will users make use of journals at these expanded levels.” Sanville (2001a) believes that the “use of information is highly elastic as access is improved with the rapidly evolving advances in electronic technology.” This means that librarians’ attitudes should change from “I know what my users need” to “Let’s find out what my users need.”

Recently, the OhioLINK results have been challenged (or, at least questioned). Davis (2002) compared usage statistics of articles downloaded from a collection of more than 200 titles in the sciences and social sciences (Academic IDEAL e-journal package) at all the institutions in the NorthEast Research Libraries Consortium (NERL). He found that each institution within the consortium has a unique pattern of use; larger institutions used a wider range of journals, except for medical schools, which used a smaller number of titles than universities. He found that no institution uses every title and some titles are used very infrequently by all institutions. Furthermore, some of the small liberal arts colleges and technical institutes used only about 30% of the collection. Based on cluster analysis of journal usage, he suggests that institutions should form a consortium with like institutions (same size and type) to purchase journals. For instance, he suggests that medical schools may want to subscribe to a group of core journals and get the rest through interlibrary loan because they do not use all the journals that a research institution such as MIT uses. Davis believes that consortiums should be formed based on institutional characteristics rather than on geographic area.

2.4.6 Tenopir and King

From 1977 to the present, Tenopir and King have surveyed more than 16,000 scientists, engineers, and social scientists in both university and non-university settings. They found consistently over time that the amount of reading varies by work field and workplace. They also found that scientists, on average, read more journal articles than do engineers, and that medical faculty read the most. Chemists and physicists read between these extremes. Faculty members read more than professionals in non-university settings and write many more articles. Still, the amount of reading remains strong or is increasing in all work fields, and both scientists and engineers are reported to value journal articles highly. Most groups readily switch to electronic journals when they are convenient and are provided at no direct cost to them, but some of their readings still come from print sources.

Since the number of personal subscriptions is declining, scientists rely more on library-provided copies. Now they also rely more on reading articles from a variety of sources, including e-print servers, author or university Web sites, journal article databases, and personal e-mails.

Scientists, engineers, and medical faculty read primarily for research (34% of all readings in a survey of national laboratories; 29.9% in surveys of medical scientists), current awareness or continuing education (22% in both groups), and communications-related presentations or consulting (16% of the national laboratory scientists; 16.9% of the medical scientists). For non-university scientists, an important purpose of reading is for background research (24%). Additional purposes reported for the university medical scientists included clinical practice (7.8%) and teaching (16.9%). Most of the readings are articles that are less than two years old, but readings of older articles are reported to be very valuable. Currency is most important to the medical faculty. In the latest survey of medical faculty, more than 87% of the readings were from the past 14 months and 94% from the past two years. In a survey of astronomers, two-thirds of the total readings were from the last year and nearly three-quarters from the last two years. The oldest article read was more than 60 years old.

The average time spent reading has fluctuated and does not appear to be increasing as much as the number of readings. Scientists, on average, are doing quite a bit more reading without spending a great deal more time. The time spent reading per article also varies among disciplines, with engineers averaging 72 readings per year (but 80 minutes per article) and medical faculty averaging 322 readings per year (but only 20 minutes per reading). Although the average amount of reading and time spent reading vary somewhat, scientists continue to show the value they place on journal articles by the time they spend reading.

Recent counts of percentages of reading that come from electronic journals varies in the studies, from a high of about 80% of all readings by astronomers to a low of 35% from science and social science faculty at one university. Habits are changing and most students and faculty prefer e-journals when they make access easier, save the reader’s time, and are known within the specific scholarly discipline.

2.4.7 LibQUAL+^TM

LibQUAL+^TM is an electronic survey administered by college, university, and health science libraries (both ARL and non-ARL institutions) that began in the spring of 2000. To date, more than 70,000 students, faculty members, and staff have responded. The percentage of respondents who use the electronic library at least weekly ranges from a low of 30% at community colleges to a high of 66% at health sciences institutions. The percentage of the same respondents who use the physical library at least weekly ranges from a low of 36% at community colleges to a high of 55% at four-year colleges (both ARL and non-ARL institutions). At ARL four-year colleges, 75% of both faculty members and graduate students use the electronic library at least weekly, while only 47% of the faculty members and 64% of the graduate students visit the physical library. On the other hand, a greater percentage of undergraduates use the physical library (53%) than the electronic library (44%) at least weekly. At ARL health science institutions, 80% of the faculty members use the electronic library at least weekly, while only 37% use the physical library. More graduate students use the electronic library (75%) than the physical library (63%) weekly, while the undergraduates use both about equally (59% and 61%). At OhioLINK community colleges, 31% of the faculty members use the electronic library at least weekly, while 69% of the faculty members at four-year OhioLINK institutions do so.

The percentage of respondents who never use the physical or the electronic library varies. The smallest percentage of respondents who never use the physical library are undergraduates at ARL health science institutions (0.6%), while 7.2% of the same undergraduates say they never use the electronic library. On the other hand, the smallest percentage of respondents who never use the electronic library are faculty at four-year ARL institutions (3.4%), while 1.9% of them say they never use the physical library. About 20% of respondents from community colleges never use the electronic library, while only 4% never use the physical library.

LibQUAL+ respondents were also asked to rate their desired level of service, their minimum acceptable level of service, and the level of service they perceive that their institution provides. The resulting gap is the perceived level of service minus the minimally acceptable level of service. Again, the results depend on the status of the respondent and the institution where respondents worked. Graduate students and faculty members at four-year ARL institutions, and faculty at ARL health sciences institutions did not believe that their libraries have the complete run of journals they deem minimally acceptable. Faculty members at OhioLINK community colleges, four-year OhioLINK institutions, four-year ARL institutions, and health science institutions all believed that the print collection was not minimally acceptable. Faculty members at four-year ARL institutions, and undergraduates and faculty members at ARL health science institutions do not believe that they can access electronic resources from their home or office at the level they find minimally acceptable.

In identifying the minimally acceptable level of service, all respondents at all institutions (except health sciences) ranked as the most important feature a library Web site that enables respondents to locate information on their own. Respondents in the health sciences ranked the library Web site and the ability to access electronic resources from home or office almost equally as the most important features at a minimum level of service.

Making electronic resources accessible from the home or office either tied with the library Web site or was second to it for faculty members and students in ARL institutions. The most desired attribute for community college respondents was having modern equipment that affords easy access to the needed information. Undergraduates at four-year ARL institutions believe that, at a minimally acceptable level of service, modern equipment and easy-to-use access tools are more important than making electronic resources accessible from home or office.

The attributes rated as least important are the complete run of journal titles and the print collection, but those ratings are based on the status of the respondent. Faculty members and graduate students at ARL institutions (both four-year and health sciences) believe the complete run of journal titles is more important than the print collection. Undergraduates in the health sciences institutions would accept an equally minimal level of service for both the print collection and electronic journals, but the electronic journals appeared to be slightly more important to them than the print collection (8.0 and 7.90, respectively). However, for the community college respondents, both the minimum (6.56) and desired (7.24) level of the print collection was more important than the complete run of journal titles (6.19 minimum level and 7.24 desired level).

The bottom line is that respondents, especially graduate students, faculty, and staff, believe that making information easily accessible, either through the library Web site or through tools that allow people to find things independently, is very important.

When OhioLINK members administered the LibQUAL+^TM questionnaire to their constituents, they added questions about electronic resources. Faculty members and graduate students from OhioLINK institutions, including four-year institutions and community colleges, indicated that the availability of online help when using their library’s electronic resources did not meet their minimum expectations. The comprehensive collection of full-text articles online did not meet the minimum acceptable level for either students or faculty at OhioLINK’s four-year institutions, or for faculty at the community colleges.

At the minimum level of service, OhioLINK students ranked the importance of various service attributes as follows:

ease of using library’s online article indexes
comprehensiveness of collection of full-text articles online, and convenience of borrowing books from other colleges (tie)
availability of online help when using their library’s electronic resources
informing them about useful library services

Faculty, by contrast, ranked the convenience of borrowing books from other colleges as most important, followed by the ease of using the library’s online article indexes. The comprehensive collections of full-text articles online and the availability of online help tied at the minimum level of service. At the desired level of service, the students most wanted a comprehensive collection of full-text articles online, while faculty thought that the convenience of borrowing books from other colleges and the ease of using the library’s online article indexes were more important, in terms of desired service, than the comprehensive collection of online journals, the availability of online help, or information about useful library services.

2.4.8 JSTOR

The JSTOR retrospective journal collection began as a project sponsored by The Andrew W. Mellon Foundation at the University of Michigan to help researchers determine whether the digitized versions of older research journals might serve as a substitute for the paper version. In 1999, JSTOR conducted a survey of print journals in which librarians counted the volumes not on the shelf to compare print journal use with the use of the electronic JSTOR journals. By comparing paper usage with JSTOR data logs at the same libraries, it was found that electronic access increased the usage of older material because it increased convenience.

Use of JSTOR in subscribing libraries (and the number of titles in JSTOR) grows each year. When logs were analyzed for a one-week period, over 90% of the searches included more than one journal title and about 85% of the searches were in pre-defined discipline-specific collection clusters. Of the cluster searches, 69% were for more than one cluster.

Of all the articles in JSTOR, more than half (51.8%) have been viewed, and 29.9% have been printed. JSTOR research shows that older literature remains important in many fields, but its importance varies among fields. For the most highly used articles in economics and mathematics, there is essentially no correlation between the age of the article and usage. However, for history, the newer articles are more apt to be printed.

The percentage of viewings represented by the top ten views for a given cluster varies between clusters. For instance, the top ten articles viewed account for 22% of the articles viewed in economics, but the top ten articles viewed account for only 1% in Asian studies. Guthrie (2002) suggests that the articles used the most might be the classic articles used in large classes.

In the fall of 2000, JSTOR surveyed faculty members at institutions of higher education in the United States and received more than 4,000 responses from humanists, social scientists, and economists. Most of the respondents (more than 60%) greatly value electronic journals. They reported they are comfortable using electronic resources, believe a variety of electronic resources are important to their research, and consider electronic databases invaluable. Faculty members report that they use online catalogs, full-text electronic journal databases, and abstracting and indexing databases the most, and they expect they will use them more extensively in the future.

When faculty members were asked about their dependence on the library, 48% said they are very dependent now but only 38% expect to be very dependent in five years. The following statement sums up how 44% of the respondents feel: “Before long, computers, the Internet, and electronic computer-based archives and databases will allow academics to conduct much of their research without setting foot in the library.” This attitude varied by field; 52% of the economists shared this view, but only 22% of the humanists did (Guthrie 2002).

Faculty members considered the following functions of the library very important, although their responses vary according to discipline:

a gateway or starting point for their researchÐ65% (80% of humanists and 48% of economists)
a trusted repository or archive of resourcesÐ77%
a buyer of resourcesÐ80%

Almost half of the respondents (48%) agreed with the following statement “Regardless of what happens with electronic archives of journals, it will always be crucial for libraries to maintain hard-copy archives.” The agreement with this statement varied according to discipline, with about one quarter of the economists but 63% of the humanists agreeing.

Fifty-six percent of the respondents did not agree with the following statement: “Assuming that electronic archives of journals are proven to work well and are readily accessible, I would be happy to see hard-copy archives discarded and replaced entirely by electronic archives.” Again, the degree of disagreement with the previous statement depended on the discipline of the respondent. Thirty-five percent of economists disagreed with the statement, and nearly three-quarters of the humanists reacted very negatively to the statement.

Guthrie (2002) interprets this to mean “that faculty do not believe that a reliable solution for electronic preservation is in place and that the hard copy is needed for backup. Furthermore, many are worried about electronic archives.” Three-quarters (76%) of faculty members, regardless of discipline, said the following statement described their point of view: “With more and more journals becoming available electronically, it is crucial that libraries, publishers, or electronic databases archive, catalog, and protect these electronic journals.”

In another study, Seeds (2002) surveyed JSTOR use at Penn State and found that as JSTOR use increased, paper journal use decreased. More important, the use of the electronic journals increased for the four journals evaluated. He concludes that it appears as if electronic access is a viable substitute for print when the print titles are remotely stored.

FOOTNOTE

¹ See Urquhart et al. (2003) for a description of the critical incident technique in information research.