next section in this report >> | previous section >> | report contents >>
Chapter 1
gives background on the MIRACLE Project, defines institutional repositories
(IRs), and describes the methods MIRACLE Project staff used to conduct a census
of IRs in U.S. academic institutions.
1.1 The Impetus for the MIRACLE Project's Census of IRs in the United States
A
considerable portion of the scholarly record is born digital, and some
scholarship is produced in digital formats that have no physical, in-the-hand
counterparts. The proliferation of digital scholarship raises serious and
pressing issues about how to organize, access, and preserve it in perpetuity.
The response of U.S. colleges and universities has been to build IRs to
capture, preserve, and reuse the intellectual output of teaching, research, and
service activities at their institutions. An IR is "a
set of services that a university offers to the members of its community for
the management and dissemination of digital materials created by the
institution and its community members" (Lynch 2003) (see also Appendix F1).
The MIRACLE (Making
Institutional Repositories a Collaborative Learning
Environment) Project is investigating the implementation of IRs at
academic institutions to identify models and best practices for the
administration, technical infrastructure, and access to digital collections.
The chief objective of the project is to identify specific factors contributing
to the success of IRs and effective ways of accessing and using IRs. The census
is the first of several activities aimed at achieving project objectives. Other
activities will study IR users, contributors, and staff through the use of
telephone interviews, case studies, personal interviews, observations, and
experiments.
Originally,
MIRACLE Project investigators proposed to survey operational IRs in North America; however,
we were concerned that we would be duplicating the efforts of Charles Bailey
and his University of Houston associates who were analyzing data from their
Association of Research Libraries (ARL)-sponsored survey of member institutions
at the same time we were making data-collection decisions for the MIRACLE
survey (Bailey et al. 2006). Other surveys targeted specific user groups such
as Coalition for Networked Information (CNI) members in the United States
(Lynch and Lippincott 2005), CNI members abroad (van Westrienen and Lynch
2005), Canadian Association of Research Libraries (CARL)-member libraries
(Shearer 2004), and early adopters of IR technology worldwide (Mark Ware
Consulting 2004).
Examining
these surveys' results, MIRACLE project investigators decided not to limit
their efforts to a particular user group, membership, or affiliation, and not
to restrict participation to institutions with an operational IR. Instead, we
sought to cast our net broadly and fill a void. Conducting a census of academic institutions in the
United States about their involvement with IRs, MIRACLE Project investigators
decided not to exclude institutions that have not jumped on the IR bandwagon.
Being more inclusive would increase our confidence that we would be able to
identify the wide range of practices, policies, and operations in effect at
institutions where decision makers are contemplating, planning, pilot testing,
or implementing IRs. At the same time, it would enable us to learn why some
institutions have ruled out IRs entirely.
1.2 Obtaining
a Mailing List of Academic Library Directors
The first
task of MIRACLE Project staff was to obtain an electronic mailing list bearing
the names and e-mail addresses of academic library directors and senior library
administrators at U.S. educational institutions. A number of companies provide
this information for a fee (for example, see American Library Association
2006). After examining their products and services, MIRACLE Project staff narrowed
options to the following four companies or products: (1) Thomson-Peterson's,
(2) Market Data Retrieval, (3) American Library Directory Online, and (4) World
Guide to Libraries Plus. After comparing these companies' products with respect
to such variables as the number of records with e-mail addresses available,
scope, and price, as well as other advantages and disadvantages, we decided to
purchase mailing lists from two vendors: (1) American Library Directory (ALD)
and (2) Thomson-Peterson's. Using ALD's online database, we downloaded a
comprehensive list (2,207 records) of all college and university main libraries
in the United States (including U.S. protectorates). Because ALD's online
database did not provide e-mail addresses for specific individuals, we
purchased a less comprehensive database from Thompson-Peterson's that we used
to add e-mail addresses to ALD data. After deleting community colleges and
duplicates, we ended up with 2,147 e-mail addresses for the nationwide census.
1.3 Conducting
Comparative Analysis of Survey Software
To compare
survey-software programs for administering our Web-based survey, MIRACLE
Project staff signed up for free trials of 10 such software programs:
SurveyMonkey, Zoomerang, Key Survey, SurveyConsole, EZQuestionnaire, iSalient,
QuestionPro, Ridgecrest Surveys, SmartSurveys, and SuperSurvey. Staff also
researched Flashlight Online, ScyWeb, and UM.Lessons. On the basis of pricing
information, flexibility, and functionality, we narrowed the list to
SurveyMonkey, Zoomerang, Key Survey, and UM.Lessons. Staff eliminated
UM.Lessons and Key Survey from consideration because of the former's limited
flexibility and functionality and the latter's cost.
MIRACLE
Project staff's decision to use SurveyMonkey instead of Zoomerang was based on
the former program's greater flexibility and functionality. Our purchase of a
one-year professional subscription to SurveyMonkey would enable us to launch an
unlimited number of surveys with an unlimited number of questions and to use
its advanced features for the survey's many complicated questions.
1.4 Drafting
and Pretesting Survey Instruments
To draft
survey instruments, MIRACLE Project investigators reviewed published and
open-access literature on IRs through 2005 (see the MIRACLE Project's bibliography
for a list of relevant publications at http://miracle.si.umich.edu/bibliography.html), talked to colleagues, and
asked advisory group members (see Appendix A) to review, comment on, and edit
draft instruments. Because the investigators expected survey respondents to
come from institutions that were at various stages of the IR effort, they could
neither ask everyone the same questions nor ask questions in the same way.
Advice from advisory group members resulted in these four categories of IR
involvement: (1) no planning to date (NP), (2) planning only to date (PO), (3)
planning and pilot testing one or more IR systems (PPT), and (4) public
implementation of an IR system at the respondent's institution (IMP). MIRACLE
Project investigators drafted four different questionnaires based on these four
categories of IR involvement.
Asking the
same or similar questions in two or more questionnaires would enable
investigators to make comparisons among institutions on the basis of the extent
of their involvement with IRs. For example, here is a question about
anticipated benefits of IRs that is worded a little differently depending on an
institution's involvement with IRs:
• For
NP respondents: How important do you think these anticipated benefits of IR
would be to your institution?
• For
PPT and PO respondents: How important are these anticipated benefits of IR to
your institution?
• For
IMP respondents: At the beginning of IR planning at your institution, how
important did you think these anticipated benefits of IR would be to your
institution?
Appendixes B,
C, D, and E contain the MIRACLE Project questionnaires for NPs, POs, PPTs, and
IMPs, respectively.
1.5 Setting
Up Survey Administration Procedures and Protocol
Having so
many institutions (2,147) in the census sample required MIRACLE Project staff
to work out a detailed distribution plan. After pretesting a few different
approaches, we decided to send an e-mail message to each institution's academic
library director or a senior administrator to tell them about the census and to
ask them about the extent of their involvement with IRs. More specifically, we
wrote, "Please tell me how you would characterize the current status of your
institutional repository (IR)." We asked them to base their response on one of
four categories: (1) no planning to date, (2) planning only to date, (3) both
planning and pilot testing one or more IR systems, and (4) public
implementation of an IR system at their institution.
On the basis
of the person's response, we replied with an e-mail message bearing a link to
the appropriate Web-administered questionnaire (see Appendixes B, C, D, and E
for NP, PO, PPT, and IMP questionnaires, respectively). We used SurveyMonkey's list-management
tool to send out initial survey links and to perform two subsequent follow-ups
with individuals who had agreed to participate but who had failed to respond to
our inquiries.
Recruiting
people to participate in the MIRACLE census in this way is the electronic
version of what those in the sales world term a "cold call." We sent
prospective respondents e-mail messages with a substantive phrase in the
"SUBJECT" line announcing our IR census and asked them to participate. It is
likely that the people who responded to our e-mail message were interested in
IRs and thus were more likely to open, read, and respond to such a message and
eventually respond positively about IRs on their completed questionnaires.
Thus, MIRACLE census respondents may be more favorably inclined toward IRs than
other academic library directors and senior administrators generally because of
how we recruited them.
1.6 Data
Collection
MIRACLE
Project staff conducted the nationwide IR census from April 19, 2006, through
June 24, 2006. Data collection was not straightforward. When few respondents
responded to our invitations and reminders, we discussed problems and
brainstormed ways of solving them. For example, coprincipal investigator
Elizabeth Yakel suggested replacing the original SUBJECT line in our e-mail
messages, "IMLS Institutional Repositories Census," with the catchier phrase,
"Be Counted! National Institutional Repository Census." This change did indeed
result in a higher response rate.
Table 1.1
summarizes the six data collection rounds that were necessary to increase the
survey's invitational response rate to an acceptable level.
Table 1.1 Data collection rounds
|
|
|
Cumulative |
Cumulative |
|
||
|
No. |
% |
No. |
% |
|||
|
2,147 |
4/19 to 4/26 |
172 |
9 |
89 |
5 |
Invitations sent through Rieh's e-mail account. Staff research 260
undeliverable messages. |
|
1,698 |
5/2 to 5/14 |
320 |
15 |
169 |
8 |
Invitations sent through Markey's e-mail account. Staff continue to
research undeliverable messages. |
|
1,805 |
5/15 to 5/22 |
467 |
22 |
273 |
13 |
Invitations sent through Markey's e-mail account. Staff change
SUBJECT line and invitation text. |
|
1,619 |
5/23 to 5/30 |
566 |
27 |
370 |
18 |
Invitations sent through Markey's e-mail account. |
|
1,511 |
5/31 to 6/7 |
627 |
30 |
420 |
20 |
Invitations sent through Yakel's e-mail account. |
|
1,446 |
6/8 to 6/24 |
676 |
32 |
500 |
24 |
Yakel's account. Staff change SUBJECT line announcing end of
census. Seven undeliverable messages. |
|
*Total number of people who responded to our invitation stating
that they were willing to participate in the MIRACLE Project census. †Total number of people who clicked on the SurveyMonkey link that
MIRACLE Project staff sent to them in response to our invitation. Generally,
these figures indicate how many people actually participated in the survey.
Because some people who clicked on the link exited the survey without
answering any questions, these percentages are inflated. After MIRACLE
Project staff had removed empty and nearly empty response sets, deleted
duplicates, etc., the census response rate was 20.8%. |
||||||
Concurrent
with sending e-mail invitations, MIRACLE Project staff e-mailed a link to the
appropriate Web-administered questionnaire to respondents within three business
days of their response to our invitation. When respondents failed to return the
completed questionnaires, staff sent them up to two reminders. The text of
these two e-mail responses (the first survey link e-mail and the reminder
e-mail) remained fairly stable throughout the census. Staff took care to send
e-mail correspondence from the same account (Rieh, Markey, or Yakel), matching
the account to which each respondent had initially responded.
A large
number of people who had agreed to participate in the census failed to follow
through. To rectify this situation, MIRACLE Project staff drafted two e-mail
messages—one for respondents who had not yet started filling out the
questionnaire and a second for respondents who had answered some questions. The
SUBJECT line of both e-mail messages was "Survey to Close 6/24 (Nationwide
Census of Institutional Repositories)." In mid-June, staff sent these e-mails
to selected respondents. Because these e-mail messages encouraged a number of
respondents to complete questionnaires, staff sent a second message to those
who had still not responded and changed the SUBJECT line to "5 Days Left: Last
Chance to be Counted in Nationwide Census of Institutional Repositories." Quite
a few people filled out questionnaires after receiving the second message. When
MIRACLE Project staff closed questionnaire administration in SurveyMonkey at 8
a.m. on June 25, 2006, the invitation response rate was 32%.
1.7 Data
Analysis
After closing
the census in SurveyMonkey, MIRACLE Project staff exported census data from
SurveyMonkey into four Microsoft Excel files (one for each version of the
survey—NP, PO, PPT, and IMP). Staff cleaned up census data, deleting the
responses of people who did not sign the informed consent form as well as those
of people who responded only to the informed consent form or to the one
question about the number of IRs at their institution. Staff deleted empty
questionnaires. They deleted multiple answer sets, keeping only the most
comprehensive response sets from respondents. Staff deleted one entry that was
submitted from a two-year college. This college had been sent an invitation
because of an error in one of the mailing lists that we had purchased. After
data cleanup had been completed, the census response rate was 20.8%.
MIRACLE
Project staff imported the cleaned-up census data into SPSS and calculated
frequency tables for the responses to each question in each of the four survey
versions. Using these SPSS calculations, staff created an Excel spreadsheet
that depicted frequency tables side-by-side for each question across the four
questionnaire versions. Staff also produced a Word document that shows
respondents' answers to open-ended questions.
MIRACLE
Project staff used related data files to probe research questions in greater
depth. For example, they downloaded a file from the Carnegie Foundation's Web
site that allowed them to determine whether census participants were
representative of educational institutions in the United States (see Subchapter
2.2) (Carnegie Foundation 2006b).
1.8 Chapter
1 Summary
Institutional
repositories are the response of U.S. colleges and universities to the problem
of organizing, providing access to, and preserving scholarship that their
learning communities produce in digital formats.
Originally,
MIRACLE Project investigators proposed to survey operational IRs in North America; however,
we were concerned that we would be duplicating previous surveys that targeted
institutions with operational IRs. We decided to cast our net broadly and to
conduct a census of American academic institutions about their involvement with IRs.
Census results would fill a void—yielding data and analyses about educational
institutions that are and are not involved with IRs.
MIRACLE
Project staff purchased mailing lists from two vendors: (1) ALD, and (2)
Thomson-Peterson's. After deleting community colleges and duplicates, we ended
with a total of 2,147 e-mail addresses for the nationwide census.
Staff
pilot-tested several Web-administered software programs and chose SurveyMonkey
because of its flexibility and functionality for the complex questions in
MIRACLE questionnaires.
Project
investigators drafted questionnaires and received feedback from advisory group
members regarding questions and response categories. On the basis of their
input, staff developed four separate questionnaires based on respondents'
extent of involvement with IRs: (1) no planning (NP), (2) planning only (PO),
(3) planning and pilot testing (PPT), and (4) implementation (IMP). (See
Appendixes B, C, D, and E for NP, PO, PPT, and IMP questionnaires,
respectively.)
Data
collection took place from April 19, 2006, through June 24, 2006. MIRACLE
Project staff sent invitations to participate in the census via e-mail to each
institution's academic library director or a senior administrator. The e-mail
explained the census and asked them about the extent of their involvement with
IRs. We replied via e-mail to those who responded to our request with a link to
the appropriate Web-administered questionnaire.
Low response
rates to our invitation resulted in changes in the text of our reminder
messages, especially the wording of the message's SUBJECT line. After data
collection ended on June 24, 2006, MIRACLE Project staff cleaned up census
data, for example, deleting empty questionnaires or responses of people who did
not sign the informed consent form. After data cleanup had been done, the
census response rate was 20.8%. MIRACLE Project staff then proceeded with data
analysis and reporting activities.
2 THE
INSTITUTIONS AND THE PEOPLE INVOLVED WITH IRs
Chapter 2 examines the extent to which certain
types of academic institutions are involved with institutional repositories
(IRs) and the nature of people's involvement with IRs at these institutions.
2.1 Census
Respondents
Of the 2,147 academic library directors and senior
library administrators MIRACLE Project staff contacted, 446 participated in the
census—a response rate of 20.8%. Characterizing the extent of their involvement
with IRs, 236 (52.9%) respondents have done no IR planning (NP) to date, 92
(20.6%) respondents are only planning (PO) for IRs, 70 (15.7%) respondents are
actively planning and pilot testing IRs (PPT), and 48 (10.8%) respondents have
implemented (IMP) an operational IR. Figure 2.1 is a graphical representation
of the extent of IR involvement by MIRACLE Project census respondents.
When MIRACLE Project staff contacted library
directors and senior library administrators by e-mail, we asked them to pass
our questionnaire to staff who were most familiar with their institution's
involvement with IRs. The questionnaires concluded by asking respondents to
identify their positions at their institution. Figure 2.2 shows the titles of
those who completed questionnaires.
Almost three-quarters of respondents are library
directors; the second- and third-largest percentages (10.2% and 7.9%,
respectively) are library staff and assistant-associate librarians,
respectively. Library directors prevail in terms of responding to the MIRACLE
Project staff's request to participate in the census. We deliberately chose to
make library directors or senior library administrators the initial contact at
academic institutions because of the difficulty identifying the names of the
key person(s) involved with IRs at academic institutions and finding address
lists to simplify and streamline contacting tasks. For example, we could have
contacted chief information officers (CIOs) instead of librarians but academic
institutions do not necessarily apply the CIO moniker across the board nor do
all institutions have such a position. The same thing probably applies to
archivists. Even more complicated would have been contacting middle management
in academic institutions—deans, directors, chairs, and heads of academic units,
research centers, and institutes. Because every academic institution is likely
to employ a librarian, we contacted librarians in top management positions to
participate in our census.
Our decision to contact librarians may have caused us to
miss academic, research, and service units that have implemented or are
planning to implement an IR. To some extent, respondents' answers to a census
question about how many IRs are available at their institutions may shed light
on what we missed (see Chapter 5.1 for answers to this question). MIRACLE
Project investigators readily admit that census results may be biased toward
libraries because our initial contact was the college or university librarian.
Table 2.1 shows a breakdown of census respondents
based on the extent of their institutions' involvement with IRs. At NP
institutions, about 90% of respondents are library directors. Percentages in
other named-position categories are very small. Of the four people classed in
"Other," three are combined library directors-CIOs, and one is head of digital
library programs.
Table 2.1. Respondents' positions
based on the extent
of IR involvement at their
institutions
|
Respondent position |
NP |
PO |
PPT |
IMP |
Total |
|||||
|
No. |
% |
No. |
% |
No. |
% |
No. |
% |
No. |
% |
|
|
Library director |
194 |
90.6 |
57 |
71.3 |
29 |
48.3 |
8 |
21.6 |
288 |
73.7 |
|
Library staff |
5 |
2.3 |
11 |
13.8 |
8 |
13.3 |
16 |
43.3 |
40 |
10.2 |
|
Assistant or associate library
director |
5 |
2.3 |
0 |
0.0 |
16 |
26.7 |
10 |
27.0 |
31 |
7.9 |
|
Archivist |
4 |
1.9 |
3 |
3.7 |
2 |
3.3 |
0 |
0.0 |
9 |
2.3 |
|
CIO |
1 |
0.5 |
5 |
6.2 |
1 |
1.7 |
1 |
2.7 |
8 |
2.0 |
|
VP or provost |
1 |
0.5 |
0 |
0.0 |
||||||