Search
Close this search box.
Search
Close this search box.

Appendix II : Assessing Institutional Web Sites

 

Prepared by John Chadwick, John H. Falk, and Brigitte O’Ryan for the Council on Library and Information Resources


The Council on Library and Information Resources engaged the Institute for Learning Innovation to conduct a pilot study on museum and library Web sites. The purpose of the research was to begin to understand why and for whom institutions develop Web sites and to gain a better understanding of the needs of online users. The Institute for Learning Innovation developed the following research questions:

  • Why are institutions building Web sites?
  • Are the goals and objectives clearly stated and written down?
  • Do all parties within an institution share the same goals and objectives for the site?
  • Who does each institution assume uses its Web site?
  • Who actually uses the sites?
  • Why does the institution think Web users visit the site?
  • Why do actual users visit the site and how do they find it?
  • What do developers think site users take away from the experience?
  • What do site visitors actually take away from the experience?

Methods

The Council on Library and Information Resources selected three libraries and three museums for this study, based on their participation in the conference Collections, Content, and the Web, held October 5-7, 1999, at the Chicago Historical Society. The six institutions were Yale University Library, Cornell University Library, New York Public Library, the Chicago Historical Society, the Art Institute of Chicago, and the Frick Collection.

Three study methods were used:

  1. site visits were made to the participating institutions and interviews with held with key personnel,
  2. a survey was sent to the participating institutions to solicit feedback from key administrators about the creation and management of their Web sites, and
  3. an online user survey was linked to each institution’s Web site, with results relayed directly to ILI’s Web site.

Results and Discussion

Two types of organizational structures were represented in this study: (1) academic libraries, which use a decentralized computing system that is consistent with the organization of their campus libraries, and (2) public institutions, including the New York Public Library and the three museums, which use a centralized computing system. The libraries at Yale University are linked with individual academic units at the university. This traditional affiliation is reflected in Yale’s Web site, which makes available online more than 40 libraries and special collections. The Cornell University Library system comprises more than 20 libraries and special collections. The New York Public Library, the largest public library system in the world, has a centralized management system for its Web-based projects. The Chicago Historical Society, the Art Institute of Chicago, and the Frick Collection are private institutions with facilities of varying sizes. Their collections also vary greatly in size and scope. Among the three museums, the departments responsible for managing online resources differ. It is important to bear in mind these differences when considering the results of this research project.

Why are institutions building Web sites?

All six institutions indicated that the definition of success for collections-based institutions is shifting from the size of collections to the services they offer. All the institutions view the Web as critical in redefining institutional success and helping to meet the new definition of success. Many of the institutions also said the Web would help increase their visibility. Some museums hoped the Web would increase visits to the museums.

Are the goals and objectives clearly stated and written down?

All the institutions participating in this research project indicated that they developed goals and objectives for their Web sites; however, only four sites had a set of written goals that had been distributed to staff members. Many respondents indicated that the goals had been shared informally through e-mail messages and face-to-face meetings. Within an institution, responses seemed to reflect confusion over what may or may not be goals and objectives for the Web site. For example, some respondents at one site indicated that there were indeed goals, while another respondent from the same institution stated that there were no written goals. The academic libraries have a much more difficult challenge in developing goals because so many libraries and special collections are closely linked to academic units. What may be appropriate goals for the main library may not be so for a special collection. Ultimately, each institution will need to develop goals and objectives that address its unique situation.

Do all parties within an institution share the same goals and objectives for the site?

If an organization has written goals and objectives, one might assume that everyone in the organization would know about and share them. The research results indicate that this assumption is not valid. The survey data show that, even when goals and objectives exist, not all organizations are communicating them. Organizations should be aware of the need to communicate larger goals and objectives to the staff involved in developing and maintaining online resources. If the goals and objectives of a Web site are not clear, the site may not be properly organized, and the online visitor may not be able to fully use the resources of the site.

Who does each institution assume uses its Web site?

Museums most often indicated the general public as their primary audience, while academic libraries stated they developed their Web sites to reach students, faculty, and staff. It is important to understand that users of both museum and library Web sites cannot always be classified into neat groups such as students, faculty, and staff. The Web has given these organizations an audience beyond the walls and traditional users of their institutions. For example, Cornell University maintains the United States Department of Agriculture (USDA) Economics and Statistics database. The reports of this database are intended to reach a global audience. Although Cornell expected that the Web site would serve primarily faculty, students, and staff, the concept of Cornell’s audience changed after the Web site was established. The largest growing community of Web site users consists of practitioners in the field.

Who actually uses the sites?

More women (62 percent) than men (38 percent) completed the online survey. The mean age of participants was 39.4 years. The mean age for men completing the survey was slightly higher than that for women. The youngest person to complete a survey was 12 years old; the oldest was 86 years old. The mean age of those who completed a survey at a museum Web site was slightly higher than that of respondents who completed the survey at a library site.

Almost half of the respondents said the Web site would have no impact on how often they visited the physical site. About 33 percent of the respondents indicated that the Web site influenced them to visit the physical location more often, and 20 percent said the Web site influenced them to visit less often. Most of the respondents who claimed they would visit the institution more often were museum site visitors, while most who indicated they would visit the institution less often were academic library site visitors. The Web master for one university’s medical school library echoed this finding, noting that the use of the Web has increased while the number of people coming to the medical school library has decreased. This is because the Web has increased the library’s ability to provide time-critical information and research to its intended audience.

Although two out of five respondents from the academic libraries indicated that they were visiting the physical site less often, an administrator at one site said that the availability of full-text journals online has actually increased the number of people coming to the library and asking for a hard copy. Another respondent noted that the Web site enabled her to do research even when the library is closed, and the time she does spend in the library is now much more efficient and productive.

Respondents to the user survey also indicated they visited the Web site quite frequently, with 55.4 percent of online visitors to museum Web sites and 70 percent of library users visiting Web sites at least once a month. None of the museum respondents indicated that they were first-time visitors to the Web site while libraries had a substantial number of first-time users.

See tables 1-3 and figure 2.

Why does the institution think Web users visit the site?

Although both libraries and museums cited information as a primary reason people visit their Web sites, the types of information and the scope of the information varied as a function of the mission and audience of the institutions. Most library administrators said that Web users visit their sites for convenient access to information about resources and collections. Several administrators also noted that users were motivated to use the Web site because of its convenience in providing electronic access to the institution 24 hours a day, seven days a week. Some museum administrators also listed education and entertainment as reasons for using the site.

Why do actual users visit the site and how do they find it?

People visit libraries for different reasons than they visit museums; therefore, it is not surprising that the visitors to library and museum Web sites have different agendas. Virtually half of the online visitors to the libraries indicated that they were visiting the site to conduct academic research; more than a third indicated that they were visiting to conduct personal research. Less than 15 percent visited for personal growth or to plan a visit to the library. Conversely, for museum site users, the reason most often cited for the visit was to seek information required for planning a visit to the physical site; this was followed closely by personal research and personal growth. Academic research was cited as a reason by only a fifth of the respondents visiting museum sites, even though the three museums in the study have strong research libraries associated with their institutions. None of the museum administrators mentioned planning an actual visit as a reason for a visit to the Web site.

Visitors to a museum site were more likely to find the site with a search engine, while visitors to a library site were more likely to enter the site either by typing the site address or through the default Web page on the browser. The lowest percentage of online visitors found the site through links at other sites. Museums and tourism agencies often provide links to museum Web sites. These links are critical to maintain in order to reach a broad audience, but they accounted for the lowest percentage of the total online visitors.

More important, cultural institutions should come to view digital conversion as a means to other things, not an end in itself. Susan Yoder, director of Integrated Information Services at RLG, has suggested that digitization efforts will be sustainable if they are justified by at least one other institutional goal beyond generating revenue (Yoder in press). For the foreseeable future, the digitization of retrospective collections will not pay for itself, but it may be a legitimate loss leader in a new service paradigm, enabling libraries and museums to compete successfully in reaching a broad range of cultural consumers.

See figures 3-7 and tables 4, 6, 7.

What do developers think site users take away from the experience?

Most of the Web administrators were vague in answering this question. Generally, administrators hoped that visitors were able to find the information they were looking for when they came to the sites, such as information about the collections and resources, and about the institution. Some museum administrators said they hoped Web users had an educational experience and were entertained.

What do site users actually take away from the experience?

More than 90 percent of the respondents rated the information on the library and museum Web sites as either “very reliable” or “somewhat reliable,” and they return to sites regularly. Ease of access to information was also rated highly among respondents from the library Web sites. Overall satisfaction with the Web sites was quite high; 84 percent of the respondents were either “somewhat satisfied” or “satisfied” with their online experience.

See figures 4, 8, 9, 10, 11 and tables 5, 8.

Conclusion

This pilot study has provided some insights into the perceptions of those who develop library and museum Web sites and those who use those sites. We have also created a model that can be used by other institutions in gaining a better understanding of their online visitors.

We have found points of convergence between institutional intent and user needs as well as points where the two diverge. Although the cultures of museums and libraries developed from very different roots and demands, the political and cultural realities of the twenty-first century are fostering a convergence in both mission and practice. This convergence is nowhere more evident than in the use and application of technology. Although significant differences exist, the similarities are becoming more evident and important.

Research in all facets of Web use is required, but further work is particularly recommended on several issues that emerged from this project. The link between visiting an institutional Web site and making a real visit needs to be explored. Understanding this connection can help libraries and museums create stronger connections for their visitors between the physical and virtual sites. For museums, a well-designed virtual experience may lead to more visits to the physical site, which can lead to increased revenue. Libraries will be able to make better and more informed budgetary decisions that will lead to improved services for visitors to the physical and virtual sites.

In addition, the frequency of visits to the Web sites in this study deserves further inquiry. By understanding the needs of repeat visitors to library and museum Web sites, administrators and developers can take advantage of the technology to better serve an audience that may not be able to visit the physical site. Finally, this research points up the need for greater clarity of goals and objectives and tools that are more refined to assess both costs and benefits of this new information tool.


 

Skip to content