1.0 Executive Summary • CLIR

1.1 Aims, Scope, and Methodology

This report provides an overview of selected key national and multinational initiatives in digital preservation occurring outside North America. It examines digital preservation initiatives in four countries-Australia, France, the Netherlands, and the United Kingdom-as well as related multinational initiatives. The initiatives were chosen in consultation with the U.S. Library of Congress (LC) and the Council on Library and Information Resources (CLIR) because they were believed to be of particular relevance and interest to the U.S. National Digital Information Infrastructure and Preservation Program (NDIIPP).

This study aims to put these initiatives into their national and international context and to outline major developments. It is intended to be a high-level survey; for this reason, it does not aim to be exhaustive or detailed in terms of practice and procedures. This report presents the key findings from the survey and details of the main initiatives in each country.

1.2 Key Observations and Recommendations

This section of the Executive Summary begins with the author’s observations on principal trends and lessons. This is followed by individual observations and recommendations from each national library on the lessons learned as a result of its work for the NDIIPP. Their views on opportunities for future international collaboration in digital preservation are presented separately in each national overview in the main body of the report.

1.2.1 Author’s Observations and Recommendations

The national libraries surveyed differ greatly in scale and scope of collections and in responsibility. Although all the libraries have responsibility for the print and literary heritage of their respective countries, their responsibilities for audiovisual materials vary substantially. Each country also has a slightly different focus in terms of digital publications and is at a different stage of developing digital collections and digital preservation.

These differences in scope and scale of collections and individual national circumstances need to be borne in mind when considering the implications and lessons of this survey for the LC or the NDIIPP.

This report and the national surveys it contains are a snapshot of the status of activity as of March 2002. As time passes, further changes and new initiatives will need to be taken into account.

Despite these caveats, the author believes that there are significant lessons and opportunities for both the LC and the NDIIPP highlighted within this report.

National Initiatives and Funding

In none of the countries surveyed is there a single national initiative for digital preservation. Rather, there are many institutional missions that are being extended into the digital domain, including those of national institutions such as the national archives and national libraries.

There are some new efforts to provide national or international coordination and collaboration between such initiatives. One national example is the Digital Preservation Coalition (DPC) in the United Kingdom. Two examples of international efforts for information exchange are the Electronic Resource Preservation and Access Network (ERPANET) and Preserving Access to Digital Information (PADI).

Digital preservation is poorly funded in relation to the scale of the challenge. Institutions have received little or no additional core funding to address digital preservation; as a result, they must rely on short-term external project funding or reallocate internal resources. There are limits to what can be achieved by such means, particularly in large institutions or national programs.

In providing a funded and coordinated national program for digital preservation, the NDIIPP is seen internationally as a leading initiative.

It is far easier to obtain funding for digitization for access than for preservation. The long-term benefits and requirements of preservation seem often to be overshadowed by the immediate benefits of current access initiatives. Many countries are placing increasing emphasis on short-term project funding and are reluctant to increase the core funding of institutions. Increases in core funding will be necessary to make the longer-term commitments needed for preserving large digital collections.

Digital preservation relies on the collaboration of stakeholders outside memory institutions and the professional sectors they represent. Much digital preservation activity as a public good is supported either from government funds or private benefactors. However, awareness of digital preservation issues among the public, government, and other key stakeholders remains low. Significant effort should be placed in targeted outreach to key individuals and audiences as part of the development of the NDIIPP to ensure it has effective support and engagement with key communities. The public relations campaign launched by the DPC in the United Kingdom is seen by its member organizations to have been highly successful, and it may provide useful parallels for part of any outreach program in the United States.

Underlying Trends

The digital domain is changing the nature of institutions’ missions and their relationships with other organizations. These changes can be summarized as follows:

Changing patterns of distribution. Increasingly, institutions do not hold physical copies of digital works but license access to them. It is unclear who will have responsibility for archiving, and the level of trust in archiving arrangements is uncertain.
Changing time horizon for preservation. Digital media are fragile, and access to them is dependent on rapidly evolving hardware and software that quickly become obsolescent. Preservation of digital materials, therefore, does not happen by accident; it requires early action, often at the point of creation. In the digital environment, memory institutions must have much closer relationships with creators and distributors than was previously necessary. In addition, preservation actions must be taken earlier. Selection decisions can be harder, because they may have to be made earlier in the life of the material and without the benefit of time, which reveals the historical importance of different trends and material.
Changes in intellectual property rights (IPR) and archiving rights. No country in the survey has comprehensive legal provisions for archiving digital publications. The term of copyright has been increasing, and the investment in and economic value of IPR have also increased dramatically. The commercial need to protect IPR can overshadow other considerations. The needs of memory institutions for legal exceptions to undertake archiving are often overlooked or not sufficiently understood.
Globalization. Activities increasingly take place on a global scale and outside the traditional national frameworks for digital preservation. With the development of international publishers who can deliver their digital publications from anywhere, the role of archiving in a national context is less clear. Similarly, the growth of the Web and the international activity it empowers transcend national boundaries.

Globalization also applies to developments in hardware and software. Information technology (IT) companies and market trends operate on a global scale and apply to many different sectors. This means that there is more substantial common ground between institutions internationally and across sectors and greater potential for and benefits from international collaboration.

The information explosion. The volume and range of information produced is expanding dramatically. Now, digital publications in many countries supplement, but have not replaced, traditional publication. This increase in both traditional and digital information is straining national institutions, particularly national libraries, which have a tradition of comprehensive collection in specified areas. At the same time, many of the traditional filtering and editing roles of publishers are disappearing as the Web opens publishing to individuals and organizations. This places greater demands on the libraries in selecting material for acquisition.

This exponential increase in information is not confined to publishing; it applies even more to data in the academic and research sectors, particularly in the sciences.

Publications and records. In the digital environment, it is no longer necessary to generate many copies in order to publish material. A single copy can be networked and made accessible to anyone with a PC and an Internet connection. The boundaries between a “publication,” a “manuscript,” and an “archival record” have blurred. The respective roles of libraries and archives may have a greater degree of overlap in the digital environment.
The cultural record. Publications are now only one aspect of popular culture and the cultural record. Film, television, and the World Wide Web define an increasing part of our culture. Mechanisms to consider new areas of collection development and future research needs may be required as part of any national scheme.
The role of the private collector. Private individuals have frequently been vital in preserving collections of material, particularly ephemera that have not been in areas of contemporary collection by curators. In the digital environment, examples of private initiatives include Brewster Kahle’s Internet Archive or the sharing of early computer games and emulators by private enthusiasts. Digital preservation challenges and copyright protection mechanisms may make such efforts harder in the future. This could result in greater reliance on institutional selection decisions and the development of new tools to support personal archiving.

Digital Preservation

Institutions such as archives and libraries have evolved over many centuries as custodians of the “collective memory.” They are custodians over very long periods of time. Other institutions and sectors may be focused on shorter time horizons and rarely have this chronological perspective. It is not surprising, therefore, that memory institutions have been first to identify the challenges associated with digital preservation.

However, the challenges identified by archives and libraries will in time affect a wide range of institutions and may have a profound effect on the individuals and society in which they operate. Digital preservation is therefore not solely a cultural heritage issue. In the longer term, it will affect the nature of the “information society” that many governments worldwide are seeking to develop. There is a surprising lack of discussion or research into these deeper trends and the implications behind digital preservation issues.

Digital preservation is still a new field. Most initiatives have focused on selection, acquisition, storage, and maintenance of digital collections. Actions needed for long-term preservation (LTP) are only now being identified and addressed. The most successful initiatives noted in the survey were located at institutions that had been working on practical implementations and policy for several years.

Collaboration and Partnership

Collaboration between institutions occurs on many levels. External funding has encouraged collaboration on research. In some cases, collaboration with other institutions has been a requirement of such research funding. Research collaboration has also occurred without this external incentive, but it is often on a less formal basis or with fewer resources.

It has been harder to collaborate and coordinate on collection policies. The PANDORA (Preserving and Accessing Networked Documentary Resources of Australia) archive, which has evolved over many years, is the only real example of this in the survey. Coordinating and distributing responsibility among institutions is also seen as important in the United Kingdom, but this country still has some way to go to put appropriate arrangements into effect.

Partnerships seem to work best when the participating institutions have their own initiatives and experience and when all parties have something to offer to and to gain from the collaboration. It is important to develop in-house expertise as well as to use experience available externally.

All the libraries in the survey emphasized the importance of working with stakeholders. This report notes many examples of successful approaches, including agreements between publishers or publishers’ trade bodies and national libraries. Outreach publications aimed at data creators have also been produced by the Arts and Humanities Data Service (AHDS) in the United Kingdom (the “Guides to Good Practice” series) and the National Library of Australia’s (NLA’s) information leaflets on safeguarding Australia’s Web resources.

Digital libraries are a relatively small sector, and they benefit not only from working together but also from being aware of trends and potential partnerships in other sectors. An example of this is the Open Archival Information System (OAIS) reference model, which is emerging as the first international standard in digital preservation. While developed by the communities engaged in earth observation, OAIS has broad applicability and has been widely adopted in the library community. The library community, in turn, has heavily influenced the development of the draft reference model.

Digital preservation and related issues such as mass storage and automation of metadata are likely to be important elements of the “research grids” being developed to support collaborative science and the scientific research infrastructure. The NDIIPP should explore and encourage opportunities for synergies with these developments.

Governments worldwide are encouraging developments in “e-government” and “information society” that are having a major impact on the provision of digital access and on the development of digital work processes and procedures. Electronic records management often figures in such programs, but longer-term issues are seldom considered. The Digital Longevity Program in the Netherlands is a rare exception to this rule. Similar awareness raising and close engagement with e-government initiatives is recommended as part of the NDIIPP.

Many of the traditional certainties of publication and archiving are in flux as we move into the digital environment. Few institutions are likely to be directly involved in digital preservation, but many are likely to be involved in providing access services that may rely on such archiving activities for long-term access. Funding and institutional models for this set of relationships remain to be defined; however, there are a number of interesting developments. One such development is that some international publishers have recognized certain archives as official archives for all their published output (see the Koninklijke Bibliotheek [KB] and Elsevier in the Netherlands). Another development is the potential growth of collaborative archiving arrangements for consortia linked to national deposit libraries or academic research libraries (see COUPERIN [Consortium Universitaire des Périodiques Numériques en France] or JISC in the United Kingdom).

National libraries offered several recurring suggestions for international cooperation in developing effective strategies for long-term preservation:

There is a need to develop a preservation technology watch for file formats and new technologies, emulators and migration routines, and information on and repositories for obsolete software. National libraries felt there is significant scope for international collaboration and potential cost benefits in developing these services on a shared basis.
In larger national programs, there may be scope to develop some shared services and central support for digital preservation in a distributed network of digital archives.
In the academic sector, the OAIS and exploration of new methods of scholarly communication are growing rapidly. The focus of these initiatives is improving current access, and there is less consideration of long-term requirements for preservation. The position of such repositories, the materials they hold, and any long-term requirements should be considered in any national collaborative scheme.
There is also a need to foster research on LTP and to develop standards and good practices. This would be an obvious area for engaging in international effort and for developing closer partnerships with national research funding bodies and academic research institutes and departments.

Staff Training and Development

Most institutions in the survey raised staff training and development issues. Digital preservation may require new positions for individuals with a crossover set of skills and a broad view of operations to coordinate and direct activities. However, most of the effort will be drawn from existing staff and will require teamwork across departments and skill sets within the institutions.

Audiovisual Materials

The audiovisual preservation community is in many ways unique within the survey. Because the media and technologies used in their industries have been impermanent and cannot be preserved long-term, digitization is widely accepted as their preferred method of audiovisual preservation. The film, audio, and video archiving communities, therefore, have a direct stake in resolving digital preservation challenges over the next decade.

Although not a primary focus of this study, audiovisual materials have been included to some degree. Three observations stand out:

The audiovisual community has undertaken extensive research and evaluation of the archival qualities of storage media such as recordable compact disks (CD-R). This work is not widely known in the library community but is highly relevant to a broader audience and deserves to be better known.
Audiovisual storage requirements are very large. Moving from offline storage, such as CDs, to mass-storage systems will require very large-scale storage systems.
The PRESTO project is one of the few examples of an attempt both to identify the scale of preservation requirements for a group of institutions and to construct an effective business case for further investment. The Library of Congress should examine the survey questionnaire, the survey outcomes, and the technologies being developed.

Research and Development

National libraries have identified Web archiving, whether by selective gathering of specific Web sites or whole domain capture for specific national territories, as an important function. It is recommended that work within the United Kingdom, France, Australia, and Scandinavia be followed closely and that opportunities for the joint development of tools and practice be explored.

The OAIS reference model has become widely accepted as a standard in digital preservation. It is recommended that OAIS be used within the NDIIPP, and that support be given to further key developments. These may include the production of accessible guides to major concepts behind the standard, dissemination and sharing of experience among implementers of the standard, and efforts to develop supporting guidelines and standards in areas such as identifiers, “ingest,” and certification.

The survey highlights the need for persistent identifiers. This is an issue not only for online publications and the Web but also for the linking and citation of primary research and data sets. Further work on persistent identifiers as part of the NDIIPP is recommended, as is close liaison with international developments in this area.

Several ingest activities-the acquisition and processing of digital objects into collections-need further development. The space science community is focusing on space mission data. Within the library community, there is a need to focus on metadata from publishers. Promising preservation metadata schema have emerged, and the RLG/OCLC Working Group on Preservation Metadata is developing an international framework.

The logical next step is to examine implementation issues. Many publishers are international and would respond positively to international standards and coordination of requirements. Links to metadata standards being developed within the publishing community, such as ONIX, would also be desirable. The needs of publishers of all sizes will need to be considered. In this respect, current work to develop Simple Onix Editor, a tool for automating production of ONIX metadata funded by the British Library at Book Industry Communication, may be of interest.

With the development of new roles and potentially new interdependencies among different organizations, it will become increasingly important to certify digital archives. Further consideration and support should be given to defining benchmarks or institutional standards for digital preservation and certification models.

There has been relatively little major research in digital preservation. This area warrants investment for further research and development.

It is recommended that NDIIPP consider carefully the outcomes of the research projects noted in the survey. Consideration should be given to building on these projects and to developing one or more digital preservation testbeds to evaluate the scalability, strengths and limitations, and costs of promising approaches.

Dissemination

Institutions vary greatly in the extent to which they distribute their work on digital preservation-for example, the information they mount on their Web sites.

There is a need to ensure that information, tools, and experience are shared effectively within the international community. This survey describes several international and national efforts having this objective. It is recommended that the NDIIPP consider carefully how information on U.S. initiatives should be disseminated, how such dissemination should be supported, and how the initiatives relate to and participate with similar activities internationally. This survey provides a number of exemplars and suggestions, including the need to ensure that such efforts are specifically funded. There is also potential to coordinate such work with that being undertaken by the DPC in the United Kingdom, ERPANET in Europe, and the NLA.

There are pronounced differences in the extent to which various institutions are exposed to, are aware of, and respond to international developments. International exposure seems highly beneficial and is apparent in many of the most successful initiatives included in the survey. Such exposure should be encouraged within the NDIIPP.

1.2.2 National Library of Australia Observations and Recommendations to NDIIPP

Know your critical stakeholders and work with them. For example, the NLA found it essential to build goodwill with publishers, particularly in the online environment.
Recognize that collaboration takes effort and leadership, and has its own limitations. Do not underestimate the investment in relationship building and the diplomatic skills needed. Give-and-take is needed, and results accumulate over time.
Make a start, and then let experience, practice, and policy evolve and inform each other. Recognize you cannot solve all the problems at once. Starting small on defined areas and building in feedback mechanisms for continuous learning are essential.
Integrate digital preservation into the institution. Do not rely on time-limited external or project funding to achieve your aims.
Build on the people and expertise you have. The NLA has developed its internal staff and established teams working across departments to bring together relevant skills. Look for internal synergies to support the activity.
Realize that it is initially hard to calculate costs. Cost models depend on many variables, and the NLA is in an experimental phase of development. Cost recognition and management can, however, be improved over time.
Recognize that the major challenges are not only conceptual but also practical. Addressing them requires developing policy and experimenting with strategies and procedures.
Consider the merits of the selective approach to archiving online resources. It is one of many approaches, but for research use, the intervention of the librarian is important and cannot be replaced.

1.2.3 Bibliothèque nationale de France Observations and Recommendations to NDIIPP

Be aware that addressing the deposit and preservation of online materials is a key issue.
Recognize the value of sharing research and jointly developing approaches to Web archiving between institutions.
Conduct research on collecting and preserving database-driven Web sites.
Be aware that trying to influence what publishers produce is a critical issue. Libraries have difficulty with CDs and other materials in proprietary standards today, and it will be even more difficult for these resources to be accessed tomorrow.
Establish early contacts with producers of electronic materials. The BnF is considering undertaking more initiatives with publishers.
Recognize that raising awareness about digital preservation within one’s own institution is as important an issue as influencing others externally.

1.2.4 Koninklijke Bibliotheek Observations and Recommendations to NDIIPP

Recognize the difference between the publishers’ value-added service environment and the underlying content. Take the publications out of the service environment and into the archiving environment of the library.
Use standards such as the OAIS where they exist.
Work with other organizations to encourage the development of commercial market solutions and systems for digital preservation.
Begin by identifying what you have in common with potential partners rather than how you differ. Use this common base to focus and scope what you want to do together.
Recognize that successful digital preservation initiatives depend on getting staff involved across the institution. Many management issues need to be addressed.
Be aware that collaboration requires time and a sense of community. Participants must make face-to-face contact and must know their partners.
Keep membership of project teams stable. Continuity is essential to maintaining progress and the relationships built up with partners.
Emphasize how important it is that institutions communicate with each other and share lessons learned. All institutions agree with this in principle, but a staffing commitment is needed to make it happen. This is rarely done in practice because of conflicting time commitments. Specific funding may be needed to allow institutions developing and practicing digital preservation to communicate their work.
Appoint project leaders who can make things work and who have a positive attitude toward problem solving.

1.2.5 British Library Observations and Recommendations to NDIIPP

Have leadership from the front on the issue of digital preservation and strong commitment from senior management.
Communicate the urgency of the problem. Preservation is a digital time bomb; failure to act may lead to total loss.
Recognize that the requirements for access in a large project such as the Digital Library Store (DLS) are very complex. There is a need for a modular approach focusing on the store as well as for access and integration via other systems. Progress with the DLS has not been easy, and the scoping of the project has been difficult.
Be aware of the need for an overarching e-strategy, particularly in very large libraries with complex systems. It is important to keep all digital developments in step and to consider the interface between systems.
Learn from parallel work in other institutions. Collaboration with the KB has been particularly useful for the BL. At the same time, differences in scale are important when looking at national libraries and transferable lessons.
Recognize that working more in partnerships will be essential if digital preservation is to be addressed successfully. However, collaboration can complicate things and has costs as well as benefits.
Do not underestimate the importance of sufficient staff. The pool of specialists and generalists in digital preservation is very small. In searching for a digital preservation coordinator, the BL recruited from Australia. Digital preservation cuts across a wide range of activities and departments. Awareness and capacity must be built internally so that a wide range of staff can contribute to digital preservation as part of their daily activities.
Get behind one initiative, such as the DPC, rather than become involved in competing groups.
Focus collaborative activity in an organization separate from any single partner but with heavy involvement from each of the key players.
Do not seek the “ultimate preservation solution,” which remains elusive, said the National Sound Archive. Not all challenges will be solved instantly, and a combination of approaches is likely to be appropriate. We must use professional skills and harness technology now to maintain holdings in our generation and to ensure we can plan to migrate them for future generations.

1.2.6 Recommendations for Further Technical Investigation

The author recommends that the Price Waterhouse Coopers Consultancy for NDIIPP undertake further detailed investigation in the following areas:

Australia

The PANDORA distributed national online collection and the software used to support this collaborative archiving effort
The NLA Digital Objects Management system and proposals for developing its capacity to manage long-term preservation
The NLA digital preservation work program
Proposals for the national Australian Digital Resource Identifier scheme

France

The Web harvesting tools and approaches being developed by BnF
The preservation technologies being developed by Institut National de l’Audiovisuel (INA) as part of the PRESTO project

Netherlands

The Deposit of Netherlands Electronic Publications (DNEP) digital deposit system for electronic publications being developed by the KB and IBM-Netherlands
The outcomes of the Long-Term Preservation Study being conducted by the KB and IBM-Netherlands and its implications for the development of a LTP module as part of the DNEP
The Digital Preservation Testbed being conducted by the Dutch Ministry of the Interior and outcomes of these experiments

United Kingdom

The British Library’s DLS
The e-preservation strategy and systems being developed by the Public Record Office
The outcomes of the Cedars research project
The outcomes of the CAMiLEON research project
The audiovisual and new media preservation technologies and projects being developed by the BBC

Other Projects and Initiatives

The audiovisual preservation technologies being developed by Radiotelevisione Italiana as part of the PRESTO project
The research outcomes and tools from the Networked European Deposit Library (NEDLIB) project
The OAIS reference model and implementations noted in the report