PART III: IMPLEMENTING THE MOA II SERVICE MODEL

Selecting Digital Archival Classes

Project staff selected a group of object types and classes from the materials suggested by institutions participating in the MoA II project. These archival object types are the core of those examined in this project. The number of items selected was limited to facilitate completion of the testbed within the project time frame.

The object types and classes selected were as follows:

1. Continuous-tone photographs: Single archival object. The photograph may have a caption or other textual information recorded on its face or on verso. Continuous-tone photographs are interesting for this project because they exist in abundance in many of the collections and because they enable a close look at the collection of administrative metadata for use in object behaviors. The most basic of objects, the continuous-tone photograph provides a solid platform upon which to base the project.
2. Photograph albums: Bound manuscript object containing a collection of continuous-tone photographs. The photograph album may contain captions that are separate from the photographs or other items such as newspaper clippings. Photograph albums are a logical extension of continuous-tone photographs, since they contain photographs that are ordered in a structured manner and that raise both structural and administrative metadata issues.
3. Diaries, journals, and letterpress books: Bound manuscript objects, usually arranged chronologically and with date notations. May have additional structure (for example, an "accounts" section noted in the back). These structured documents have the further possibility of additional metadata in the form of partial text (dates and other markers) included for additional navigation. With the inclusion of full texts, such as the William Henry Jackson diaries at the New York Public Library, full searching and navigation are possible.
4. Ledgers: Bound manuscript objects that contain accounting records. They are usually arranged by account although sometimes they are in chronological order. The structure of documents of this sort is different from that of diaries and journals; however, in terms of structure and navigation, they may be considered a variation on a theme rather than a different object type. For these objects, inclusion of more text, while costly, allows for more sophisticated searching and navigation.
5. Correspondence: Objects of this class may be simple (a one-page letter) or complex (a long letter with an envelope and enclosures). Investigating correspondence allows the project to examine these sometimes-complicated documents and the structural metadata relationships between the subdocuments (letter to envelope, for example).

These types of materials have been selected for the testbed because the participating institutions hold large quantities of them or because they offer the project important challenges in terms of the behaviors needed to view, navigate, or manipulate them. The complex structure of photograph albums, for example, requires that users be able to see individual photographs, a photograph with its caption, photographs and captions in the context of pages, and pages in the context of an entire album. With diaries and journals, users may want to see individual entries or to jump from one entry to another or from an index to an entry. Diaries and journals also raise the issue of individual page scans that do not correspond with the logical structure of the document. For example, entries frequently end in the middle of a page and a new entry begins on that same page. In addition, when different levels of metadata are available, these materials allow for display and navigation experiments. For example, a minimal digital diary might consist of a series of page images with only a base set of behaviors that can be implemented (for example, "turn to the next page"). A richer digital diary may have encoded text transcribed that allows for a variety of tool behaviors for each page image. For example, the tools could display a table of contents for the diary, jump to a particular page or entry, or search for text strings. The structure of letterpress books and ledgers offers the potential for interaction between indexes in a document and its individual entries or parts. The project is also exploring how the structure of these items differs from those of diaries and journals. While ledgers, letterpress books, journals, and diaries are different classes from an archivist's point of view, they may be quite similar from a structural metadata perspective.

The MoA II testbed will give participants and the broader archival and library communities a chance to evaluate different practices for encoding the relationships among objects. In particular, it will help the community understand the advantages and drawbacks of using these practices based on how tools implement different behaviors for each practice. For example, a series of correspondence could be scanned and:

1. placed into a single base object;
2. created as separate objects (one for each letter) and linked through the creation of a new aggregate collection;
3. created as separate objects inside an embedded collection (folder) object. A collection object has metadata for the collection, followed by the embedded objects, each with its own metadata and content. This approach differs from item 2 in that the objects are embedded rather than linked; or
4. organized through a finding aid in which the container list points to any of the above.

Each base object, whether it stands by itself or is part of an aggregation or embedded collection object, can be divided into sections by text encoding. For example, a diary can have dated entries identified by the text encoding that can be used for display and navigation. In the same manner, any type of object may also be a compound object through linking to other objects or embedding other objects inside itself.

While the concepts of compound, linked, and embedded objects are not new, the MoA II testbed will give archivists and librarians a tool with which to better evaluate options for digital archival objects, particularly in the context of distributed repositories. The MoA II testbed will give the DLF and the wider community an opportunity to create objects using all the practices listed above. It will also allow for the evaluation of each practice to identify how tools can best use each practice to meet audience needs.

MoA II Testbed Services and Tools

The digital library service model described in Part II of this report is a three-tier model consisting of services, tools, and digital objects. The MoA II testbed is implementing the standard archival model within the digital library service model. That is, USMARC collection-level records in a catalog will link to their related finding aids, which will link to the digital archival objects in that collection.

The top tier in the model (see fig. 1) is a services layer composed of suites of tools that focus on supporting particular groups of users (such as scholars, undergraduates, or K­12 students). An archivist, for example, would require different tools for the discovery, display, navigation, and manipulation of digital archival objects than would a fourth-grade student. Initially, the MoA II Testbed Project is focusing on general services for scholars who are using the classes of digital objects selected for the project. Future research projects could include developing service models for novice users or customized services for specialized scholars (such as what is envisioned for the Digital Scriptorium Project).

The suite of tools to be developed in the MoA II testbed will initially include the following:

• an online catalog (OCLC's SiteSearch) used to discover and display the USMARC collection-level records;
• an SGML-compliant database (INSO's DynaWeb) used to search, display, and navigate the electronic finding aids that are compliant with the EAD; and
• display and navigation tools to be used with MoA II-compliant digitized photographs, photograph albums, diaries, journals, and correspondence. As the project entered the production year, project staff consulted various parties (including archivists, scholars, and librarians) to understand the behaviors required in this tool set.

The MoA II testbed implementation of the digital library service model is shown in figure 2.

Fig 2. MoA II model implementation

Behaviors and Methods"What Tools Do"

Part II of this paper introduced the concepts of behaviors and methods in the context of object-oriented design. This section outlines those concepts in detail.

Definitions

The digital library service model defines behaviors as the ways in which users describe what tools do for them. Engaging users in a dialogue about behaviors can help identify user needs in terms of the functions performed by tools. System designers can then map these user-level behaviors into methodsdefined as discrete segments of program code that execute operations for tools. The translation from behaviors to methods represents the transition from defining user needs to system design. In many cases, high-level methods in a tool have the same name as a user-level behavior. The ability to create methods that model a user's desired behaviors is one of the strengths of object-oriented design.

This definition of methods will be applied to specify the range of activities that should be supported through the metadata described elsewhere in this paper. In an object-oriented model, the methods are embedded in the digital objects; digital objects reveal methods to tools interacting with them. Many repositories in the MoA II environment will not deploy object-oriented models; in this case, the methods will be made available to tools that interact with repositories, rather than with the digital objects themselves. Nevertheless, this model is likely to be helpful both in conceptualizing the nature of the tasks supported and in preparing for a type of digital library that may scale more effectively than current architectures.

The digital library should support methods that both common and exceptional operations users expect to perform with the digital objects. For an image collection, a method might be facilitating a pan or zoom on a portion of an image or providing an enlargement of an image. For an encoded diary, the method might involve providing the tool information about levels and types of organization (such as "One volume, including 128 dated entries, an itinerary, and a list of contacts with addresses"). The encoded diary's methods might also yield both simple (next entry, previous entry) and complex navigation (for example, "Locate the first dated entry in November 1884;" "Find entries where dates are separated by more than 10 days"). Although no object or repository would be required to support the full range of methods, the model proposed here will facilitate the development of increasingly sophisticated tools that can be scaled for use on a growing body of complex archival objects.

Contexts and Constraints

The methods of the digital library reside in tools that may be client-based or server-based, depending on the state of technology. The location of that method (that is, with the client or server) may shift with changes in technology. For example, widespread adoption of an image client that supports progressive transmission of image data might shift image processing from server to client, thereby expediting the processing and reducing the load on the server. However, an interim measure might rely on a server-based compression-decompression process in which the server can generate pan or zoom views at the user's request in real time. This relieves the client of processing responsibility and shifts the work to the server.

Just as the methods may move from client to server and back again, they will also separate into specialized functions or merge into high-level, multifaceted functions. For example, print and display might be different methods, with one object optimized for screen display and another optimized for printing. In Adobe Acrobat, for instance, display and print are merged in the same tool. If it is argued that Acrobat handles display poorly, a clearer separation of these two methods might be advisable. By separating the methods conceptually, it is possible to assess the applicability of the tool in the service of the method.

High-level methods frequently consist of a series of calls to lower-level methods. For example, because print is a behavior that most users require in a tool, most tools will have a high-level method expressed something like "print(a1,a2aN)." The information inside the parentheses represents arguments that tell the method what object to print, which printer to use, and so on. The print method would execute a series of lower-level methods. It may, for example, ask an object to deliver its content in a format suitable for printing by executing the proper method. Next, it may execute an operating system­specific method that sends (spools) the formatted content to that particular printer.

Some methods are applicable to all or most objects, while others may need to be finely tailored to the type of objectso finely tailored, in fact, that they rely on entirely different functions or primatives. An obvious example is the difference between navigation of pages and navigation of portions of an image. A system that navigates a bound book might use operations such as "Display next page" or "Show page list." A system that navigates a continuous-tone image might use operations such as "Display 200 by 200 pixels centered on coordinates X by Y."

The following sections define methods that are central to creating the digital library. When possible, the descriptions are sufficiently generic to apply to a variety of objects; in some cases, the methods described are specific to some data types.⁵

Navigation

General Navigation

Navigation consists of a request, a receive, and a display action. Each action interacts with a reference to objects or metadata for objects rather than to the objects themselves. A significant portion of the user's activity is navigation. For example, the user who finds a digital scrapbook in a repository may request a table of contents. Depending on the extent to which the book was processed, that table of contents may consist only of a stream of page references or of a nested list of chapter and section headings. In navigating the scrapbook, the user's navigation tool will

1. request references (to pages listed by page number or to sections listed by section headings);
2. receive the references in a discernible format; and
3. display that information in a way that is meaningful for the user.

The user expects a series of references, not actual delivery of the objects, which will not be sent until requested.

Navigation also depends on an understanding of relationship primitivesparent, child, and siblingthat are the generic references a tool uses to facilitate navigation. The navigation method is affected by the tool requesting, receiving, and displaying the parents, children, or siblings of an object that the user has located. For example, to navigate a fully encoded and logically organized scrapbook, a navigation tool might request references to the first-level children in the scrapbook. A user would be presented with a list that looked like the following:

• dated entries
• accounting/budgetary data
• names and addresses
• itinerary

Each major heading may be presented as a link for further expansion, perhaps offering second-level headings to the user (for example, annual groupings of entries in the dated entries section). A threshold setting in the navigation tool may instruct the repository to send no more than N references at a time. The repository would make a determination that, for example, all first-level and second-level headings fit within the threshold (providing the annual groupings within the dated entries section at the same time it provides the four major headings listed above). At some point, the children of a given parent will be links to the objects themselves. In the scrapbook example cited earlier, the user could be presented with a navigation list of dated entries, any of which could be selected for display. Types of object references vary, depending on the type of resources, the amount of funding available to process the materials, and the programmatic or other purposes of an initiative. Examples include conceptual and structural references (as in the scrapbook example), simple page lists (such as Project Open Book and the University of Michigan Making of America sites), and pages of thumbnail representations of larger format images.

Image Navigation

In contrast to the generic form of navigation just discussed, image navigation uses image-specific information. Systems that display image information need information analogous to that used in geographic references; for example, X and Y coordinates, along with dimensions of the portion of the image to be displayed. Increasingly, tools for image management and manipulation use segmentation to optimize the relatively confined space of a video display as well as other resources in short supply (such as network capacity, memory, and CPU capacity). Some of these technologies are primarily server based, while others shift responsibility to the client. Wavelet compression, for example, allows a repository to store an extremely high-resolution image and generate lower-resolution versions and subsets in real time, at the request of the user or an intermediary. Another approach, implicit in tools or formats such as JPEG Tiled Image Pyramid (JTIP), segments the image into overlapping tiles in a pyramidal structure and allows the user to pan and zoom on a full-resolution image by requesting the next tile or a corresponding tile at a higher resolution. The image-navigation tool receives information about resolutions, resolution ratios, and window sizes to make the navigation possible; the image display tool uses that information to pan, zoom, crop, or otherwise use images.

Display and Print

The display method uses a reference to a known object to deliver an item to a screen-oriented tool, such as a graphical Web browser. In contrast to navigation, where the user tool requests object references, the display tool works from an object identifier it has received from an intermediary or can infer from a query where only one object exists. From this relatively simple operation emerge more complex issues, one of which is the variety of known item references that an intermediary must handle. At its simplest, a display tool will encounter references to page images in a format clearly identified by structural metadata. Similarly, the tool may receive a reference to an encoded text section such as a chapter, again in a format identified by structural metadata. Examples of references that are slightly more complex include requests to display the next or previous sibling of the digital object (next page or previous chapter) or the parent of the digital object (the chapter that includes this page). For images, an intermediary may request the display of a known item at a specific resolution. More challenging will be the standard articulation of a reference to display a portion (for example, "250 by 250 pixels, centered on pixel X.Y") of an image at a specific resolution. Panning, zooming, and cropping of an image are variations on this type of request.

Printing is a method similar to the display method; it differs only in its use of printers, plotters, disks, and other output devices. The option to print may use the same format as the option to display, as is the case with systems relying primarily on encapsulating images in portable document format (PDF) for delivery. The display and print methods are closely intertwined and differ according to the formats available from the repository and user preferences. For example, imagine that a repository of bitonal, 600 dpi page images offers graphics interchange format (GIF) images with interpolated grayscale, Postscript, and PDF. A user without Adobe Acrobat may choose to display the GIF images but print using the Postscript files containing encapsulated images. A user with Acrobat may choose to rely entirely on the PDF image files to print and display from the same source.

Combination or Comparison

As the body of materials in the digital library grows, the ability to create combinations of data or perform comparisons becomes increasingly important. Combining and comparing methodsmost common with art and architectural images (consider the common use of two slide projectors in art historical instruction)are applicable to both images and text. Support for the comparison of two passages of text or the display of a text alongside an image is also needed. Common applications might include the following:

• synchronous scrolling of a text in two different languages or a text with commentary;
• the side-by-side display of a text and an image (Dante Gabriel Rossetti's paintings and poetry, often using the same title or treating the same theme);
• the display of two images side by side;
• the display of an indeterminate number of image objects positioned in a grid; or
• the display of a number of image objects positioned in a grid of specific dimensions.

Somewhat more complex is the requirement to combine objects. The need to apply layers to, or remove them from, an object is especially important. This problem is considerably easier when a single repository has provided these layers with the base image; however, a more generic method is needed to allow the combination of layers from diverse sources. For example, the allegorical drawings found in nineteenth-century journals such as Harper's Weekly often contain contemporary public persons portrayed as historical figures. A server at one DLF institution might provide the images of the pages, while a second DLF institution might overlay commentary identifying each person. A tool supporting the display method might offer that image in any number of different resolutions; it might even display portions of the image cropped and enlarged. The ability to coordinate the annotations of the second institution with the page image from the first institution will require carefully controlled metadata about coordinates that stay constant with the cropped portion. While these types of administrative and structural metadata may be too challenging for early portions of the testbed project, the model must be flexible enough to accommodate this information in later iterations.

Repository Search

Repository search methods, more than most other methods, tend to be exhibited by server-side programs rather than by client-side tools.⁶ At least portions of these methods could eventually migrate to the user's desktop, but considerable standardization must first take place. In a repository search, an intermediary collects information about the user's query and the characteristics of the available collections and begins to process results (for example, in a sorted list by object or by collection). This intermediary has distinct discovery and retrieval functions.

To support discovery within and among repositories, each collection must participate in a conversation with the client or intermediary. This conversation constitutes the method associated with discovery. The repository's discovery method must include the ability to understand the search parameters of the repository, including gathering information on searchable fields, on the sorts of operators that can be applied, and other constraints. These mechanisms have been specified in protocols such as Z39.50, but it is important to explore more flexible mechanisms such as the search interface language (SIL) specified in Nigel Kerr's Personal Collections and Cross-Collection Technical White Paper (1997).

To support the retrieval of results within and among repositories, the conversation must include a well-specified retrieval method. Results must come from the repository or repositories in a well-articulated and easily parsed syntax. The tool will use this syntax, for example, to build result lists, to bring together results from multiple repositories, and to compile results from multiple repositories. An example of a proposed specification for such a retrieval method can be found in Nigel Kerr's white paper.

Color Analysis

An admittedly challenging set of methods will come with richer and more reliable color metadata. The availability of a Color Look-up Table (CLUT), which provides color, shape, and texture distribution that can be processed through automatic means, will aid in a variety of tasks. For example, a color-matching behavior might take CLUT information from manuscript fragments, locating fragments that are more likely to be from the same paper stock because of color or texture. CLUT information can also be used to measure subtle variations such as shape and patterns; this would enable the user to detect, for example, hidden features such as characters obscured by palimpsest erasures. Methods using the CLUT could support these types of analyses.

Bookmarks, Annotations, and Links

As the digital library grows in maturity and capability, the array of interactions with objects will become more complex. Users want intraobject bookmarks, annotations, and more sophisticated linking methods, none of which is outside the capabilities of readily available desktop technology. Nevertheless, the ability to support these methods is hampered by unreliable or incomplete metadata, the absence of generalized notions of user authentication and authorization, and a lack of support from repositories. Most important, libraries and archives lack the tools to exploit methods in these areas. Many of these methods are explored in detail in the research applications developed by Robert Wilensky and his team as they pursue notions of "multivalent documents" (Phelps and Wilensky 1996). Moreover, emerging standards such as the extensible markup language (XML) and extensible linking language (XLL) will help articulate complex links (such as a span of information or "the third paragraph in the fourth section") within a remote document. These methods will be best supported through the articulation and adoption of architectures within which effective tools can be built and through metadata that document a full range of digital object features.

MoA II Metadata

Metadata can be in a header, a MARC record, a database or an SGML file, or they can be distributed among a variety of locations. An object or repository needs only to be able to reconstitute the metadata and present them to a user or application when requested (the discovery, navigation, and administrative functions).

Descriptive Metadata

The library community has a long history of developing standards and best practices for descriptive metadata (for example, MARC and the Dublin Core). Given existing standards and ongoing work to investigate descriptive metadata issues, the MoA II proposal did not focus on this area. Instead, the MoA II testbed used a union catalog with MARC records contributed by the participants. The participants also contributed finding aids encoded to the EAD community standard.

In the discovery process, users will search the MoA II union catalog of MARC collection-level records that will be linked to their corresponding finding aids, and the finding aids will then be linked to the appropriate archival digital library object. Of course, it will also be possible to search the finding aids directly to discover archival library objects.

Structural Metadata

The terminology of the digital library is evolving rapidly. Consequently, there is considerable variation in how the library community uses certain terms, one of which is structural metadata. (For more information on the definition of structural metadata, readers are referred to Structural Metadata Notes in the Appendix.) For the purposes of this paper, the term structural metadata is defined as those metadata that are relevant to the presentation of a digital object to the user. Structural metadata describe the object in terms of navigation and use. The user navigates an object to explore the relationship of a subobject to other subobjects. Use refers to the format or formats of the objects available for use rather than formats stored.

Current thinking divides digital library metadata into three categories (descriptive, administrative, and structural) or two categories (descriptive and structural, with structural metadata subsuming administrative metadata). This report separates administrative and structural metadata. More important than this separation, however, are the categories proposed for inclusion in the MoA II architecture.

Although the categories defined here are presented in SGML, the data in a repository will not necessarily be stored in an encoded form or in a table. This document does not advocate a particular method for storing data; various approaches will be necessary in different institutions, and several approaches may even exist within the same institution. For example, descriptive metadata may be stored in USMARC, portions of structural and administrative data in relational tables, and other portions in SGML. These examples are intended to illustrate the type of data presented in interactions between repositories and intermediaries. The authors assume that these metadata will be extracted from metadata management systems in interactions with intermediaries such as tools. Further, default and inherited values will be expressed explicitly at the level of the subobject, even if implicitly associated with the subobject through the metadata management system.

Structural Metadata Elements and Features Tables

Tables 1 and 2 recommend the full set of possible structural metadata elements that an individual collection may find useful. Some repositories will use only the minimum set of required elements. Others will also use other elements that can be derived in an automated fashion. Still others will use elements that are easy to capture or derive. The tables include both minimal and maximal values, identify required and repeatable fields, and identify whether field values may be inherited or supplied manually. Some elements can fulfill both administrative and structural functions.

Some elements are relevant to raw data (such as page scans) which do not require extensive examination of the data structure. Other elements are relevant to "seared" data (such as chapter divisions and headings), which require only minimal examination of data structure to generate appropriate metadata. Still other elements are primarily relevant to "cooked" data (such as SGML marked-up text), which require serious examination of structure.⁷

Table 1 describes structural metadata defining the object, and table 2 describes structural metadata defining the digital subobjects (for example, the individual digital pages). Thus, the structural information for a digital object is divided into information referring to the constituent objects that cohere into a whole (such as a description of the extent of a digital book) and information specific to the individual parts (such as page or image references).⁸ The distinction between object and subobject metadata is in some ways artificial. For example, a tool might assemble information related to the constituent parts of a photograph album by querying each of the constituent subobjects rather than by querying a specially designed digital object. However, certain economies, such as stating ownership only once in the object rather than with each subobject, prevail when storing information. This model strives to balance the specification of elements accordingly.

Administrative Metadata

Administrative metadata consist of the information that allows the repository to manage its digital collection. This information includes the following:

• data related to the creation of the digital image (such as date of scan, resolution)
• data that can identify an instantiation (version or edition) of the image and help determine what is needed to view or use it (storage or delivery file format, compression scheme, filename or location)
• ownership, rights, and reproduction information

Some metadata elements are both structural and administrative and may be used for similar purposes in those two areas. For example, content type is a structural metadata element used to present available file formats to a service, while file format is an administrative metadata element that tells systems administrators the format of a particular file.

Administrative metadata are critical for long-term file management. Without well-designed administrative metadata, image file contents may be as unrecognizable and unreadable a decade from now as Wordstar or VisiCalc files are today. Administrative metadata should help future administrators determine the file type, creation date, source of original, methods or personnel that might have introduced artifacts into the image, and location where different parts of this digital object (or related objects) reside. Eventually, administrative metadata may help support the long-term management of objects; for example, the metadata contained within the objects will allow for automation of migration from an older file format to a newer one, for refreshment, and for regular backup.

In the past, certain administrative metadata (such as file formats) resided in file headers, while others resided in accompanying databases. In the future, all administrative metadata may reside within the file header. Such a system would be ineffective, however, until there are community standards that specify where they would go within the header, how to express them, and so forth. Work is already under way to develop those standards. In April 1999, NISO, DLF, and RLG convened a meeting to discuss technical metadata elements for images. The results of the meeting are available on the Web (Bearman 1999). For the purposes of this paper, the necessary administrative metadata fields are defined without regard to a particular syntax of where they will actually reside. For the purposes of the MoA II project, administrative metadata will be delivered external to the image file header.

This section primarily concerns administrative metadata for master files. In the future, however, repositories are likely to see master files that are themselves derivatives of previous files. To make the administrative metadata that the authors identify as compatible as possible with future developments, some information dealing with derivative files (or other instantiations of a work) is included. In this way, the project will lay the groundwork for future research projects to identify and trace the provenance of a particular digital work.

Administrative Metadata Elements and Features Tables

Tables 3–6 recommend the full set of administrative metadata elements that an individual collection may find useful. Some repositories will use only the minimum set of required elements; others will also use elements that can be derived in an automated fashion. Still others will choose to use elements that are easy to capture or derive. The tables include both minimal and maximal values and specify which are allowed to repeat. Some elements can fulfill both administrative and structural functions.

Although the number of metadata fields seems daunting, a high proportion may be the same for all the images scanned during a particular session. For example, metadata about the scanning device, light source, or date are likely to be the same for an entire session. Some metadata about the different parts of a single object (such as the scan of each page of a book) will be the same for that entire object. This kind of repeating metadata will not require keyboarding each field for each digital image; instead, they can be handled either through inheritance or by batch-loading various metadata fields. This report attempts to identify best practices for metadata development. Individual repositories will follow these practices to the extent that they can afford.

Tables 3–6 describe elements needed for the creation of a digital master image; identification of the digital image and tools for viewing or using it; linking the parts of a digital object or its instantiations; providing context; and ownership, rights, and reproduction information. Tables 3 and 4 show the type of data that uniquely identify a particular representation of a work. For future derivative images, these could be iteratively nested to represent the provenance of a work.

Encoding: Best Practices

Encoding Archival Object Content and Finding Aids

Many MoA II materials require text encoding. This may be the case whether the documents are carefully transcribed and edited versions of the original documents, whether they simply organize (conceptually) a mass of automatically generated text via optical character recognition (OCR), or whether only the framework of a document is encoded, with pointers to images. Moreover, finding aids for many resources will be encoded to support fine-grained access to a collection. There has been substantial work and community effort in both areas, and efforts are under way to organize discussions around the use of the available guidelines.

Project participants should use the EAD for the encoding of finding aids.⁹ While the EAD guidelines allow considerable latitude for the application of markup to finding aids, work with the EAD must grow out of local assessment of the needs for finding aid support and the way that these finding aids will be used. Discussions are under way in the DLF about interinstitutional searching of EAD-encoded collections and the resulting scrutiny of local practice. Using locally defined needs to drive the application of EAD will help clarify the range of needs for interinstitutional applications.

Text encoding efforts in MoA II will be well supported by the SGML articulated in the Text Encoding Initiative Guidelines (TEI).¹⁰ The TEI support a range of document types and methods, including the transcription of primary sources and damaged documents. More important, the TEI Guidelines and the associated DTDs (document-type descriptors) offer support for encoding the range of possible structures in MoA II documents, whether or not transcriptions are included. The TEI, like the EAD, offers considerable flexibility in the ways documents can be encoded.

A further note on the relevance of XML to these two central encoding schemas may be useful. XML promises to bring richly encoded documents to the user's desktop through widely available browsers. Moreover, a growing array of XML-capable tools should be available through mainstream software development. XML-compliant versions of both the TEI and the EAD DTDs are likely to be available soon. One editor of the TEI guidelines has been centrally involved in writing the XML specifications, and the TEI editors have declared their intention to create XML-compliant versions of the widely used TEI DTDs.

Encoding to Encapsulate Metadata and Content Inside the Archival Object

After this report has been circulated and discussed, the project team will have gathered enough information to define an encoding scheme for the archival objects that will populate the MoA II testbed. A draft of the MoA II XML DTD has been completed and readers are referred to view both the DTD and documentation available at http://sunsite.Berkeley.EDU/MOA2/ (see the section on MoA II Tools). This XML DTD will define the transfer syntax used for MoA II objects. Selecting an XML to encode the object does not mean that any repository must use that encoding for internal object storage. However, it does give the DLF and the larger community an opportunity to discuss and evaluate XML as transfer syntax.

Footnotes

⁵ Generic and specific are difficult to define in this context. The discussions that take place in the MoA II process are anticipated to help define and agree on generic methods.

⁶ The MoA II project will rely primarily on a union catalog to effect discovery. A union catalog obviates problems associated with inter-repository searcheshow one characterizes the search to the various systems and brings together results from those different collections. Nevertheless, the ability to perform a search across a number of distributed repositories becomes increasingly important as we distribute responsibility, while maintaining important elements of institu-tional autonomy. Repository search, and especially the means to support inter-repository search, is discussed here not because it must be explored in MoA II but because it is an important method to keep in mind when conceptualizing the digital library.

⁷ The University of Michigan (UM) uses the terms raw, seared, and cooked to describe levels of processing for the Making of America I materials (www.umdl.umich.edu/moa/). For a discussion of MoA I processing at UM, see www.dlib.org/dlib/july97/america/07shaw.html and various sections in http://www.umdl.umich.edu/moa/about.html.

⁸ For its organization and for many of the elements, this model owes a great deal to the Structural Metadata Dictionary for LC Repository Digital Objects. Located at http://lcweb.loc.gov:8081/ndlint/repository/structmeta.html.

⁹ Information about the EAD, including guidelines for the application of the EAD and DTDs, is available at http://lcweb.loc.gov/ead/.

¹⁰ Information about the TEI can be found at the TEI Consortium home page http://www-tei-c.org/. A searchable version of the TEI Guidelines can be found at the TEI Consortium home page, and via the Humanities Text Initiative pages at http://www.hti.umich.edu/docs/TEI/.

Next     Previous

Return to CLIR Home Page >>