CLIR Issues Number 93
Number 93 • May/June 2013
ISSN 1944-7639 (online version)
Taking our Measure: The Correlation of Metrics and Leadership
Report Examines Policies for Open Access to Images of Museum Works
Andrew W. Mellon Foundation Awards CLIR $1.55 Million Operating Grant
CLIR Seeks Curriculum and Research Strategist
Looking for a Job? Want to Post a Job?
Welcome to New CLIR Sponsors and DLF Members
We invite you to check out our blog series, The weekly blog features perspectives from a variety of contributors on topics relating to the emerging digital environment, research, and higher education. In this week's blog, DLF Program Director Rachel Frick reports on the LODLAM 2 conference and what made it remarkable.
CLIR Issues is produced in electronic format only. To receive the newsletter electronically, please sign up at http://www.clir.org/pubs/issues/signup.html. Content is not copyrighted and can be freely distributed.
Follow us on Twitter @CLIRNews
by Charles Henry
A few years ago, Robert Crease wrote an article in the New York Times, “Measurement and its Discontents,” which discussed our long, often exasperating history of trying to accurately measure the objects of our environment: phenomena that can be weighed, sized, and easily compared.1 The article also described the difficulty of trying to assess and measure more pliable and nuanced entities, such as intelligence, skills, and the qualities we esteem. In philosophy, Crease notes, the distinction is often referred to as ontic versus ontological measurement, and we appear to be mixing these approaches to our detriment. “Confusing the two ways of measuring seems to be a characteristic of modern life. As the modern world has perfected its ontic measures, our ability to measure ourselves ontologically seems to have diminished. We look away from what we are measuring, and why we are measuring, and fixate on the measuring itself. We are tempted to seek all meaning in ontic measuring—and it’s no surprise that this ultimately leaves us disappointed and frustrated, drowned in carefully calibrated details.”
Measuring is parcel to higher education and librarianship. We have lived with and by statistical counts for decades: our citation indices can determine an individual’s promotion and tenure just as annual assessments of library acquisitions, expenditures, and subscriptions can determine an institution’s ranking. These numbers, in turn, can influence a scholar’s future choices or an organization’s strategic planning. In this respect metrics are not simply a capture of past activity, but can have a contributory causal effect on future actions as well.
It is this causal effect that deeply interests me. CLIR, as a national organization committed to enhance research, teaching, and learning environments, sponsors several programs focused on leadership—programs that identify and foster the skills and qualities requisite to enhance those environments. These include the Leading Change Institute, the Postdoctoral Fellowships in Academic Libraries, and the Rick Peterson Fellowship. From CLIR’s perspective, metrics and leadership are correlate, and profoundly so: a profession content with an arithmetic counting of its assets or numbering of its citations will value and promote individuals who are comfortable with these assessment methodologies. The type of people we choose to elevate and agree to follow can be extrapolated in part from the means by which we measure our resources. Simply put, metrics are determinative of a particular kind of leadership. The philosophical tradition of drawing a hard line between ontic and ontological is misleading: the ontic has a great deal to say about the ontological self; the lens by which we calibrate our efficacy is a mirror as well.
Not surprisingly, the need to rethink the way certain disciplines in higher education measure performance and influence has gained momentum of late. This is in part a response to a widening acknowledgement that traditional methods of counting do not adequately capture the more complex aspects of knowledge discovery and reuse in a digital era. For many decades the sciences could seemingly quantify the uptake and thereby the inferred influence and strength of a publication in a given field by enumerating its subsequent citations. Academic libraries could similarly satisfy their performance mandate by pointing to a steady increase in acquisitions and subscriptions, with often a concomitant rise in budgetary spending. Today, these metrics are under serious scrutiny.
A recent article in the Chronicle of Higher Education described the current tension between traditional and alternative metrics (usually shortened to altmetrics).2 A succinct source for gleaning the frustrations of the older measuring schemas and the new is the Altmetrics Manifesto.3 The authors cite new aspects of scientific research and new means of communication as requiring new approaches to assessing and gauging the importance of a particular article or publication. Conceptualizing the product of research as a fixed object usually published in paper form or designed to fit a paper-based publication model, the manifesto argues for considering other forms of expression and background information used in the research as equally important. Data sets, raw data, and code are integral to a researcher’s argument but are currently difficult to access, let alone assess. Nanopublication, or semantic publication, in which a single quote or selected passages are incorporated into a new research work, can be essential to the new research, but the use of short passages is presently not monitored or valued. The proliferation of blogs, annotations, comments, and other forms of what the manifesto defines as self-publication also carry considerable intellectual weight and influence, but again have been considered marginal and not integrated into an assessment of a scientist’s influence.
In the field of academic library assessment, alternative metrics have appeared over the last decade or so. LibQual+ began in 2000 in response to the perceived limitations of the Association of Research Libraries’ annual statistics compilation for analyzing the quality of library services, and strove for a more nuanced assessment of such programs and services. LibQual+, however, also relies on a quantitative scale when finally tabulating the results of interviews and conversations. MISO (Measuring Information Services Outcomes) is a promising method for measuring the efficacy of college library and IT operations that has been used for several years with considerable success. Its rigorous, more qualitative set of metrics has enjoyed wide adoption.
When viewed from within the disruptive churn of digital technology that touches nearly every facet of our workplace, and acknowledging the powerful tools and applications at hand that can discern far more subtle patterns and connections within the commonwealth of knowledge, an assessment of many of our current methodologies is unambiguous. The long tradition of collecting numerical statistics—the counting of books, journals, digital objects, subscriptions, datasets, staff, and dollars—is inadequate to capture the dynamic response of modern research libraries at the analog-digital boundary. This is an unprecedented historical circumstance. The compilation of statistics presents a static, nineteenth-century formula for silos of physical mass, a construct that essentially pits libraries in a counterproductive competition, works against collaboration, encourages redundancy and waste, and fosters disengagement with the wider world of accelerating change, impeding the kind of leadership requisite for our contemporary conditions. The continued reliance on baseline statistics can also mask a lack of innovation and strategic engagement of institutions.
A related, fundamental problem is the interpretation of these statistics. For decades, these routine compilations have been regarded as performance metrics: they were used to infer institutional standing, accepted as indicators of progress (or lack of progress), and employed as benchmarks when setting goals. Yet, in a world of powerful, networked resources in which information created every three weeks is equivalent to the previous three thousand years of recorded history, the only relevant performance captured by these statistics is the counting itself.
New standards of performance and new approaches to measurement need to be adopted. A research library's functional categories should be deconstructed, exposing its component parts. These elements can then be recontextualized within a nest of interrelationships that form the core of the library's purpose. These connections include the relationship of a library’s internal components to the library as a whole; its relationship to the university as defined by its response to the primary constituents it serves; its response to trends and influences in research beyond the campus; its ability to meaningfully partner and collaborate; and the methods and means that the library uses to discover and appropriate resources, tools, and ideas from the wider world. One way to visualize this is to bring to life the interconnections that lie invisible in the white spaces between the rows of our currently sleeping numbers.
For the professionals who will analyze and parse the information generated from new metrics, an ability to extrapolate a successful strategy from these levels of interplay is also paramount. In this way the questions that help construct a new benchmarking schema can help frame more accurately and honestly the new responsibilities, mission, and strategic evolution of contemporary research libraries. Using new measuring tools, new leadership can be fashioned.
1 Crease, Robert P. 2011. Measurement and its Discontents. The New York Times Sunday Review, October 11. Available at http://www.nytimes.com/2011/10/23/opinion/sunday/measurement-and-its-discontents.html?_r=1&
2 Howard, Jennifer. 2013. Rise of ‘Altmetrics’ Revives Questions About How to Measure Impact of Research. The Chronicle of Higher Education, June 19. Available at http://chronicle.com/article/Rise-of-Altmetrics-Revives/139557/?cid=at&utm_source=at&utm_medium=en
Art museums have long controlled the images of objects in their collections by charging fees for their use. In recent years, however, several art museums in the United States and United Kingdom have adopted policies permitting more open access to these images.
A new report, prepared for The Andrew W. Mellon Foundation and published by CLIR, describes the current approaches of 11 art museums to the use of images from their collections, when the underlying works are in the public domain. The report, Images of Works of Art in Museum Collections: The Experience of Open Access, was written by Kristin Kelly. Ms. Kelly, a freelance museum professional and writer, spent nine years as the manager of administration at the J. Paul Getty Museum, and nine years overseeing public programming and communications at the Getty Conservation Institute.
The author found that each of the eleven museums studied has taken a slightly different approach to making images of the works in its collection more openly accessible. Some have put high-resolution digital files of works online for use by anyone for any purpose. Others have established a “fee and free” system that considers whether use is for commercial use or for promotion of scholarship. Still others evaluate each request on its individual merits. By presenting the thought processes and methods used in these institutions, the report aims to inform the decision making of other museums that are considering open access to images in their collections.
Among the report's key findings:
- Providing open access is a mission-driven decision. Every staff member of each museum in the study emphasized that museums exist to educate and serve their audiences, and that providing access to images of works in their collection is part of their institutional mission.
- Different museums look at open access in different ways. Some museums have the technological, financial, and human resources to provide free, immediate, high-quality downloads of collection images, while others are taking the process in steps as resources and time permit.
- Internal process is important. The decision to provide open access to images can affect many people in a museum. Each is a stakeholder in the process, and each needs to understand and participate in the decision making. Senior-level commitment is critical.
- Loss of control fades as a concern. While many museum staff had legitimate questions and concerns about providing open access to images of works in their collection, their worst fears have not been realized. Several of the museums are part of the Google Art Project, or have contributed to Wikimedia or other social media sites, which means that images of many of their works are already available online.
- Technology matters. While a decision to provide open access to images is not based solely on the available technology, it is important to have clean and complete metadata, an effective digital asset management system, generally solid museum technology, and the staff to manage all of these systems.
- Revenue matters less than many institutions think it does. While revenue remains a topic of interest to many museums, staff generally acknowledge that their desire to provide information about the collection in as open a manner as possible trumps revenue concerns.
- Change is good. No museum that has made the transition to open access for the images in its collection would return to its previous approach.
“The report's findings are compelling,” said CLIR President Chuck Henry. “From the interviews with staff and analysis of data, important changes over time can be noted in the museum community. These findings, among many other insights offered in the report, are of considerable value in helping us refine our strategies of access moving forward.”
The report is available as a PDF download at http://www.clir.org/pubs/reports/pub157/pub157abst.
CLIR has received a 24-month, $1.55 million grant from The Andrew W. Mellon Foundation to support general operations starting July 2013.
“We are deeply grateful to the Mellon Foundation for this award, which is testimony to the value of our programs and will help ensure that CLIR’s investments of time and resources in new and successful activities can continue to be realized for the benefit of the communities we serve,” said CLIR Board Chairman Herman Pabbruwe.
“CLIR aspires to help build a new digital environment for teaching and research,” said CLIR President Chuck Henry. “This includes creating a new paradigm for publishing and preserving digital humanities, and evolving a new profession that will support, with consistent standards and protocols, academic data at a very large scale. We thank the Mellon Foundation for supporting our efforts toward these goals.”
In the next two years, CLIR will commission research on large-scale technology architecture and templates for interoperable datasets, and on the economic efficiencies of broad-based collaboration. CLIR will continue to seek and cultivate partners from the scholarly, library, information, and administrative spheres to develop methods, guidelines, and recommendations that will allow academic leaders to create sustainable communities of practice.
CLIR has named eleven recipients of CLIR Postdoctoral Fellowships in Academic Libraries, five recipients of CLIR/DLF Postdoctoral Fellowships in Data Curation for the Sciences and Social Sciences, and five recipients of CLIR/DLF Postdoctoral Fellowships in Data Curation for Medieval Studies.
Fellowships are awarded to individuals who recently received a Ph.D. degree in the humanities, social sciences, or sciences. Nine fellows from the previous cohort, including seven Postdoctoral Fellows in Data Curation for the Sciences and Social Sciences, are spending a second year at their host institutions.
The CLIR/DLF Postdoctoral Fellowships in Data Curation for Medieval Studies, launched this year with funding from The Andrew W. Mellon Foundation, provide recent Ph.D.s with professional development, education, and training opportunities in data curation for Medieval Studies. Through this program, CLIR seeks to raise awareness and build capacity for sound data management practice throughout the academy.
All Postdoctoral Fellows will begin their program at an orientation seminar at Bryn Mawr College from July 28 to August 6, 2013. The seminar will introduce fellows to issues facing twenty-first-century libraries and provide an opportunity for fellows to meet others in their cohort who can share experiences and information. Portions of the seminar are devoted to discussing data curation and management.
CLIR administers the fellowship program in collaboration with academic institutions as a means of recruiting talent into the library profession. Information on the fellowships is available at http://www.clir.org/fellowships/postdoc.
Ph.D. Psychology, University of New Mexico
Host: University of Michigan
Ph.D. Comparative Literature, University of Michigan
Host: University of Illinois at Urbana-Champaign/HathiTrust Research Center
Ph.D. Medieval Studies, University of Toronto
Host: University of Toronto
Ph.D. Neuroscience, Tulane University
Host: University of California, Davis
Ph.D. English, Emory University
Host: University of Alabama
Ph.D. English with Certificate in Digital Humanities, Texas A&M University
Host: North Carolina State University
Ph.D. Archaeology, University of Exeter, England
Host: Arizona State University
Ph.D. Philosophy, Temple University
Host: Villanova University
Ph.D. Biological Sciences, Columbia University
Host: California Digital Library
Ph.D. Educational Computing, University of North Texas
Host: University of North Texas
Ph.D. Business Administration, Western University, Canada
Host: Harvard Business School
Ph.D. Institute of Liberal Arts at Emory University
Host: University of Pennsylvania
Ph.D. Anthropology, Temple University
Host: University of Alberta
Ph.D. Medieval Studies, University of Exeter, England
Host: Johns Hopkins University
Ph.D. Comparative Studies in Discourse and Society, University of Minnesota
Host: University of Minnesota
Ph.D. Anthropology, Stony Brook University
Host: University of Notre Dame
Ph.D. Anthropology, Arizona State University
Host: University of Colorado at Boulder/National Snow and Ice Data Center
Ph.D. History, Harvard University
Host: University of Texas-Austin
Ph.D. English Language and Literature, University of Michigan
Host: University of Alabama
Ph.D. English Literature, Stanford University
Host: Stanford University
Ph.D. Polymer Science and Engineering, University of Massachusetts, Amherst
Host: California Institute of Technology
CONTINUING CURRENT FELLOWS
B. Dewayne Branch
Ph.D. Educational Research and Policy Analysis, North Carolina State University
Host: Purdue University
Ph.D. Musicology, University of California, Los Angeles
Ph.D. Geography and Earth Sciences, McMaster University
Host: McMaster University
Ph.D. Cellular and Molecular Biology, University of Wisconsin, Madison
Host: University of California, Los Angeles
Ph.D. Information Science, Indiana University
Host: Indiana University
Natsuko Hayashi Nicholls
Ph.D. Political Science, University of Michigan
Host: University of Michigan
Fe Consolacion Sferdean
Ph.D. Chemistry, University of Massachusetts Amherst
Host: University of Michigan
Ph.D. Geological Sciences, University of North Carolina at Chapel Hill
Host: Lehigh University
Ph.D. Economics, McMaster University
Host: McMaster University
CLIR seeks candidates for a newly created position of Curriculum and Research Strategist. The Curriculum and Research Strategist will be responsible for coordinating the planning, design, and delivery of online and face-to-face education and training across CLIR’s programs, including the activities of the Digital Library Federation Program. He or she will work principally on the Postdoctoral Fellowship Program and the DuraSpace/ARL/DLF E-Science Institute.
More information about the position and a link to the application is available at http://www.clir.org/about/news/curriculum-and-research-strategist.
ArchivesSpace Developer, HathiTrust Print Holdings Database Developer, Gray Chair for Innovative Library Services at Oregon State University… these are just a few jobs posted in the past week on CLIR’s Job Connect, a free job-posting space at http://connect.clir.org/Home/. Job seekers can get real-time updates by joining the Job Connect community. If you want to post a position and are already a CLIR Connect user, just sign in with your username and password. If you are new to CLIR Connect, it takes only a minute to create an account and sign in. (Note that job postings are moderated, so they will not appear instantaneously.)
New CLIR Sponsors:
Folger Shakespeare Library
Montana State University
Simon Fraser University
University of Guelph
University of Houston
University of Nevada, Reno
New DLF Members:
California Institute of Technology
Georgia Institute of Technology
Montana State University
Oregon State University
University of Georgia
University of Houston
University of Wisconsin, Madison