Assessment
Like their cohorts across the country, the University of California libraries individually and collectively report statistical information about their collections and services to a variety of agencies. Decisions about effective services have long been guided by extensive statistics on collection strengths, on the use of collections and of services such as reference and interlibrary loan, and through comparison with peer institutions on staffing, budgets, foot traffic, and the like.
Indeed, the Association of Research Libraries has been mining such information for more than a century and collecting it in standardized form for more than 40 years.
With changing times — driven by technology, shifting economics, and evolving visions of information management — libraries need to broaden their assessment skills and activities to include deeper understanding of access to their collections and services, the needs of their faculty, student, and other patrons, and the effectiveness of their technologically mixed environments.
While reliably producing traditional statistics, the UC libraries have led the way in many new assessment activities that stress access to print and digital materials, user needs and services, and operational effectiveness.
See sections below:
Print-Based Collections and Services
The UC libraries collect traditional statistics, which include total number of volumes, new volumes added to the collection, serial subscriptions, interlibrary loan transactions, expenditures, and number of staff:
Valid arguments have been posed about relying solely on these traditional "input measures" to assess or rank libraries. Measures of total access to materials, user services, and ability to share and collaborate to provide greater access have been called for to take their appropriate place in assessment programs.
Digital Collections and Desktop Access
The nature of digital collections presents significant challenges for keeping library statistics. Digital collections represent new forms of materials: digital journals, bibliographic and reference databases, digital books, digital images of manuscripts, photographs, maps, art works, museum artifacts, and new forms of "publications" such as genomic databases.
Technological innovations such as faxing, scanning, and the Internet have increasingly made materials from remote locations rapidly available. Furthermore, online databases of library holdings and journal citations allow library patrons to use their desktop computer to navigate from a citation to the full online content, or to immediately request an item in print held at any campus library. UC-eLinks and Request are examples of these innovative services.
Although digital collections and access that begins and ends at a desktop computer do not align well with traditional metrics of library collections and their use, the UC libraries collect data to characterize the materials in and integrating services of the California Digital Library's shared digital collection. To learn more about the scope and use of digital materials, see the CDL Key Indicators.
In the research library community, various efforts are underway to define and collect data on the use and value of electronic resources:
Understanding Patron Needs
Individually and collectively, the UC libraries are increasing their efforts and capacity to understand faculty, student, and staff needs for library and information management services.
Needs assessment and usability studies and methods include:
UC Libraries also participate in ARL's LibQUAL +, meant to "define and measure library service quality across institutions and to create useful quality-assessment tools for local planning."
Participating UC libraries include:
Understanding a Mixed Print and Digital Environment
The UC libraries are sharing the results of the Collection Management Initiative — a research project funded by the Andrew W. Mellon Foundation — to study use and management aspects of collections that comprise both print and digital materials.
The study has examined and documented:
- Behavior and attitudes of users when selected print journals for which electronic access is provided are temporarily relocated elsewhere and primary use is of the electronic version.
- Change in usage of digital and print versions of selected journal titles when print is relocated to storage.
- Costs incurred and avoided for providing electronic access to selected journals when print copies of the journals are relocated to a storage facility and primary use is of the electronic version.
- Institutional implications for library organization and operations, including consultation and decision-making processes, facilities planning, capital budgeting, and systems and resource management.
Emerging Assessment Challenges
The Association of Research Libraries reports that between 1986 and 2002, the number of items borrowed through interlibrary loan at its member universities (more than 100 institutions) has tripled. This trend can be seen at UC, where intercampus borrowing of returnable items has increased by more than 350 percent during the same period. This indicator of increasing interdependence among universities suggests that traditional metrics for evaluating the quality of an academic library are shifting.
Beyond interlibrary loan, which is difficult to measure as a collection asset, shared collections present new challenges in accountability of collections. A pressing issue for UC is how to account for collections for which governance and management are shared Universitywide, such as the digital resources acquired by the CDL.
Simple counts, such as the 8,000 electronic journal titles reported in CDL Key Indicators do not adequately characterize the impact of ubiquitous access to the UC community, or the role of co-investment. Shared digital collections are financed jointly using a combination of central (CDL) funding and co-investment from all campuses, and processed and cataloged by CDL/UC libraries units currently located at UCSD.
Beyond collections, new and shared services will require innovations in assessment. Among the challenges are establishing effectiveness criteria for efforts to reshape scholarly communication by influencing traditional publishing economics and providing alternative methods for sharing scholarship.
In addition, the UC libraries and their cohorts are forging partnerships with instructional technologists to better integrate library resources with learning and teaching technologies. The libraries are well aware that assessment methods, criteria for efficiency and effectiveness, and using assessment results in further developments will be crucial in these endeavors.
|
