February 4, 2014
I wanted to update you on MOU results for FY13 and share next steps for developing FY15 MOUs.
For the FY13 MOU results and lessons learned, please see a memo I sent to administrative deans, library directors and the Library Leadership Team in the fall below.
This week, in response to the helpful conversations held in the fall, I started to form a working group to develop the FY15 MOUs and tasked the Shared Service Heads (Franziska Frey, Matthew Sheehy and Scott Wicks) with creating a process that will allow us to finalize the FY15 MOUs by May 2014. We will share updates on this work regularly as it progresses.
Vice President for the Harvard Library and
Roy E. Larsen Librarian of the Faculty of Arts and Sciences
To: Administrative Deans, Library Directors, Library Leadership Team
From: Sarah Thomas, Vice President for the Harvard Library and Roy E. Larsen Librarian of the Faculty of Arts and Sciences
Date: October 17, 2013
Re: Library Memoranda of Understanding: FY13 Results and Next Steps
As part of the creation of the Shared Services of the Harvard Library, memoranda of understanding (MOUs) were agreed between the Harvard Library and the Schools to which services were being provided via the operations of the Shared Services. These MOUs were intended to be living documents that outline the basic provisions of library services and set up a framework for how the Shared Services work together with the School libraries. Now that the MOUs have been operational for a year, it is appropriate to review progress and consider possible next steps.
Earlier this summer, following informal discussions by Deputy Provost Peggy Newell with administrative deans and others, a decision was made to "pause" further development of the MOUs. I asked the Shared Service heads to continue with the existing MOUs, and to make common sense adjustments if there are significant variations in needs and expectations.
Background of MOUs: In the spring and early summer of 2012, during the creation of the Shared Services, the MOUs were formed from discussions between the executive director of the Harvard Library, library directors and School administrative deans. They were intended as brief documents, as similar as possible, outlining the most basic elements of the relationship between the Shared Service heads and library directors and their respective management teams, and they served as a starting point for a new relationship. Seventy-two meetings were held with senior leaders to develop the MOUs.
The MOUs specify a base level of service with targets, based on estimated outputs in FY11. The metrics were created under time pressure, with an emphasis on pragmatism and achievability. It was agreed that progress along these metrics would be discussed with each School twice annually as a part of a broader review of the relationship between the Shared Services and the School libraries. The first reviews were held in spring 2013. We welcome your input on how best to review the FY13 year end results and will be seeking your feedback at an upcoming Library Leadership Team meeting and the November executive vice president's meeting.
When reviewing progress against the targets in the MOUs, several factors should be borne in mind:
- Different types of measurements: Some metrics (e.g. maintaining opening hours) are largely within the control of a single Shared Service department. Others reflect decisions and actions of more than one department. For such metrics, if actuals are not at 100%, the reasons behind it may or may not be within the control of the specific department.
- Measurement limitations: MOU targets are often pragmatic, measurable proxies for arguably richer, more meaningful (but more difficult to agree on and obtain) performance metrics. For this reason, the MOUs do not yet contain qualitative data. The MOU measurements are far from ideal or complete, and must be viewed as part of a larger whole.
- Imperfect methods of measurement: Some metrics can be obtained by querying objective library information systems. Others are done by self-reporting. Some measurements are still being developed. Over time, the Shared Services are refining systems to generate more complete, accurate and objective data. Until then, it is worth discussing the methods of measurement and what is, and is not, contained within them.
- Imperfect targets: The executive director developed the FY13 targets by asking library staff to report FY11 figures and summing these across libraries. In some cases these reported figures were based on actual performance; in others the targets amalgamated different methods of counting; and in others, they were estimates or even aspirations. The sums were not reconciled with aggregate totals or capacities. If the performance does not match the targets, in some instances it is more a reflection of the target than of the performance.
FY13 MOU Results
With the above as a background, a summary of the FY13 performance along MOUs is as follows:
Access Services: Access Services is performing largely consistently with MOU targets. They have opened and staffed library desks close to 100% of the agreed hours, and in some cases provided additional support for libraries. Interlibrary loan processing time is well within the agreed 1-2 days and has been steadily decreasing throughout the year. Similarly, Borrow Direct—a new service for Harvard—has operated with less than a day’s processing time, well within the 1-2 days metric. Scan and Deliver processing time exceeds the 1-2 days agreed. Prior to the MOUs, Scan and Deliver processing time was 3-4 days. Access Services is performing within these historical averages with volumes that are up 33% over last year. The 1-2 day target may have been too ambitious; nonetheless, the turnaround time has reduced from historical levels and Access Services is working to improve it further.
Numbers for HD accessions are not specified in the MOUs. Rather, Access Services commits to accommodating forecast demand, which it did. Accessions numbers for FY13 were lower than in the recent past. This may be the result of the Transition; budget uncertainties led to caution in hiring students to prepare materials for the Depository. We have now hired students and that aspect of staffing is proceeding according to plan. However, projections for the coming years show no decline in demand for space.
|Chart 1: Access Services FY13: MOU Targets vs. Actuals|
|Hours Opened||Actual library hours opened/Committed library hours opened (%)||100%||100%|
|ILL Borrowing Request||Time between request submission and forwarding to potential lenders (days)||1-2||0.8|
|Scan and Deliver||Time between request submission and delivery (days)||1-2||3.1|
|Borrow Direct||Time between item receipt and delivery to hold shelf (days)||1-2||0.8|
Information and Technical Services (ITS): Owing to the variety and number of distinct practices, it is challenging to get an accurate picture of operational performance from the current MOU metrics and systems. The first metric below, Titles Acquired/Cataloged, reflects the number of items cataloged in HOLLIS by ITS. It blurs together measures of the volumes the Schools requested for acquisition, the number or percent of such requests filled by ITS and the speed with which ITS received and cataloged these requests. ITS-processed volumes were 81% of MOU volume targets, though on closer examination, some of the libraries’ targets exceed historic performance. There was no growth in processing backlogs, which suggests that the underlying goal of matching acquisitions with bibliographic records was met for the year.
Challenged by a strong pull to maintain distinct and distributed workflows, ITS is still working out the operational means to measure the next two metrics: rush orders and serials check-in. Rush orders can be tracked in the FAS libraries. Here, rush orders processed greatly exceeded MOU commitments. For the majority of accounts, effective with FY14, staff members use uniform coding to track rush orders. We anticipate full code implementation by the 3rd quarter of FY14. Serials check-in reported numbers that are substantially under MOU targets. This gap is driven by inconsistent serials check-in methods and tools. ITS is working towards uniform practices. Nonetheless, there are no backlogs of check-in.
|Chart 2: Information and Technical Services FY13: MOU Targets vs. Actuals|
|Titles Acquired/Cataloged||Number of items cataloged in Aleph (#)||186,017||150,509 (81%)|
|Rush Orders||Number of rush orders processed (#)||4528||5,723* (126%)
(256% counting reporting libraries only)
|Serials Check-In||Number of serial pieces checked in (#)||186,447||16,394** (9%)|
*This is an understatement, as methods to capture this figure are not available in all libraries.
**This is a dramatic understatement, as methods to capture this figure are not available in most libraries.
Preservation, Conservation and Digital Imaging (PCDI): PCDI is performing largely consistently with MOU targets. Each of the four areas in PCDI has different tracking mechanism for MOUs. Special Collections is measured in hours. In FY13, special collections treatments exceeded MOU agreements. Collections Care, or treatment to the general collections, is measured in items treated and (separately) in items bound. In FY13, Collections Care treated 90% of MOU volumes, and bound 85% of MOU volumes. Media Preservation and Conservation has only one MOU, with the Music Library. It completed 100% of the MOU volume. There are no specified MOU volumes for digital imaging, although for some libraries the MOU specifies continuation of past volumes. That said, digital imaging volumes were within expectations.
|Chart 3: Preservation, Conservation and Digital Imaging FY13: MOU Targets vs. Actuals|
|Special Collections||Special Collections Treatment (hours)||12,448||13,415 (107%)|
|Collections Care||General Collections Treatment (# items)||34,405||31,012 (90%)|
|Binding||Items Bound (# items)||36,225||30,636 (85%)|
Lessons Learned and Possible Next Steps
A year into the MOU's, we offer the below observations and ideas for future consideration:
1. No one set of metrics can convey the full contribution of the services. Some services are not measured in the MOUs. Other services were not anticipated in the MOUs, but subsequently requested by Schools and delivered by the Shared Services. It is possible that neither quantitative nor qualitative data can convey the richness of the benefits. Therefore:
- The metrics should continue to serve as one of many means to an end: working together for a common aim.
- Given that any measurement and communication systems will be time-consuming, continuous assessment between the benefit of the information provided and the cost of collecting and disseminating it should occur.
- Metrics should be refined over time. As the ability to collect meaningful data on the operations of the Shared Services grows, the metrics chosen should be adapted. At the same time, using a few metrics in each area is optimal, so that clarity and focus persist.
2. Libraries and Schools value flexibility with MOUs. FAS seems interested in shifting amounts from one library to another. Other libraries asked if they could shift volumes from one service to another, or for new services altogether. Building in some flexibility may be helpful. Specifically:
- Assess whether the agreements should continue by library or whether the University would be better served with agreements by School. The latter would allow FAS to more easily treat its collections as one (e.g., by reviewing preservation needs across its collection rather than with targets for each of its libraries) and more closely align with funding models.
- Consider whether some of the targets may be best expressed in ranges, to account for variability in the complexity of work (e.g., it takes much more time to catalog a book originally than it does to copy catalog) and to facilitate shifting among services.
3. MOUs were developed independently of financial and operational analysis. This led to inconsistencies between funding and service levels and tensions between service provision and costs. Going forward:
- Targets can and should be refined based on historic actual operational performance, Shared Service capacities, School benchmarks and developing library priorities.
- Closer and more deliberate links among the Shared Services’ targets, with School libraries’ plans and with finance will ensure that the targets are internally coherent.
Additionally, because communication of the MOUs had been independent of finance, the relationship between MOUs and the library funding model was not always made explicit, i.e., some library senior staff were not told that the library is not funded on a fee-for-service model but rather is based off of allocated total costs. In absence of this conversation, there is an understandable misperception of what the MOUs entail.
4. The existence of MOUs creates a mystique with staff. Staff members reported that concerns about individual performance measurements and possible punitive actions were heightened by the MOUs. Library directors and administrative deans had differing views on the appropriate level of transparency of School targets. Resultant inconsistent communications may have enhanced confusion and suspicions during the transition. A clear communications plan, with the full agreement of the Schools, is an important part of any next steps for the MOUs.
5. MOUs may send a signal of non-parity in the relationship, and might impair collaboration between the Shared Services staff and School library staff. MOU measurement is one-way, without an opportunity to review how Schools incorporated Shared Service support. In some instances, MOUs may have encouraged conversations focused on identifying/measuring problems rather than joint solution-finding. A greater emphasis on two-way conversation and working together at the right organizational level to develop real-time solutions outside of formal MOU reviews may enhance service provision. A conflict resolution section was included in the MOUs. In some, but not all, cases this was used as an effective first measure. Introducing additional steps to encourage informal resolution, (e.g., setting the expectation that anything escalated to the formal MOU review has already been attempted to be resolved through informal channels), are worth consideration.
In sum, we have concluded the first year of operation of Shared Services under MOUs. On the whole, targets were met. There were, and continue to be, some areas of the transition that have gone less smoothly, but at the same time, the Harvard Library undertook a massive and very complex realignment, and Harvard's libraries have continued to provide helpful and invaluable services for users. As we continue our development in our second year, we look for ways to provide cost-effective service at a level commensurate with standards for our great Library and the Harvard community. Your contribution to the assessment of FY13 and the evolution in FY14 is much appreciated.
I look forward to discussing the role and substance of the MOUs with you in the coming weeks.