You are here

Cobweb: Collaborative Collection Development for Web Archives

Andrea Goethals, Manager of Digital Preservation and Repository Services, shares an update on a new development in web archiving. 


Cobweb is a one-year IMLS-funded project led by the California Digital Library (CDL) partnering with Harvard Library and the UCLA Library. Cobweb will be an open-source collaborative collection development platform for creating comprehensive web archives by coordinating the activities of the web archiving community.

Much of the technical infrastructure for collecting the web already exists. Cobweb will add the missing essential piece to help libraries and archives decide what to collect. Cobweb will provide public transparency for websites that will be or already have been collected, giving curators the knowledge they need to build archival collections that fill gaps and complement the collections of other institutions, leading to more efficient use of resources and more comprehensive collections for researchers. The shared curatorial decision-making that Cobweb enables will benefit libraries and archives that already engage in web archiving as well as those that have not yet started. For researchers and other users, Cobweb will improve discovery of relevant archived websites and allow them to participate directly in deciding what to collect.

The Cobweb platform will support three key functions: nomination, claiming, and holdings, all performed in the context of thematic-collecting projects. The nomination function will allow any interested stakeholder to suggest URLs that relate to a project theme. The claiming function will allow archival programs to indicate an intention to capture some subset of the nominated URLs. The holdings function will allow those programs to specify what material has actually been captured, along with the location and terms under which it is accessible. Cobweb will be a centralized aggregated catalog of web collection metadata; the actual web content will remain hosted and accessible by the individual institutions that have captured it.

The project consists of coordinated and complementary activities led by each of the partners: developing the open-source Cobweb system (led by UCLA); user interface design and usability testing (led by Harvard); and community outreach, engagement and production deployment (led by CDL).

 This project was made possible in part by the Institute of Museum and Library Services, #LG-70-16-0093-16.

By Andrea Goethals, Manager of Digital Preservation and Repository Services. 

Published on November 2, 2016.