We Welcome Our Email Overlords: Highlights from the Archiving Email Symposium

This post is co-authored with Erin Engle, a Digital Archivist in the Office of Strategic Initiatives.

Despite the occasional death knell claims, email is alive, well and exponentially thriving in many organizations. It’s become an increasingly complex challenge for collecting and memory institutions as we struggle with the same issues: How is email processed differently from other collections? Are there donor issues specific to email? What are the legal or regulations surrounding email records for cultural heritage institutions? Are there standard preservation file formats for email? How can we make email archives available for research?

On June 2, 2015, the Library of Congress and the National Archives and Records Administration co-hosted the Archiving Email Symposium at the Library to share information about the state of practice in accessioning and preserving email messages and related attachments. The approximately 150-person audience included a wide range of practitioners, from technologists and software developers, librarians, curators, records managers, lone arranger archivists and academics, and representatives from large federal agencies with many thousands of employees as well as grant funding programs including the National Endowment for Humanities, Institute for Museum and Library Services and National Historical Publications and Records Commission. In addition, we hosted an informal workshop on June 3 with a subset of participants to discuss issues and challenges identified during the Symposium in order to better define the gaps in our tools, processes and polices for archiving email collections.

In this first post in a series about the event, we’ll cover the overarching themes of the Symposium. Future posts will go into more depth about each of the four perspectives described below, which will include links to webcasts of the presentations, and a summary of the June 3 workshop.

The idea for this project first took root last August when we gathered an informal group of practitioners to share our collective but disparate work to preserve email. This led to the formation of the (again informal) Email Interest Group which initiated a series of online discussions and tool demonstrations from projects including Stanford Library’s ePADD project, Harvard’s Electronic Archiving System, the Smithsonian Institution Archives and the Rockefeller Archive Center coordinated CERP project and more. The strong attendance and engagement during these meetings demonstrated a significant and sustained interest in the multifaceted problems of email preservation from a variety of perspectives including selection, processing, accessioning, format identification and normalization and long-term preservation and use.

Clearly, we were onto something. The “email problem” had legs as the saying goes. Online discussion is great but sometimes, a face-to-face meeting is in order to  investigate more deeply the issues and to network with others working in the same space. As we started working on the agenda, our program committee helped us bring four different perspectives on the email problem into focus. The full agenda (PDF)  lists speakers for each perspective.

• The Technical perspective looked at institutional approaches to processing and archiving email in which presenters discussed the reasons and approaches for the normalization of email archives (or the considerations of when normalization might be appropriate), strategies for PII and other redaction needs, tools for providing patron access, repository needs including ingest requirements, and workflow selections for implementing specific technical email archiving solutions.

• The Archival perspective focused on practical approaches to accessioning and processing email from “boots on the ground” archivists who presented lessons learned, including real life challenges and successes stories, to help participants understand how policies and decision-making practices were applied to accessioning and processing email archives.

• The Records Management perspective considered the challenges of “email as a record” including technological barriers, legal mandates and retention periods.

• The Policy and Guidelines Development perspective included Institutional approaches to how private, public and state government institutions are managing email to not only maximize long-term research value but also to comply with technical, legal, access and intellectual policy issues in processing email archives.
These information-packed sessions were bracketed at the start by welcoming remarks from senior leaders from both hosting institutions and at the end of the day with a thought provoking summary by Chris Prom, assistant university archivist and assistant professor of library administration at the University of Illinois Urbana-Champaign and author of the guide to email preservation (PDF) for the Digital Preservation Coalition’s Technology Watch Report series.

By all accounts (including the Twitter hashtag #ArchEmail), the Symposium was a rousing success. So yes, we welcome our email overlords with open arms. And we should – we are all already under email’s thumb. It’s not going anywhere except into our respective repositories and archives. Let’s continue the conversation so we can learn from each other how to manage these substantial and challenging issues.

Next up in this series on the Archiving Email Symposium, an in-depth look at the institutional approaches to processing and archiving email from the Technical perspective.