We Welcome Our Email Overlords: Highlights from the Archiving Email Symposium

This post is co-authored with Erin Engle, a Digital Archivist in the Office of Strategic Initiatives.

Despite the occasional death knell claims, email is alive, well and exponentially thriving in many organizations. It’s become an increasingly complex challenge for collecting and memory institutions as we struggle with the same issues: How is email processed differently from other collections? Are there donor issues specific to email? What are the legal or regulations surrounding email records for cultural heritage institutions? Are there standard preservation file formats for email? How can we make email archives available for research?


Archiving Email Symposium. Photo courtesy of Erin Engle.

On June 2, 2015, the Library of Congress and the National Archives and Records Administration co-hosted the Archiving Email Symposium at the Library to share information about the state of practice in accessioning and preserving email messages and related attachments. The approximately 150-person audience included a wide range of practitioners, from technologists and software developers, librarians, curators, records managers, lone arranger archivists and academics, and representatives from large federal agencies with many thousands of employees as well as grant funding programs including the National Endowment for Humanities, Institute for Museum and Library Services and National Historical Publications and Records Commission. In addition, we hosted an informal workshop on June 3 with a subset of participants to discuss issues and challenges identified during the Symposium in order to better define the gaps in our tools, processes and polices for archiving email collections.

In this first post in a series about the event, we’ll cover the overarching themes of the Symposium. Future posts will go into more depth about each of the four perspectives described below, which will include links to webcasts of the presentations, and a summary of the June 3 workshop.

The idea for this project first took root last August when we gathered an informal group of practitioners to share our collective but disparate work to preserve email. This led to the formation of the (again informal) Email Interest Group which initiated a series of online discussions and tool demonstrations from projects including Stanford Library’s ePADD project, Harvard’s Electronic Archiving System, the Smithsonian Institution Archives and the Rockefeller Archive Center coordinated CERP project and more. The strong attendance and engagement during these meetings demonstrated a significant and sustained interest in the multifaceted problems of email preservation from a variety of perspectives including selection, processing, accessioning, format identification and normalization and long-term preservation and use.

Clearly, we were onto something. The “email problem” had legs as the saying goes. Online discussion is great but sometimes, a face-to-face meeting is in order to  investigate more deeply the issues and to network with others working in the same space. As we started working on the agenda, our program committee helped us bring four different perspectives on the email problem into focus. The full agenda (PDF)  lists speakers for each perspective.AES_Welcome

• The Technical perspective looked at institutional approaches to processing and archiving email in which presenters discussed the reasons and approaches for the normalization of email archives (or the considerations of when normalization might be appropriate), strategies for PII and other redaction needs, tools for providing patron access, repository needs including ingest requirements, and workflow selections for implementing specific technical email archiving solutions.

• The Archival perspective focused on practical approaches to accessioning and processing email from “boots on the ground” archivists who presented lessons learned, including real life challenges and successes stories, to help participants understand how policies and decision-making practices were applied to accessioning and processing email archives.

• The Records Management perspective considered the challenges of “email as a record” including technological barriers, legal mandates and retention periods.

• The Policy and Guidelines Development perspective included Institutional approaches to how private, public and state government institutions are managing email to not only maximize long-term research value but also to comply with technical, legal, access and intellectual policy issues in processing email archives.
These information-packed sessions were bracketed at the start by welcoming remarks from senior leaders from both hosting institutions and at the end of the day with a thought provoking summary by Chris Prom, assistant university archivist and assistant professor of library administration at the University of Illinois Urbana-Champaign and author of the guide to email preservation (PDF) for the Digital Preservation Coalition’s Technology Watch Report series.

By all accounts (including the Twitter hashtag #ArchEmail), the Symposium was a rousing success. So yes, we welcome our email overlords with open arms. And we should – we are all already under email’s thumb. It’s not going anywhere except into our respective repositories and archives. Let’s continue the conversation so we can learn from each other how to manage these substantial and challenging issues.

Next up in this series on the Archiving Email Symposium, an in-depth look at the institutional approaches to processing and archiving email from the Technical perspective.

We Did All That? NDSA Standards and Practices Working Group Project Recaps

The end of the school year often finds me thinking about time gone by. What did I work on and what can I show for it? The NDSA Standards and Practices Working Group members were in the same frame of mind so we recently did a survey of our projects and accomplishments since the NDSA […]

Digital Forensics and Digital Preservation: An Interview with Kam Woods of BitCurator.

We’ve written about the BitCurator project a number of times, but the project has recently entered a new phase and it’s a great time to check in again. The BitCurator Access project began in October 2014 with funding through the Mellon Foundation. BitCurator Access is building on the original BitCurator project to develop open-source software […]

Nominations Now Open for the 2015 NDSA Innovation Awards

The National Digital Stewardship Alliance Innovation Working Group is proud to open the nominations for the 2015 NDSA Innovation Awards. As a diverse membership group with a shared commitment to digital preservation, the NDSA understands the importance of innovation and risk-taking in developing and supporting a broad range of successful digital preservation activities. These awards […]

Users, Use Cases and Adapting Web Archiving to Achieve Better Results

The following is a guest post from Michael Neubert, a Supervisory Digital Projects Specialist at the Library of Congress. In a blog post about six months ago I wrote about how the Library of Congress web archiving program was starting to harvest “general” internet news sites such as Daily Kos, Huffington Post and Townhall.com, as […]

One Size Does Not Always Fit All

The following is a guest post from Michael Neubert, a Supervisory Digital Projects Specialist at the Library of Congress. Recently, I talked with Kristen Regina, Head of Archives and Special Collections at the Hillwood Estate, Museum and Gardens in northwest Washington and Jaime McCurry, Digital Assets Librarian, about workflows and issues for web archiving, an […]

Checking in with NGAC and the National Spatial Data Infrastructure

Several times a year I attend meetings of the National Geospatial Advisory Committee, a federal advisory committee that reports to the chair of the Federal Geographic Data Committee. The NGAC pulls together participants from across academia, the private sector and all levels of government to advise the Federal government on geospatial policy and ways to […]

Digital Audio Preservation at MIT: an NDSR Project Update

The following is a guest post by Tricia Patterson, National Digital Stewardship Resident at MIT Libraries This month marks the mid-way point of my National Digital Stewardship Residency at MIT Libraries, a temporal vantage point that allows me to reflect triumphantly on what has been achieved so far and peer fearlessly ahead at all that […]