FADGI MXF Video Specification Moves Up an Industry-organization Approval Ladder

The following is a guest post by Carl Fleischhauer, who organized the FADGI Audio-Visual Working Group in 2007. Fleischhauer recently retired from the Library of Congress.

The Federal Agencies Digitization Guidelines Initiative Audio-Visual Working Group is pleased to announce a milestone in the development of the AS-07 MXF video-preservation format specification. AS-07 has taken shape under the auspices of a not-for-profit trade group: the Advanced Media Workflow Association. AS-07 is now an official AMWA Proposed Specification, and the current version (CC by SA Creative Commons license and all) has been posted at the AMWA website. Although this writer retired from the Library in April, he helped shepherd the specification through this phase.

FADGI logo

AS-07 is one of three new AMWA specifications announced in June. Another one is the organization’s new process rule book. The new AMWA process is patterned on the Requests for Comment approach used by the Internet Engineering Task Force. In the new AMWA scheme, there are three levels of maturity:

  • Work in Progress
  • Proposed Specification
  • Specification

Two earlier versions of AS-07 were exposed for community comment at the AMWA website, beginning in September 2014, and this met the requirements for a Work in Progress. For more information about the history of AS-07, refer to the FADGI website.

AMWA logo

AS-07 is a standards-based specification. For the most part it is a cookbook recipe for a particular subtype of the MXF standard. MXF stands for Material eXchange Format, and that format’s complex and lengthy set of rules and options is spelled out in more than thirty standards from the Society of Motion Picture and Television Engineers. AS-07 also enumerates a number of permitted encodings and other components, each of which is based on other standards from SMPTE, the International Organization for Standardization and International Electrotechnical Commission, the European Broadcast Union, and special White Paper documents from the British Broadcasting Corporation. It is no wonder that a cookbook recipe is called for!

Why the emphasis on standards? The short answer is that standards underpin interoperability, in the digital world just as surely as they have for, say, the dimensions of railroad tracks, so my boxcar will roll down your rail line. It is worth saying that, in our preservation context, interoperability has both current and future dimensions. Today, cooperating archives may exchange preservation master files and these must be readable by both parties. More important, however, is temporal interoperability: today’s content must be readable by the archive of tomorrow. AS-07’s extensive use of standards-based design supports both types of interoperability.

At a high level, the objectives for video archival master files (aka preservation masters) are like those for the digital preservation reformatting for other categories of content. Archives want their masters to reproduce picture and sound at very high levels of quality. In addition, the preservation masters should be complete and authentic copies of the originals, i.e., in the case of video, they should retain components like multiple timecodes, closed captions and multiple soundtracks. And–back to temporal interoperability–the files must support access by future users.

AS-07 cover page

First page of the AS-07 specification document

What are some of the features of AS-07? The specification emphasizes encodings that ensure the highest possible quality of picture and sound, including requirements for declaring the correct aspect ratio and handling the intricacies of interlaced picture, a characteristic of pre-digital video. Beyond those elements, AS-07 also specifies options for the following:

  • Captions and Subtitles
    • retain and provide carriage for captions and subtitles
    • translate binary-format captions and subtitles to XML Timed Text
  • Audio Track Layout and Labeling
    • provide options for audio track layout and labeling
  • Content integrity
    • provide support for within-file content integrity data
  • Timecodes
    • provide coherent master timecode
    • retain legacy timecode
    • label multiple timecodes
  • Embedding Text-Based and Binary Data
    • provide carriage of supplementary metadata (text-based data)
    • provide carriage of captions and subtitles in the form of Timed Text (text-based data)
    • provide carriage of a manifest (text-based data)
    • provide carriage of still images, documents, EBU STL, etc. (binary data)
  • Language Tagging
    • provide a means to tag Timed Text languages
    • retain language tagging associated with legacy binary caption or subtitle data
    • provide a means to tag soundtrack languages
  • Segmentation
    • provide support for segmented content

AS-07 has not been exclusively developed in writing (“on paper,” in oldspeak). The format is based on pioneering work done by Jim Lindner in the early 2000s, when he developed a system called SAMMA (System for the Automated Migration of Media Archives). SAMMA produces MXF files for which the picture data is encoded as lossless JPEG 2000 frame images. It also operates in a robotic mode, to support high-volume reformatting.

Jim’s design for SAMMA was motivated by the forecasts for high-volume reformatting at the Library’s audio-visual center in Culpeper, Virginia (today’s Packard Campus for Audio-Visual Conservation), which was then in its planning phase. The Packard Campus began operation in 2007 and, since then, more than 160,000 videotapes have been reformatted using the SAMMA system. AS-07 is very much a refinement and elaboration of the SAMMA format. In order to get a better look at those refinements, in 2015, the AS-07 team commissioned the production of custom-made sample files.

What next? The interesting — and I think proper — feature of the new AMWA process concerns the movement from Proposed Specification to Specification. The rulebook lists several bullets as requirements but the gist is this: you gotta have implementation and adoption. AS-07 at this time is, metaphorically, a recipe ready to test in the kitchen. Now it is time to cook and taste the pudding. After there are instances of implementation and adoption, these will be reported to the AMWA board with a request to advance AS-07 to the level of [approved] Specification. (Of course, if the process reveals problems, the specification will be modified.)

The first steps toward implementation are under way. On FADGI’s behalf, the Library has contracted with Audiovisual Preservation Solutions and EVS to assemble additional test files, and to have them reviewed by an outside expert. At the same time, James Snyder, the Senior Systems Administrator at the Packard Campus, is working with vendors to do some actual workups. (James oversees the campus’s use of SAMMA and has been an active AS-07 team member.) We trust that these implementation efforts will bear fruit during the remaining months of 2016.

Demystifying Digital Preservation for the Audiovisual Archiving Community

The following is a guest post by Kathryn Gronsbell, Digital Asset Manager, Carnegie Hall; Shira Peltzman, Digital Archivist, UCLA Library; Ashley Blewer, Applications Developer, NYPL; and Rebecca Fraimow, Archivist and AAPB NDSR Program Coordinator, WGBH. The intersection of digital preservation and audiovisual archiving has reached a tipping point. As the media production and use landscape […]

Avoid Jitter! Measuring the Performance of Audio Analog-to-Digital Converters

The following is a guest post by Carl Fleischhauer, a Project Manager in the National Digital Initiatives unit at the Library of Congress. It’s not for everyone, but I enjoy trying to figure out specialized technical terminology, even at a superficial level. For the last month or two, I have been helping assemble a revision […]

Keeping Our Tools Sharp: Approaching the Annual Review of the Library of Congress Recommended Formats Statement

The following post is by Ted Westervelt, head of acquisitions and cataloging for U.S. Serials in the Arts, Humanities & Sciences section at the Library of Congress. Since first launching its Recommended Formats Statement (then called Recommended Format Specifications in 2014), the Library of Congress has committed to treating it as an important part of […]

Survey: How Do You Approach Web Archiving?

Do you have fifteen minutes to tell the National Digital Stewardship Alliance about your organization’s web archiving activities? If the answer is yes, please contribute to the NDSA Web Archiving Survey. By filling out this short survey, your institution will be part of a multi-year project to track the evolution of web archiving programs in […]

ODF: The Open Document Format

The following is a guest post by Carl Fleischhauer, a Digital Initiatives Project Manager at the Library of Congress. During December 2015, the Library’s Format Sustainability website added descriptions of eleven members of the Open Document Format family, aka OpenDocument and ODF. These eleven join a number of other format descriptions mounted in 2015, many […]

Digitizing Motion Picture Film: FADGI Report on Current Practices and Future Directions

The following is a guest post by Carl Fleischhauer, a Digital Initiatives Project Manager at the Library of Congress. More often than not, the Federal Agencies Digitization Guidelines Initiative Working Groups (one for still images, one for audio-visual) find themselves walking a line between codifying widely adopted practices and exploring new ideas and new technologies […]

Access Historic Audio and Video Programs: AAPB Launches Online Reading Room

The following is a guest post by Karen Cariani, AAPB Project Director and Director WGBH Media Library and Archive, Alan Gevinson, AAPB Project Director and Special Assistant to the Packard Campus Chief, and Casey Davis, Project Manager, American Archive of Public Broadcasting, WGBH Educational Foundation. The American Archive of Public Broadcasting (AAPB) Project Team at […]

Announcing the 2015 Innovation Award Winners

On behalf of the National Digital Stewardship Alliance Innovation Working Group, I am excited to announce the 2015 NDSA Innovation Award winners! This year, the annual innovation awards committee reviewed over thirty exceptional nominations from across the country. Awardees were selected based on how their work or their project’s whose goals or outcomes represent an […]