The following is a guest post by Carl Fleischhauer, who organized the FADGI Audio-Visual Working Group in 2007. Fleischhauer recently retired from the Library of Congress.
The Federal Agencies Digitization Guidelines Initiative Audio-Visual Working Group is pleased to announce a milestone in the development of the AS-07 MXF video-preservation format specification. AS-07 has taken shape under the auspices of a not-for-profit trade group: the Advanced Media Workflow Association. AS-07 is now an official AMWA Proposed Specification, and the current version (CC by SA Creative Commons license and all) has been posted at the AMWA website. Although this writer retired from the Library in April, he helped shepherd the specification through this phase.
AS-07 is one of three new AMWA specifications announced in June. Another one is the organization’s new process rule book. The new AMWA process is patterned on the Requests for Comment approach used by the Internet Engineering Task Force. In the new AMWA scheme, there are three levels of maturity:
- Work in Progress
- Proposed Specification
Two earlier versions of AS-07 were exposed for community comment at the AMWA website, beginning in September 2014, and this met the requirements for a Work in Progress. For more information about the history of AS-07, refer to the FADGI website.
AS-07 is a standards-based specification. For the most part it is a cookbook recipe for a particular subtype of the MXF standard. MXF stands for Material eXchange Format, and that format’s complex and lengthy set of rules and options is spelled out in more than thirty standards from the Society of Motion Picture and Television Engineers. AS-07 also enumerates a number of permitted encodings and other components, each of which is based on other standards from SMPTE, the International Organization for Standardization and International Electrotechnical Commission, the European Broadcast Union, and special White Paper documents from the British Broadcasting Corporation. It is no wonder that a cookbook recipe is called for!
Why the emphasis on standards? The short answer is that standards underpin interoperability, in the digital world just as surely as they have for, say, the dimensions of railroad tracks, so my boxcar will roll down your rail line. It is worth saying that, in our preservation context, interoperability has both current and future dimensions. Today, cooperating archives may exchange preservation master files and these must be readable by both parties. More important, however, is temporal interoperability: today’s content must be readable by the archive of tomorrow. AS-07’s extensive use of standards-based design supports both types of interoperability.
At a high level, the objectives for video archival master files (aka preservation masters) are like those for the digital preservation reformatting for other categories of content. Archives want their masters to reproduce picture and sound at very high levels of quality. In addition, the preservation masters should be complete and authentic copies of the originals, i.e., in the case of video, they should retain components like multiple timecodes, closed captions and multiple soundtracks. And–back to temporal interoperability–the files must support access by future users.
What are some of the features of AS-07? The specification emphasizes encodings that ensure the highest possible quality of picture and sound, including requirements for declaring the correct aspect ratio and handling the intricacies of interlaced picture, a characteristic of pre-digital video. Beyond those elements, AS-07 also specifies options for the following:
- Captions and Subtitles
- retain and provide carriage for captions and subtitles
- translate binary-format captions and subtitles to XML Timed Text
- Audio Track Layout and Labeling
- provide options for audio track layout and labeling
- Content integrity
- provide support for within-file content integrity data
- provide coherent master timecode
- retain legacy timecode
- label multiple timecodes
- Embedding Text-Based and Binary Data
- provide carriage of supplementary metadata (text-based data)
- provide carriage of captions and subtitles in the form of Timed Text (text-based data)
- provide carriage of a manifest (text-based data)
- provide carriage of still images, documents, EBU STL, etc. (binary data)
- Language Tagging
- provide a means to tag Timed Text languages
- retain language tagging associated with legacy binary caption or subtitle data
- provide a means to tag soundtrack languages
- provide support for segmented content
AS-07 has not been exclusively developed in writing (“on paper,” in oldspeak). The format is based on pioneering work done by Jim Lindner in the early 2000s, when he developed a system called SAMMA (System for the Automated Migration of Media Archives). SAMMA produces MXF files for which the picture data is encoded as lossless JPEG 2000 frame images. It also operates in a robotic mode, to support high-volume reformatting.
Jim’s design for SAMMA was motivated by the forecasts for high-volume reformatting at the Library’s audio-visual center in Culpeper, Virginia (today’s Packard Campus for Audio-Visual Conservation), which was then in its planning phase. The Packard Campus began operation in 2007 and, since then, more than 160,000 videotapes have been reformatted using the SAMMA system. AS-07 is very much a refinement and elaboration of the SAMMA format. In order to get a better look at those refinements, in 2015, the AS-07 team commissioned the production of custom-made sample files.
What next? The interesting — and I think proper — feature of the new AMWA process concerns the movement from Proposed Specification to Specification. The rulebook lists several bullets as requirements but the gist is this: you gotta have implementation and adoption. AS-07 at this time is, metaphorically, a recipe ready to test in the kitchen. Now it is time to cook and taste the pudding. After there are instances of implementation and adoption, these will be reported to the AMWA board with a request to advance AS-07 to the level of [approved] Specification. (Of course, if the process reveals problems, the specification will be modified.)
The first steps toward implementation are under way. On FADGI’s behalf, the Library has contracted with Audiovisual Preservation Solutions and EVS to assemble additional test files, and to have them reviewed by an outside expert. At the same time, James Snyder, the Senior Systems Administrator at the Packard Campus, is working with vendors to do some actual workups. (James oversees the campus’s use of SAMMA and has been an active AS-07 team member.) We trust that these implementation efforts will bear fruit during the remaining months of 2016.