New FADGI MXF AS-07 Specification and Sample Files Published

The following is a guest post by Kate Murray, organizer the FADGI Audio-Visual Working Group and Digital Projects Coordinator at the Library of Congress.

The Federal Agencies Digital Guidelines Initiative (FADGI) is pleased to announce the publication of a new version of the MXF AS-07 Application Specification (with CC BY-SA 4.0 license) and its accompanying set of revised graded sample files.

Cover of Federal Agencies Guidelines Initiative AS-07 publication

Federal Agencies Digital Guidelines Initiative.
AS-07: MXF Archive and Preservation Format Application Specification

As mentioned in a previous blog post, AS-07 can be thought of as a sort of “cookbook recipe for a particular subtype of the MXF standard.” MXF, or the Material Exchange Format, is a wrapper or container format for video and audio media which is defined across a series of standards documents published by Society of Motion Picture and Television Engineers. AS-07 offers a detailed specification for a subset the MXF container which highlights functionality essential for archiving and preservation. In addition to SMPTE standards, AS-07 also enumerates a number of permitted encodings and other components from the International Organization for Standardization and International Electrotechnical Commission, the European Broadcast Union, and special White Paper documents from the British Broadcasting Corporation.

The core user group for AS-07 will be archives that maintain complex audiovisual content for the long term. Memory institutions with older obsolescing analog and digital videotapes in their collections, especially complicated commercially produced broadcast materials, will wish to archive the highest possible quality of image and sound (uncompressed or losslessly compressed), as well as retaining source data such as multiple timecodes, captions and subtitles, and also embed metadata that will support authentication and management of the content for the long term.

Top Level Source Package schematic

Schematic of MXF Top Level Source Package including picture, audio and data tracks (orange), timecode tracks (gray, yellow and pink) and metadata descriptors at the bottom. Image courtesy of Oliver Morgan and MetaGlue for FADGI.

James Snyder, the Senior Systems Administrator at the Library’s Packard Campus in Culpeper, Virginia, said, “The availability of the new AS-07 specification is terrific news.”  The facility has been digitizing historic videotapes for preservation since 2009.  “We have been using an early implementation of the MXF file format and lossless JPEG2000 picture encoding,” Snyder said, “successfully creating nearly two hundred thousand preservation masters.”  In addition, the Library has acquired tens of thousands of MXF files in this format from the American Archive of Public Broadcasting (AAPB) project.  The updated specification, Snyder said, will enable the Packard Campus “to begin capturing closed captions as text suitable for indexing, embed additional metadata, and correct MXF formatting errors in the early-implementation files.”  In the future, the Packard Campus team will be able to “re-wrap” their existing files (re-encoding will not be required) to bring them up to the AS-07 requirements.

AS-07 allows for a wide range of technical variety in moving image picture content typically required by memory institutions and other archives.  For high picture quality, the preferred picture encodings are JPEG 2000 picture encoding (such as the mathematically lossless version used by the Library of Congress) and uncompressed.  The rasters may range to sizes as great as 8Kx8K, and picture may be in any bit depth, color mode or space, and interlaced or progressive. Embedded metadata within the AS-07 files, especially that contained with the DMS or Descriptive Metadata Schemes, make the file self-describing so that they would retain their value even if they were the only extant copy of an item.

Federal Agencies Digital Guidelines Initiative logo

The AS-07 development project is led by the Library of Congress and other members of FADGI in collaboration with community and commercial collaborators. While AS-07 is vendor neutral by design, at various times throughout the development of the specification, the FADGI team heard from outside experts in broadcasting and other parts of the professional video industry. Under both the AMWA and FADGI auspices, there was active participation from several external commercial entities.  From the beginning of the project, long-time FADGI consultant AVPreserve (through audiovisual standards expert and founder and president Chris Lacinak) served as the principle investigator and data wrangler. George Blood of George Blood LP, a Philadelphia-based provider of services to memory institutions archives, was a strong early contributing member of the AS-07 working team while MetaGlue (US and UK-based developer of specialized applications for broadcasters, and home base for MXF standards expert Oliver Morgan) and EVS (Belgian-French provider of production and archiving systems for broadcasters, and home base for MXF standards expert Valérie Popie) were engaged as paid expert advisors. In addition, Cube-Tec (German provider of production and archiving systems, home base for SMPTE standards expert Jörg Houpert) contributed to the project as an expert reviewer of the specification language and sample files because they saw value in a process that would establish an open, public specification. The collaboration with industry is essential to the continued adoption and success of AS-07 because it’s the tools and workflows that make AS-07 compliance and integration possible.

screenshot of AS-07 graded sample files

AS-07 graded sample files, courtesy of EVS for FADGI

Jörg Houpert, CEO of Cube-Tec said, “We felt it important to actively participate in the detailed review of the published AS-07 draft specification and sample files. Including AS-07 functionality into our Quadriga video digitization system helps fill the gap in the cultural heritage community for transferring of video tapes into robust and self-describing archive files on a large-scale.” The AS-07 team agreed that Cube-Tec’s and others’ active participation in the open review periods provided invaluable industry perspective on the project.

FADGI also has plans to work with the open source community to build AS-07 capabilities in a variety of systems for different communities of practice, including developing an open source tool to batch embed metadata into MXF files.

While previous versions of AS-07 were published through AMWA, the most recent September 2017 version is published by FADGI. For more information about the history of AS-07 and a complete list of updates in the new version, refer to the FADGI website.

 

New FADGI Guidelines for Embedded Metadata in DPX Files

The Federal Agencies Digitization Guidelines Initiative Audio-Visual Working Group is pleased to announce that its new draft publication, Embedding Metadata in Scanned Motion Picture Film Files: Guideline for Federal Agency Use of DPX Files, is available for public comment. The Digital Picture Exchange format typically stores image-only data from scanned motion picture film or born-digital […]

FADGI MXF Video Specification Moves Up an Industry-organization Approval Ladder

The following is a guest post by Carl Fleischhauer, who organized the FADGI Audio-Visual Working Group in 2007. Fleischhauer recently retired from the Library of Congress. The Federal Agencies Digitization Guidelines Initiative Audio-Visual Working Group is pleased to announce a milestone in the development of the AS-07 MXF video-preservation format specification. AS-07 has taken shape […]

Demystifying Digital Preservation for the Audiovisual Archiving Community

The following is a guest post by Kathryn Gronsbell, Digital Asset Manager, Carnegie Hall; Shira Peltzman, Digital Archivist, UCLA Library; Ashley Blewer, Applications Developer, NYPL; and Rebecca Fraimow, Archivist and AAPB NDSR Program Coordinator, WGBH. The intersection of digital preservation and audiovisual archiving has reached a tipping point. As the media production and use landscape […]

Avoid Jitter! Measuring the Performance of Audio Analog-to-Digital Converters

The following is a guest post by Carl Fleischhauer, a Project Manager in the National Digital Initiatives unit at the Library of Congress. It’s not for everyone, but I enjoy trying to figure out specialized technical terminology, even at a superficial level. For the last month or two, I have been helping assemble a revision […]

Keeping Our Tools Sharp: Approaching the Annual Review of the Library of Congress Recommended Formats Statement

The following post is by Ted Westervelt, head of acquisitions and cataloging for U.S. Serials in the Arts, Humanities & Sciences section at the Library of Congress. Since first launching its Recommended Formats Statement (then called Recommended Format Specifications in 2014), the Library of Congress has committed to treating it as an important part of […]

Survey: How Do You Approach Web Archiving?

Do you have fifteen minutes to tell the National Digital Stewardship Alliance about your organization’s web archiving activities? If the answer is yes, please contribute to the NDSA Web Archiving Survey. By filling out this short survey, your institution will be part of a multi-year project to track the evolution of web archiving programs in […]

ODF: The Open Document Format

The following is a guest post by Carl Fleischhauer, a Digital Initiatives Project Manager at the Library of Congress. During December 2015, the Library’s Format Sustainability website added descriptions of eleven members of the Open Document Format family, aka OpenDocument and ODF. These eleven join a number of other format descriptions mounted in 2015, many […]

Digitizing Motion Picture Film: FADGI Report on Current Practices and Future Directions

The following is a guest post by Carl Fleischhauer, a Digital Initiatives Project Manager at the Library of Congress. More often than not, the Federal Agencies Digitization Guidelines Initiative Working Groups (one for still images, one for audio-visual) find themselves walking a line between codifying widely adopted practices and exploring new ideas and new technologies […]