New FADGI MXF AS-07 Specification and Sample Files Published

The following is a guest post by Kate Murray, organizer the FADGI Audio-Visual Working Group and Digital Projects Coordinator at the Library of Congress.

The Federal Agencies Digital Guidelines Initiative (FADGI) is pleased to announce the publication of a new version of the MXF AS-07 Application Specification (with CC BY-SA 4.0 license) and its accompanying set of revised graded sample files.

Cover of Federal Agencies Guidelines Initiative AS-07 publication

Federal Agencies Digital Guidelines Initiative.
AS-07: MXF Archive and Preservation Format Application Specification

As mentioned in a previous blog post, AS-07 can be thought of as a sort of “cookbook recipe for a particular subtype of the MXF standard.” MXF, or the Material Exchange Format, is a wrapper or container format for video and audio media which is defined across a series of standards documents published by Society of Motion Picture and Television Engineers. AS-07 offers a detailed specification for a subset the MXF container which highlights functionality essential for archiving and preservation. In addition to SMPTE standards, AS-07 also enumerates a number of permitted encodings and other components from the International Organization for Standardization and International Electrotechnical Commission, the European Broadcast Union, and special White Paper documents from the British Broadcasting Corporation.

The core user group for AS-07 will be archives that maintain complex audiovisual content for the long term. Memory institutions with older obsolescing analog and digital videotapes in their collections, especially complicated commercially produced broadcast materials, will wish to archive the highest possible quality of image and sound (uncompressed or losslessly compressed), as well as retaining source data such as multiple timecodes, captions and subtitles, and also embed metadata that will support authentication and management of the content for the long term.

Top Level Source Package schematic

Schematic of MXF Top Level Source Package including picture, audio and data tracks (orange), timecode tracks (gray, yellow and pink) and metadata descriptors at the bottom. Image courtesy of Oliver Morgan and MetaGlue for FADGI.

James Snyder, the Senior Systems Administrator at the Library’s Packard Campus in Culpeper, Virginia, said, “The availability of the new AS-07 specification is terrific news.”  The facility has been digitizing historic videotapes for preservation since 2009.  “We have been using an early implementation of the MXF file format and lossless JPEG2000 picture encoding,” Snyder said, “successfully creating nearly two hundred thousand preservation masters.”  In addition, the Library has acquired tens of thousands of MXF files in this format from the American Archive of Public Broadcasting (AAPB) project.  The updated specification, Snyder said, will enable the Packard Campus “to begin capturing closed captions as text suitable for indexing, embed additional metadata, and correct MXF formatting errors in the early-implementation files.”  In the future, the Packard Campus team will be able to “re-wrap” their existing files (re-encoding will not be required) to bring them up to the AS-07 requirements.

AS-07 allows for a wide range of technical variety in moving image picture content typically required by memory institutions and other archives.  For high picture quality, the preferred picture encodings are JPEG 2000 picture encoding (such as the mathematically lossless version used by the Library of Congress) and uncompressed.  The rasters may range to sizes as great as 8Kx8K, and picture may be in any bit depth, color mode or space, and interlaced or progressive. Embedded metadata within the AS-07 files, especially that contained with the DMS or Descriptive Metadata Schemes, make the file self-describing so that they would retain their value even if they were the only extant copy of an item.

Federal Agencies Digital Guidelines Initiative logo

The AS-07 development project is led by the Library of Congress and other members of FADGI in collaboration with community and commercial collaborators. While AS-07 is vendor neutral by design, at various times throughout the development of the specification, the FADGI team heard from outside experts in broadcasting and other parts of the professional video industry. Under both the AMWA and FADGI auspices, there was active participation from several external commercial entities.  From the beginning of the project, long-time FADGI consultant AVPreserve (through audiovisual standards expert and founder and president Chris Lacinak) served as the principle investigator and data wrangler. George Blood of George Blood LP, a Philadelphia-based provider of services to memory institutions archives, was a strong early contributing member of the AS-07 working team while MetaGlue (US and UK-based developer of specialized applications for broadcasters, and home base for MXF standards expert Oliver Morgan) and EVS (Belgian-French provider of production and archiving systems for broadcasters, and home base for MXF standards expert Valérie Popie) were engaged as paid expert advisors. In addition, Cube-Tec (German provider of production and archiving systems, home base for SMPTE standards expert Jörg Houpert) contributed to the project as an expert reviewer of the specification language and sample files because they saw value in a process that would establish an open, public specification. The collaboration with industry is essential to the continued adoption and success of AS-07 because it’s the tools and workflows that make AS-07 compliance and integration possible.

screenshot of AS-07 graded sample files

AS-07 graded sample files, courtesy of EVS for FADGI

Jörg Houpert, CEO of Cube-Tec said, “We felt it important to actively participate in the detailed review of the published AS-07 draft specification and sample files. Including AS-07 functionality into our Quadriga video digitization system helps fill the gap in the cultural heritage community for transferring of video tapes into robust and self-describing archive files on a large-scale.” The AS-07 team agreed that Cube-Tec’s and others’ active participation in the open review periods provided invaluable industry perspective on the project.

FADGI also has plans to work with the open source community to build AS-07 capabilities in a variety of systems for different communities of practice, including developing an open source tool to batch embed metadata into MXF files.

While previous versions of AS-07 were published through AMWA, the most recent September 2017 version is published by FADGI. For more information about the history of AS-07 and a complete list of updates in the new version, refer to the FADGI website.

 

FADGI’s 10th Anniversary: Adapting to Meet the Community’s Needs

This is a guest post by Kate Murray, IT Specialist in the Library of Congress’s Digital Collections and Management Services. Started in 2007 as a collaborative effort by federal agencies, FADGI has many accomplishments under its belt, including the widely implemented Technical Guidelines for Digitizing Cultural Heritage Materials (newly updated in 2016); open source software, […]

New FADGI Guidelines for Embedded Metadata in DPX Files

The Federal Agencies Digitization Guidelines Initiative Audio-Visual Working Group is pleased to announce that its new draft publication, Embedding Metadata in Scanned Motion Picture Film Files: Guideline for Federal Agency Use of DPX Files, is available for public comment. The Digital Picture Exchange format typically stores image-only data from scanned motion picture film or born-digital […]

FADGI MXF Video Specification Moves Up an Industry-organization Approval Ladder

The following is a guest post by Carl Fleischhauer, who organized the FADGI Audio-Visual Working Group in 2007. Fleischhauer recently retired from the Library of Congress. The Federal Agencies Digitization Guidelines Initiative Audio-Visual Working Group is pleased to announce a milestone in the development of the AS-07 MXF video-preservation format specification. AS-07 has taken shape […]

Avoid Jitter! Measuring the Performance of Audio Analog-to-Digital Converters

The following is a guest post by Carl Fleischhauer, a Project Manager in the National Digital Initiatives unit at the Library of Congress. It’s not for everyone, but I enjoy trying to figure out specialized technical terminology, even at a superficial level. For the last month or two, I have been helping assemble a revision […]

Digitizing Motion Picture Film: FADGI Report on Current Practices and Future Directions

The following is a guest post by Carl Fleischhauer, a Digital Initiatives Project Manager at the Library of Congress. More often than not, the Federal Agencies Digitization Guidelines Initiative Working Groups (one for still images, one for audio-visual) find themselves walking a line between codifying widely adopted practices and exploring new ideas and new technologies […]

Improving Technical Options for Audiovisual Collections Through the PREFORMA Project

The digital preservation community is a connected and collaborative one. I first heard about the Europe-based PREFORMA project last summer at a Federal Agencies Digitization Guidelines Initiative meeting when we were discussing the Digital File Formats for Videotape Reformatting comparison matrix. My interest was piqued because I heard about their incorporation of FFV1 and Matroska, […]

All in the (Apple ProRes 422 Video Codec) Family

We’ve spent a lot of time recently thinking about digital video issues. As mentioned in a previous blog post, the Federal Agencies Digitization Guidelines Initiative published several reports on this topic including “Creating and Archiving Born Digital Video.” Work on the “Eight Federal Case Histories” (PDF) report nudged us to add the Apple ProRes 422 […]

New FADGI Report: Creating and Archiving Born Digital Video

As part of a larger effort to explore file formats, the Born Digital Video subgroup of the Federal Agencies Digitization Guidelines Initiative Audio-Visual Working Group is pleased to announce the release of a new four-part report, “Creating and Archiving Born Digital Video.” This report has already undergone review by FADGI members and invited colleagues including […]

Comparing Formats for Video Digitization

The following is a guest post by Carl Fleischhauer, a Digital Initiatives Project Manager in the Office of Strategic Initiatives. FADGI format comparison projects. The Audio-Visual Working Group within the Federal Agencies Digitization Guidelines Initiative recently posted a comparison of a few selected digital file formats for consideration when reformatting videotapes. We sometimes call these […]