Observing the Slightest Motion: Using Visual Tools to Preserve Sound

This is a guest post written by Peter Alyea. Peter is a Preservation Science Specialist in the Research and Testing Division of the Library of Congress and has been working with Lawrence Berkeley National Laboratory on imaging recorded sound collections for preservation and access since the inception of the IRENE project.

Although making audio recordings today is as trivial as unlocking your phone and pressing a button, the technical innovations that kicked off the recorded sound era were revelatory at the time. The first sound recordings ever made were with a device called the phonautograph invented by Édouard-Léon Scott de Martinville and patented on March 25th, 1857. The phonautograph was not designed to reproduce sound but produced a visual representation of the sound waves on paper. This important step towards recording and reproduction of audio echoed, almost a century and a half later, the usefulness of visualizing sonic events that are naturally invisible to our eyes.

The vinyl disc that still survives today is the culmination of many early formats that also included the popular cylinder record. These historic audio formats inscribe the sounds as tiny motions in a groove on the media’s surface. Reproduction of the sound is achieved by riding a stylus in the groove that vibrates in response to these dislocations which can eventually drive speakers and recreate the sounds that we can hear. Although the patterns in the groove of a record were designed to be traced with a stylus in motion, they can also be examined as a static pattern with modern imaging equipment.

Photograph of the IRENE system.

The first IRENE System located at the Library of Congress Packard Campus in Culpeper, VA. Photo Credit: Peter Alyea

In the early 2000’s, physicists at Lawrence Berkeley National Laboratory started experimenting with the idea of using a high-resolution microscope to scan the surface of a disc to extract the sound. This work eventually resulted in a collaboration with the Library of Congress to develop methods and tools to help preserve recorded sound collections. With the ability to image in two and three dimensions the IRENE (Image Reconstruct Erase Noise Etc.) System preserves sound from a wide variety of historic audio media formats and is named after the first disc imaged at Berkeley Lab, “Goodnight, Irene” as performed by The Weavers.

Imaging in two dimensions was originally achieved by illuminating the disc with an intense white light perpendicular to the disc’s surface. With the surface illuminated in this fashion, a camera captures images of the top and bottom of the groove in white, with the groove side walls appearing much darker. These light and dark areas reveal the motion of the groove. The specialized camera, called a line scanning camera, has a single pixel line sensor instead of the rectangular sensors that are used in phone cameras. This line of pixels is positioned across numerous revolutions of the groove so that the orientation is the same for each image captured, effectively unwinding the spiral and facilitating image processing to extract the motion.

IRENE is both a process, which is in use to transfer collections, and an evolving technology project.  Over the course of the project we have benefited from numerous advances in technology.  Currently we are exploring the use of polychromatic illumination to robustly gather redundant data from lateral grooved recordings, most significantly 78 rpm shellac discs.  This offers a new approach to noise reduction while acquiring data at high speed. – Dr. Carl Haber, Senior Scientist Lawrence Berkeley National Laboratory

Image of the IRENE output of a shellac disc

Images of a rough, light dispersing 78rpm shellac disc (top) and a smooth, mirror-like reflecting disc (bottom)

Although materials, such as vinyl, used to manufacture records are quite reflective, other common discs are made from a much rougher material. The 78 rpm (revolutions per minute) shellac disc has this rough quality and, when illuminated, disperses light in many directions.  This diffusion of the light obscures the exact path of the groove captured by the images, which results in higher noise during reproduction than one would expect from a stylus playback. The stylus comes in contact with numerous points of the groove wall as it is tracing out the motion, reducing noise by averaging over the area. Finding a way to image more of the groove motion would similarly help reduce noise with the IRENE System. If a disc is scanned multiple times with the direction of the light altered for each scan, summing the results does reduce the noise; however, it is time consuming to rescan a single side of a disc multiple times. When multiple lights are arranged at different angles for a single scan, the lights bleed into each other reducing the accuracy of the light and dark areas in the image. It would also be possible to arrange duplicate camera and lighting setups to cover different quadrants of the disc so that in a single scan the lights wouldn’t interfere with each other. This approach would be expensive and complicates the scanning process. The solution that is currently being explored is to use a single camera and multiple lights simultaneously, but to swap out the white light for red, green, and blue lights and the gray scale camera for a color camera. This arrangement doesn’t stop the lights from bleeding into each other but does allows for filtering out colors within areas of interest, resulting in good quality groove motion detection. With lights at 45o, 90o, and 135o, the scanning process is only slightly slowed down but produces three times the amount of groove motion information in a single scan.

Image output with the Color 2D IRENE system.

Three color, two-dimensional image of a 78 rpm shellac disc taken by the IRENE System.

Early in 2021, the Library of Congress was fortunate to have Leo Zipeng Lin contribute to the image-to-sound software though the internship program. Leo was able to quickly learn the C# programming language and write code that analyzed and combined the multi-color light data to reduce noise in the resulting audio files. Leo is continuing to contribute to the development of the IRENE System through the Undergraduate Research Apprenticeship program at the University of California, Berkeley.

My favorite moment of working on this project was when I clicked the “play” button in the Weaver program; I could hear the beautiful melody of songs of historical significance. I realized how my contribution of algorithms to this project could help revive music tracks that witnessed history. – Leo Zipeng Lin

Screenshot of the IRENE image processing software.

Image processing software locating the motion along the bottom of the groove of a disc recording.

Once the images are captured, they are processed with this specialized software package that can follow the contours of the groove motion and extract sound from it. The relationship of the motion in the groove to the sound is not positional. The relationship of the motion in the groove to the sound is not positional. The sound is not dependent upon where on the disc the stylus is sitting. The audio is the combination of the speed at which the disc moves and how fast and far the stylus moves. Laterally (side-to-side) cuts discs are the most common disc format, although vertically cut discs were also commercially available. If the stylus is moving very fast side-to-side, higher frequency sounds are produced and conversely, when it is moving very slowly, low frequency sounds are produced. As the stylus travels further side-to-side, the louder the sound will be. For audio events with a broad frequency range, such as music with a variety of instruments, the motion in the groove is very complex. By extracting the positional information from the images, the software can then simulate the motion the stylus would make and produce an audio file. Experimentation has produced positive results and continues to determine the best methodologies for reducing the noise with multiple light sources.

The next time you listen to a recording, even if it is a modern recording not originally captured in the groove of a disc, consider how audio was first recorded. Sound waves were converted into a motion that was cut into a soft material, creating a pattern that could be traced to reproduce the sound. By examining those patterns visually, sounds that might have been lost are able to be heard once again.

Merry Christmas from IRENE! Check out this recording of holiday favorite, Jingle Bells.

To hear IRENE audio samples and learn more about IRENE, click the links below.

IRENE Audio Samples

Smithsonian Museum of American History: Hear My Voice exhibit

Northeast Document Conservation Center: IRENE

Lawrence Berkeley National Laboratory: IRENE

But, What Is It? How Research Science Informs Preservation

I’m a research scientist and I’ve worked in a laser, microscopy, or spectroscopy laboratory (and sometimes a lovely combination of the three) for the past seventeen years. The last five of those years I’ve been lucky to call myself a Chemist at the Library of Congress, working in the optical properties lab of the Preservation […]