This is a guest post written by Jacob Kowall and Hilary Szu Yin Shiue, 2021 Junior Fellows in the Digital Collections Management & Services Division (DCMS) under the mentorship of Kate Murray, Digital Projects Coordinator.
Jacob and Hilary assisted in updating and expanding the Sustainability of Digital Formats website, which provides information and analysis on over 500 digital file formats and offers guidance on the long-term preservation of digital content at the Library. Through their work, they assisted in providing current information on file formats to users at the Library of Congress and throughout the international digital preservation community.
In this blog post, Jacob and Hilary discuss their research on the HyperCard file format. As part of this effort, they have connected with external partners and other technical registries, helping to enhance engagement between the Library’s file formats resources and those of the larger community.
HyperCard Research
As part of our work as Junior Fellows, we have been involved in research on HyperCard, a discontinued Apple program used to design a wide range of computer applications in the late-1980s and 1990s. We’ve built on the previous efforts of Kathleen O’Neill and Chad Conrady, archives specialists in the Manuscript Division, who also engaged with HyperCard files during their time as Staff Innovators with LC Labs in 2020. Chad and Kathleen’s project, Born Digital Access Now!, focused on reviewing the Manuscript Division’s born-digital files through a combination of file analysis and file emulation. Their work on this project has been documented in a number of previous Signal blog postings, including: Introducing the 2020 Staff Innovators! (July 20, 2020), Analyzing the Born-Digital Archive (Kathleen O’Neill, October 22, 2020), and An Archivist’s Perspective on Legacy Files (Chad Conrady, November 16, 2020).
Prior Work with HyperCard
During the Staff Innovator project, Kathleen and Chad learned that identifying the formats of many files in the Manuscript Division’s born-digital collections would demand a huge amount of effort. Although they could generate reports using powerful file analysis tools, like DROID and Siegfried/Brunnhilde, these reports still failed to identify all files. Many of the unidentified files were in outdated – or “legacy” – formats. By using resources at the Sustainability of Digital Formats site (also referred to as the Formats site), Kathleen and Chad were able to identify some of the mystery files, but a lack of thorough documentation on certain legacy formats remained a major challenge to their work.
While analyzing born-digital materials in the Nina V. Fedoroff Papers, for instance, Chad encountered a number of legacy HyperCard files. Our preliminary research on HyperCard revealed that the software was a widely popular application in the 1980s and 1990s. In fact, Hypercard is an important piece of Library of Congress history. The American Memory project, an early collection of digitized LC materials, was first built with HyperCard in the pre-Internet era, as mentioned in a previous Signal post, “It’s Dead, Jim”: Resurrecting an Obsolete File, Part 2.
Revealing a Forgotten Gem
Despite its widespread usage in the 1980s and 1990s, HyperCard remains largely absent from the leading file format registries. For example, PRONOM, the UK National Archives’ comprehensive registry of digital file formats, has no entries for HyperCard formats, nor are any included in the Internet Assigned Numbers Authority (IANA) media type list. The Siegfried file format identification tool also fails to identify HyperCard files, indicating that none of the major format registries include entries for HyperCard. Although Wikidata has a record for the HyperCard file format (Q27996244), it provides little information useful for format analysis. For all the above reasons, we determined that both the Manuscript Division and the wider digital preservation community might especially benefit from further research and aggregated documentation on HyperCard files. Thus began our adventure into exploring the history and resurgence of HyperCard.
Figure 1: These images show cards from “HyperMaize” stacks from the Nina V. Fedoroff Papers, as they appear in an emulated Apple OS 9 environment. HyperMaize stacks recorded and presented information about the genomics of maize. The image on the left shows the opening screen for HyperMaize, while the image on the right shows the interface for recording information about genetic crosses. (From the Nina V. Fedoroff Papers, Manuscript Division, Library of Congress, Washington, D.C.) You can learn more about the emulation of HyperMaize stacks at the Library in Chad Conrady’s previous Signal post, “An Archivist’s Perspective on Legacy Files.”
HyperCard: Background and History
What types of programs and applications do you regularly use on your personal computer? Video games? E-book readers? Office productivity software? Have you ever designed your own software applications?
When developing HyperCard in the 1980s, its designer, Bill Atkinson, wanted to create a tool that both programmers and non-programmers could use to write their own computer applications. After its release in 1987, Atkinson likened the new program to an “erector set” for software, alluding to the popular toy construction set [1]. Users of HyperCard created documents called “stacks,” which were made up of a series of related “cards.” Each card in a stack could contain text, images, interactive elements, and sound. HyperCard featured a classic, black-and-white user-interface. Despite its relatively humble appearance, the potential uses of HyperCard were extensive. An early user guide for the program listed some examples of applications that could be constructed using HyperCard stacks, such as “an appointment calendar with cards for the various days, weeks, or months; a photo collection; an atlas; or even a library card catalog” [2]. Users found many other creative applications for the program, such as in designing interactive stories and even video games. For instance, the popular video game Myst (released in 1993) was first developed with HyperCard [3].
Bill Atkinson allegedly came up with the idea for HyperCard during an LSD trip, but the conceptual design of HyperCard stacks drew heavily on common physical card-filing systems, like Rolodex [4]. What made HyperCard revolutionary was that it gave Macintosh users a new way to easily link virtual objects and information, and the program is often credited as a principal inspiration of the World Wide Web [5]. After its release, HyperCard was included free of charge with all Macintosh computer systems, and it quickly became a “widespread standard” for software design [6]. HyperCard’s official run lasted a little under twenty years, and following the final release of HyperCard (version 2.4.1) in 1998, Apple discontinued sales of the products in 2004.
Though HyperCard file format specifications were never made available by Apple Inc., fans of HyperCard and members of the digital preservation community have created unofficial documentation by reverse-engineering available stack files. These community-produced specifications help to explain how the structure of an individual stack appears when opened with a hex editor, a tool for viewing the raw data in a digital file at the byte level. All HyperCard stack files are made up of common components, or “blocks”. Each block is identified by four pairs of hexadecimal values, corresponding to four ASCII characters [7]. For instance, a stack file begins with a stack block (signified by the hexadecimal values, “53 54 41 4B,” which correspond to the ASCII characters, “STAK”), and the file is terminated by a tail block (signified by the ASCII characters, “TAIL”). When viewing HyperCard stack files from the Nina V. Fedoroff Papers in a hex editor, we confirmed that these files matched this structure. Figure 2 presents a stack file when viewed in a hex editor.
Figure 2: The beginning of a HyperCard stack file from the Nina V. Fedoroff Papers, viewed in a hex editor. A hex editor represents the file’s binary data as hexadecimal values. Each pair of hex digits represents one byte. The ASCII characters that correspond with those values are displayed to the right. In this image, the start of the HyperCard stack block is signified by the hexadecimal values, “53 54 41 4B,” corresponding to the ASCII characters for “STAK.”
Because the HyperCard stack format is not yet represented in the PRONOM technical registry, we reached out to the UK National Archives to share our findings. From this communication, we hope to determine if it would be appropriate to submit a registration for a PRONOM Unique Identifier (or PUID) for the legacy format. Through contacts at the U.S. National Archives and MIT Libraries, we’ve learned that other collecting institutions have HyperCard stacks in their digital collections. Eventually, we hope to link the PRONOM PUID for HyperCard stacks to a new file format description document for the Sustainability of Digital Formats, which we have also begun to draft. We hope that these new resources will help add to communal knowledge on HyperCard and will benefit work performed both at LC and within the wider digital preservation community.
Conclusion and Reflections
As Junior Fellows and early career information professionals, the experience conducting HyperCard research introduced us to larger topics in software preservation and access. File format specifications are needed for maintaining and preserving intricate digital formats. Because Apple never released any official documentation on HyperCard, there remain serious challenges in preserving HyperCard stacks, putting digital content created by a wide array of communities at risk of loss. But through this project, we witnessed the resilience of HyperCard users, and learned how community-led efforts are also indispensable for preservation. For instance, HyperCard stacks are now open to analysis largely through the collaborative efforts of passionate HyperCard fans who worked to reverse-engineer technical specifications and documentation. (Examples of these specifications come from Pierre Lorenzi and Rebecca Bettencourt.) We’ve also learned about ongoing efforts to provide meaningful access to legacy files, such as the Software Preservation Network’s Emulation-as-a-Service Infrastructure (EaaSI) program. In fact, one of our favorite surprises during our HyperCard research was finding the Internet Archive’s emulated HyperCard Stacks collection, which contains over 3,600 stacks as of July 2021!
Our research on HyperCard is going to be refined to form the basis of a new entry for the Sustainability of Digital Formats site. Our contributions in this process have revealed to us the truly herculean effort that goes into updating and maintaining the Formats site. Kate Murray estimates that it takes a total of 40 hours to prepare a new entry on the site, and we can certainly attest to this! (You can read more about the process of writing a format description document on Marcus Nappier’s Signal post from September 2020.) In the end, the labor is well worth the outcome, as the Formats site serves as an invaluable resource to the Library, the global preservation community, and beyond.
References
[1] Bill Atkinson in HyperCard Power: Techniques and Scripts by Carol Kaehler (Addison-Wesley, 1988), page xiv, https://archive.org/details/Hyper_Card_Power/page/n11/mode/2up.
[2] Carol Kaehler, HyperCard Power: Techniques and Scripts (Addison-Wesley, 1988), https://archive.org/details/Hyper_Card_Power/page/n39/mode/2up.
[3] Matthew Lasar, “30-plus years of HyperCard, the missing link to the Web,” May 25, 2019, Ars Technica, accessed July 20, 2021, https://arstechnica.com/gadgets/2019/05/25-years-of-hypercard-the-missing-link-to-the-web/.
[4] Triangulation 247: Bill Atkinson Part 2 [interview]; Kitty J. Mackey, “Automating overdues in a non-automated library: The HyperCard solution,” College & Research Libraries News, Vol 50, No 1 (1989), pages 15-16, https://crln.acrl.org/index.php/crlnews/article/view/19563/23035.
[5] Samuel Arbesman, “The forgotten software that inspired our modern world,” July 23, 2019, BBC Future, accessed July 21, 2021, https://www.bbc.com/future/article/20190722-the-apple-software-that-inspired-the-internet.
[6] Terry Winograd, Bringing Design to Software (Addison-Wesley, 1996), https://hci.stanford.edu/publications/bds/10p-prototype.html.
[7] Pierre Lorenzi, “HyperCard File Format,” accessed July 12, 2021, https://github.com/PierreLorenzi/HyperCardPreview/blob/master/StackFormat.md.