I’m here with Dame Wendy Hall, Kluge Chair in Technology and Society, Regius Professor of Computer Science at the University of Southampton and early pioneer in web protocols; with Alexandre Loktionov, AHRC Fellow at the Kluge Center and an expert on hieroglyphic and cuneiform legal texts; and with Jessica Lingel, Kluge Fellow, assistant professor at the Annenberg School for Communication at the University of Pennsylvania and an expert on social media.
We ventured into talking about emoji and social media during a hallway conversation and thought it would be fun to pursue this further via blog.
The text of our Google Docs conversation was edited for length and clarity.
DT: There is much to explore, but it began with emoji, so let’s start there: elevated art form or corruption of language?
AL: For me, they’re essentially hieroglyphs and so a perfectly legitimate extension of language. They’re signs which, without having a phonetic value of their own, can ‘color’ the meaning of the preceding word or phrase. In Egyptology, these are called ‘determinatives’ — as they determine how written words should be understood. The concept has been around for 5,000 years, and it’s remarkably versatile because of its efficiency. You can cut down your character count if you supplement words with pictures — and that’s useful both to Twitter users today and to Ancient Egyptians laboriously carving signs into a rock stela.
DT: How does everyone feel about using emoji to write literature? The Library of Congress acquired an emoji version of none other than “Moby Dick” just a few years ago.
AL: I think you can definitely write literature with emoji — the question is, who will be able to read it? Do we have enough standardization in sign deployment? I think a full emoji dictionary/sign list would be necessary, unless, of course, we want to create a literature with multiple strands of interpretation (in a literal sense — where people see the same signs but interpret them in different ways).
JFL: I think part of it is about a fascination with how technology may be reshaping cultural production. I’m thinking of games around Twitter and literature, for example; the Guardian ran a challenge asking authors to write a story in 140 characters or less. (There’s a long and wonderful history of literature produced through challenges/games like these; I’m thinking of Shelley and Hemingway.) At the root, I think, is an anxiety around what it means to make art and how technology is making art better or worse.
DT: I’m optimistic because I see technological innovations opening up the range of what is possible artistically — Gutenberg, and so forth. On the other hand, certain technological turns have been very specific in their application. Think of Morse code: incredibly useful in certain contexts, but unlikely that we will ever write a novel in Morse.
AL: I think that gets to the heart of it — we have to think of the purpose of the means of communication, and in the case of emoji, we as a culture need to decide what they are: do we want them to be a bona fide script with full capability, or are they just a tool reserved for very specific purposes (alongside conventional means of writing)?
JFL: I don’t know about Morse code novels, but Morse code poetry is definitely a thing.
AL: It’s also worth thinking about canonicity — can emoji become canonical, in a way in which originally purely utilitarian hieroglyphs could after several millennia? Are we in this for the long run?
DT: Right, will there ever be an emoji dictionary? Perhaps there is already?
WH: There is a crowd-sourced emoji dictionary. It’s not very helpful at the moment, but then, neither was Wikipedia initially.
AL: Yes, exactly — I know there’s a grammar (of sorts), but not sure about a dictionary. And the trouble is, you can write a dictionary or a grammar, but it then has to gain acceptance on a wide level to actually be of any use.
WH: Absolutely, this is all about evolution. If anyone took the trouble to formalize emoji at the moment, the emoji world would have completely changed by the time any formalization was agreed on. I’m not aware of a grammar for emoji. Shows you how little I use them really.
JFL: Well, in a very literal sense there is definitely an emoji dictionary, in that for emoji to function across different devices, they need to exist in unicode, which includes definitions of what each character/image/symbol means. But these definitions have little to say about the different cultural appropriations of symbols. Part of what Wendy and Alexandre are talking about here is the way that the flexibility of emoji has allowed for their popularity.
AL: Very true — maybe computers can standardize them in a way we humans struggle to at the moment. Computers communicating among themselves one day, maybe? With all the talk of artificial intelligence.
DT: It would be constantly updated and crowdsourced, so this would be a very different, rapidly evolving way of canonizing.
AL: Absolutely, and especially because levels of preservation are probably going to be very low. We probably don’t have as many printed emoji as we have hieroglyphs, alphabetic words or whatever else. They get used, and then so often just disappear into the ether. And (traditional) dictionaries rely on being able to collate attestations. So does canon, at least in conventional form.
DT: I’m curious about use. To play into stereotypes, my sense is that men use emoji less than women. This is coming from someone who can barely use a semicolon at the end of a sentence to indicate mirth;
DT: That is about my range of emoji use. What do you think, have men embraced this form of communication, or is it still uncharted territory?
AL: I definitely haven’t. I feel like I can maybe comment on it in an academic sort of way, but personally I’m very attached to our tried-and-tested alphabetic way of doing things.
JFL: Meh. I think this is very culturally dependent. My sense is that emoji use is somewhat bimodally distributed in terms of age. My undergrad students love emoji, but so does my mom. There is definitely a cultural preoccupation with whether or not men can or should use emoji. And then, of course, there’s a longstanding set of generalizations about women being better at communication than men, or perhaps more precisely, the idea that communication is a feminine skill, as opposed to hyper-rational thinking, which is typically (in the West) coded as a feature of masculinity. (Insert skeptical comments regarding women’s preoccupation with feeding families as somehow frivolous and men’s preoccupation with sports as manly, etc.)
DT: Perhaps there is an irony here, in that emoji originated, if I’m not mistaken, out of a corporate context, with a preoccupation with marketing and brand identity. And now, social media companies are seeking to measure emoji use to quantify emotional states and produce commercially useful data.
AL: That’s essentially the notion of the “determinative,” which is a central pillar of Ancient Egyptian. A single sign determines an emotional state, but the difference is that the sign accompanies a word for that state which is spelt out. Here it’s more complex and harder to quantify, I would imagine — there are fifty types of happy emoji or whatever, with fluid meaning, how exactly do you quantify the intensity or “type” of happiness being felt? We’re back to the challenge of operating without a dictionary or standardization — effectively just power of the crowd. I think they are also very useful to non-native speakers (I’m thinking of members of my family in Russia, who try to express themselves in English, fail, and then convert to emoji-speak to get their message across).
DT: I’ll venture emoji are still looked upon with some suspicion, as in they are best used in frivolous types of conversation. Are they simply a parallel form of language, useful in some social contexts, or with new technological developments, are we moving toward greater standardization and perhaps a more formalized use?
JFL: Arguably, this goes back to the discussion of gender. Are emoji considered frivolous because they’re used to convey emotion, and thus feminized? There are also parallels to Twitter here, where early Twitter users were characterized as pointless and frivolous, even though these records of everyday life are deeply constitutive of who we are as a society.
DT: A great point. And now Twitter drives public opinion, policy and elections.
WH: Dan — here is the reference I sent yesterday that talks about the different ways men and women do and don’t use emoji.
JFL: Also, here’s an academic study (from 2000) about men vs. women using emoticons.
AL: We could end up with parallel scripts, perhaps. That’s historically very common — in my field, Ancient Egypt, there was a period, for example, when there were three scripts in operation simultaneously — one for carving on stone, and mainly to do with death; one for high-order literature, some royal administration and other important tasks done in life (written with a brush on papyrus); and a shorthand for routine tasks, like counting cattle or whatever.
AL: And again here, it’s noteworthy that these multiple scripts are a reflection of the different material forms of writing — on a stone surface with a chisel, or with a brush or reed pen on papyrus. The material on which you write is so important — in that sense I firmly believe that it’s natural for something like a smartphone to give rise to a new script.
AL: I love the emoji grammar. The sign list is basically essential to early Egyptological textbooks developed back in the Thirties, like this one, for instance (obviously it was digitized more recently).
WH: Don’t forget how the Chinese language has developed over the years and how they are using cartoons/pictures now to escape censorship. This is developing (I believe) into quite a sophisticated language that defeats the machines (at the moment).
JFL: Absolutely. danah boyd and Alice Marwick have written about how teens use cultural references to hide messages from people (read: parents) who are in their social networks but can’t follow cultural references. For what it’s worth, I’ve written a paper with Aram Sinnreich about different modes of hidden communication used by people who are incarcerated and how that could inspire forms of protest among media activists.
WH: There is ongoing discussion about when machines will actually be able to “understand” the words/pictures/concepts they are manipulating. When will machines be able to understand emoji? Will the evolution of emoji make cognitive AI easier or harder to achieve?
DT: Wendy can you say a bit more about cognitive AI? What do you mean by this?
WH: It has been defined as the fourth wave of AI (the first being Turing and Minsky in the 50s and 60s — can computers think; the second being the rule-based expert systems of the 70s and 80s; the third being the machine learning/deep learning systems that we can build today because there is so much more computing power than there was when AI was first conceived). We will have cognitive AI when machines actually understand the information/concepts they are dealing with. At the moment they appear intelligent (e.g . Siri) but without actually “knowing” what they are talking about. This is years away — it will take another radical evolution in computers to achieve.
DT: It sounds like we are all optimistic about the creative potential of emoji — and perhaps alternative forms of communication in general. What I am hearing from Alex is that we have historical precedents in ancient language. Wendy and Jessa, you are pointing out their subversive nature — how activists and others are using new forms of language to circumvent traditional authorities.
AL: From a historical perspective, we’re certainly very rich in precedent — and in that sense I always feel a little surprised when people talk about modes of communication like emoji, or text-speak for that matter, as being somehow “new.” No — the technology driving them may be new, but the phenomenon of parallel scripts or using pictures to convey emotion (pictograms/determinatives) has been around for millennia. Society is currently adjusting to a new writing medium — the phone/tablet, or indeed even just the computer. In historical terms, even the computer is incredibly new. When paper first appeared in China, or papyrus in Egypt, there was a floruit of new written expression (it being a much more versatile medium than the stone surface which preceded it). Now we’ve got another medium which is more versatile than paper and, importantly, which has the capacity for instant deletion. In that sense, of course, emoji (or indeed anything typed on a computer) can afford to be more frivolous as its creator can send it back into the ether with one hit of the backspace. You can’t do that with an inscription.
DT: Well said, Alex. Jessa, Wendy, concluding thoughts?
DT: Thanks everyone. It’s been super;