Evolution of Emoji 🖼🔤✨

A presentation on emoji history.
Presented at DurasnoConf in Durazno, Uruguay.

Part of the deck includes GIFs, you can see them in this tweet thread.
https://twitter.com/kosamari/status/1047978154143506434

Transcript

1. (This is a slides from my talk I gave at DuraznoConf in Durazno, Uruguay. October 4th 2018) (Hand drawn illustrations are by me @kosamari, and each pictures used have credit at the bottom) Hello Durazno() Conf !
2. I’m Mariko, you can find me as @kosamari on twitter. I’m a web developer form NYC working with team that builds Google Chrome.
 Say hi or send me questions!
3. I usually write things like how does browser work & how to make compiler. With …
4. … hand drawn illustrations like these.
5. Because I really like learning about complex topics and explaining in a simple way.
6. … so of-cause my compiler post has hand drawn pictures too :)
7. Today, I want to tell you about emoji! Where it started, and how it evolved to be what it is now.
8. But why? It’s a mix of many reasons why I’m interested in Emoji.
 I studied communication in university. I’m fascinated by how people comunicate especially using computer technologies and the internet.
 I also love history. Especially computing history. I can get lost in reading old mailing list archive or archival video.
9. I love discovering cause and effects in history. 
 Perhaps that’s the reason why I ended up in web development. I build tools for communication, by dealing with cause and effects everyday in code.
10. So, emoji - Even though it might look like it comes from “emoticons”… it is actually a Japanese word.
11. …which literally translates to picture letters.
12. So how did emoji start?
13. If you google “first emoji” you will find this one. MoMA actually acquired original emoji to their collection.
14. This “original” emoji set is from i-mode - a Japanese mobile web network started in 1999
15. So in 1999, you could buy a phone that look like this, which had a browser and email client. You could select emoji in your text message or if you are web developer, you could use emoji on your page.
16. If you go to Computer History Museum in Mountain View, they describe i-mode as “first popular system for mobile Web access”
17. Even thought i-mode was the first one that included a set of emoji. Emoji was used in communication device already. This Japanese version of iPhone (DP-211SW) had touch pad, supported Kanji characters, and emoji (spot the !).
 Also, pager was really popular device, from business person to high school girls. If you dial 88, certain pagers displayed — which was very popular.
18. So in 1997, NTT DoCoMo starts i-mode project. 2 years before it’s release.
19. To research for this new mobile web service, a project member was sent to California to try AT&T’s PocketNet service, which had mobile web access. Shigetaka Kurita - who was testing Pocket Net would send email from the PocketNet phone on a train in SanFrancisco, make international phone call to the office in Tokyo and ask if they received it. 

20. While using PocketNet, like looking up weather, Kurita noticed that reading weather in text was rather dry… and wondered if he can use weather icons like they do in TV weather forecast.
21. Kurita comes back home and pitch the emoji idea to phone device manufacturer. Manufacturer says “Fine, we’ll do it, but you have to tell us exactly what you want (in bitmap art)” So Kurita was tasked to design emoji set.
22. But why emoji needed to be text and displayed as font? Why not image? Because back then i-mode had to have separate sockets - one for text and one for image, so that was not only slow to download but also used lot more data. Also, i-mode only supported GIF for image format. Back in late 1990s, patent of GIF was still not clear so they wanted to stay out of potential trouble.
23. If icons are delivered as text (or code point) then they can save connection and data because it is delivered as text and pictogram is already installed in the phone.
24. So at the end of the design process, this was the first emoji set.
25. It has weather icons just like Kurita imagined while using PocketNet.
26. Since they were designing information portal, Rock Paper Scissors were added for potential game use.
27. Kurita says visualizing “location service” was difficult task but ended up with “a pin” which you can see later in services like Google Maps.
28. Emoticons were added for expressive communication.
29. And 4 different kinds of hearts ! Kurita worked as sales person for pager (pocket bell) and experienced 1st hand some customers turn away from device because it didn’t support —. So this time, he made sure there were hearts included.
30. Interesting one is Cars, original set had 2 different types of car. 
 Back then Kurita was really into snowboarding so regular town car and SUV for outdoor was different thing for him :)
31. You might notice there is no religion or race/gender related icons in the original set. Kurita says “I purposefully did not include emoji related to religion, country, race, or gender to avoid future conflicts”. Perhaps because it was done in conservative place (NTT - DoCoMo’s parent company was THE biggest phone company which used to be owned by government). Perhaps it was great vision in designing features that has never existed before.
32. While I was researching, 2 resources came up often as inspirations for emoji.
33. 1st is Tokyo Olympic 1964. This was the first Olympic game hosted in Asia. Signs written in Japanese does not even use alphabets, so Olympic organizers needed to come up a way to communicate with international audience. They used pictogram for that. Each sports had its own pictogram (this became Olympic tradition, every Olympic game since then has their own pictograms.)
34. Not only for type of sports, they also defined informational icons like bathroom sign and restaurant/ cafe signs. This is a news paper reporting the design effort back in 1064.
 
 At the end of design process, designers abandoned their copyright & released pictograms they designed as public domain so that it can be used everywhere.
35. So in original emoji, you see trace of that universal signs.
36. Another inspiration is Japanese comic book (or Manga).
37. Manga has it’s own visual idioms to communicate emotions. This might not look anything to you, but to me who grew up reading manga and immersed in this visual communication, I can see 3 different scene/emotions just by adding small visual elements. (surprised, anxious (cold sweat), sad/devastated (shadow))
38. You can see those visual idioms in the original emoji set
39. So when I read text like this, as Japanese who is trained in those visual idiom, I read the text in much more comical/expressive way than plain text.
40. In a way, I think emoji is still under used for its original purpose.
41. This is a “text only” NPR site created in response to hurricane. There are no image/video/ads so that people can read news even in the unreliable connection.
42. I think this is a exactly the situation emoji can shine! Emoji can add more visual flare without adding any more data (because emoji fonts are already installed in the device). Plus, don’t you think it’d be nice to have more visual differentiator when you are in the stressful mental state, in stead of trying to parse wall of text?
43. Let’s look at what happened after original emoji was introduced.
44. Other phone companies followed DoCoMo’s success and provided their own emoji for their service.
45. In fact, emoji became a competitive aspects of phone service. Companies advertised “we have more emoji!” Or “our emoji animates!”. In result, a lot more emoji were introduced to the market.
46. So let’s look at how emoji was handled on encoding level.
47. In many encoding system our phone or computers turn data they received into a reference (like code points) and use that to look up letters. ASCII or Central European encoding has been used by languages that use alphabets, for Japanese we have one called Shift-JIS that supports a lot more characters that we use in Japanese. Shift-JIS also have empty space left for application specific use, so emoji used this space.
48. Things to note here is that each phone company defined their own emoji system. There were no interoperability.
49. So back in 2000 if you send emoji from Phone company A to B, emoji would turn into ˤ(geta). In modern days it might be □(tofu).
50. Finally in 2006, three major phone companies in Japan made it possible to use emoji interchangeably.
51. It was done as conversion table system.
52. And unfortunate bug (like sending music note emoji turn into poop) occurred.
53. Conversation table system is not really optimal, each company has to maintain table and if new company comes into market, they have to create new table again. Better approach may be having single table to look up to (this will come in later!)
54. To recap, emoji was release to public in 1999, gained popularity in Japan in 2000s.
55. Next comes globalization phase.
56. While emoji mature in Japanese market, it started gaining adoption outside of Japan.
57. Gmail supported emoji input in 2008. This is also the time Google was trying to break into Japanese market with gmail. To get adoption in Japan as communication service, emoji support was a kay.
58. iPhone first supported emoji only in Japanese market with iOS 2.2.
 It is rumored Masayoshi Son, CEO of Softbank which had exclusive rights to sell iPhone in Japan, asked Steve Jobs to support emoji in iPhone in order to sell them in Japan.
59. With those activities happening, Google and Apple delegations decides to propose emoji to Unicode.
60. Unicode is a project to document every letters for every language in one place. 
 Systems that support Unicode can reference one master source. (no more encoding differences).
61. Here is a screen shot from the proposal.
 Of-cause, some people questioned if emoji really need to be included in Unicode.
62. Rationale for including emoji in Unicode was 1. Interoperability 2. Already done same for ARIB 
 ARIB symbols are Japanese broad cast specific symbols which are brought into Unicode before emoji.
63. When you read mailing list discussion from back then, you see discussion like “do you really need 4 different kind of books?” “Do you really want to include these racial stereotype emoji?” “why don’t my country have emoji flag?”.
64. Flag is an interesting one. It was proposed as FLAG_SYMBOL_[country] but flag designs could change. So next proposal was to change to EMOJI_SYMBOL_[country] and let implementers decide how to display them. But country could also change, so it was settled as combination of 2 REGIONAL_INDICATOR_SYMBOLS. (visual representation is still up-to implementers. In reality mostly displayed as flag).
65. Because Israel’s country code is IL and Liechtenstein’s country code is LI. If you put array of Israel flag emoji in code editor and hit back space, you see Liechtenstein flag show up.
66. Flag and country is a highly political subject. Recently there was a bug which crashed iPhone if iPhone user with region set to China received Taiwanese flag. Since Taiwan is not recognized as country in China, Apple added system to mask Taiwanese flag if Chinese user received Taiwanese flag, unintentionally this caused the bug that crashed iPhone entirely.
67. Another interesting one from this time is this document made by Japanese publisher explaining Japanese comic expressions.
68. They noticed proposed emoji example (in orange line) was loosing original meaning. They explained the manga visual idioms and proposed change (in green line).
69. In the process of defining emoji for Unicode, skin tone modifier was added to support diverse skin colors.
70. In order to display emoji with different skin color, a default emoji and skin tone modifier are sent as set. It is then displayed as one emoji if the system supports them or 2 (default emoji + skin tone) if not supported.
71. Ways to create compound emoji was developed in order to create more emoji from existing emoji.
72. Compound emoji use thing called zero width joiner which glues multiple emoji into one.
73. A family is constructed from 4 emoji…
74. … so if you type family emoji and hit backspace in code editor, you can see emoji changes, deleting one family member at a time.
75. Two ladies kissing is constructed with two women, a heart, and a kiss emoji.
76. Rainbow flag is made of plain flag and rainbow emoji.
77. This is how emoji with different profession are made.
78. Lady computer programmer is Woman + Computer
79. Lady mechanic is Woman + Wrench
80. Lady justice is Woman + Scales
81. To recap, emoji adoption in different services caused emoji to be included in Unicode which brought emoji to more universal icon set.
82. So what’s the future of emoji?
83. Probably the most interesting thing about emoji is that you get to help define new emoji. I think it’s the only language in Unicode that is actively developed in stead of just documenting existing letters.
84. Unicode has a process to propose new emoji. There is a guideline so you can’t just add your brand’s icon as emoji, but if you think certain emoji would be universally useful, you can make your own proposal.
85. For example, Apple proposed accessibility emoji set that includes guide dog and person with cane. It is now a draft candidate so this might make it to official unicode someday!
86. This is how person with headscarf (Hijab) emoji was added. A lady who wear hijab thought her community should be represented in emoji so she contacted Unicode.
87. Dumpling or Empanada depending on where you are, was also added this way.
88. Salt shaker emoji is in newest version of Unicode and coming to device near you soon!
89. And that’s muy salado right? (I was told in Uruguay Spanish muy salado is a slang for “very good”). Thank you for listening! If you have any questions, comments, concerns, please send me tweet at @kosamari