Upgrade to Pro — share decks privately, control downloads, hide ads and more …

the world of characters

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for orisano orisano
September 13, 2018
8
1.5k

the world of characters

Avatar for orisano

orisano

September 13, 2018
Tweet

More Decks by orisano

See All by orisano
OSS Performance Tuning Tips
Avatar for orisano orisano
8
6.1k
Docker-Compose & BuildKit
Avatar for orisano orisano
4
1k
Container Build Talk
Avatar for orisano orisano
3
2.6k
dockerignore talk
Avatar for orisano orisano
2
7.3k
Better docker image+
Avatar for orisano orisano
6
6.5k
Socket.IO Introduction
Avatar for orisano orisano
0
3.3k
Profiling Go Application
Avatar for orisano orisano
11
8k
Multi-stage Builds Patterns & Practice
Avatar for orisano orisano
6
5.3k
better docker image
Avatar for orisano orisano
22
30k

Featured

See All Featured
The untapped power of vector embeddings
Avatar for Frank van Dijk frankvandijk
2
1.6k
Agile that works and the tools we love
Avatar for Rasmus Luckow-Nielsen rasmusluckow
331
21k
Noah Learner - AI + Me: how we built a GSC Bulk Export data pipeline
Avatar for Tech SEO Connect techseoconnect
PRO
0
140
Beyond borders and beyond the search box: How to win the global "messy middle" with AI-driven SEO
Avatar for David Carrasco davidcarrasco
3
74
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
Avatar for Michelle Schulp Hunt marktimemedia
31
2.7k
Primal Persuasion: How to Engage the Brain for Learning That Lasts
Avatar for Mike Taylor tmiket
0
290
Easily Structure & Communicate Ideas using Wireframe
Avatar for Afnizar Nur Ghifari afnizarnur
194
17k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
Avatar for Chris Coyier chriscoyier
508
140k
HDC tutorial
Avatar for Michiel Stock michielstock
1
540
Navigating Algorithm Shifts & AI Overviews - #SMXNext
Avatar for Aleyda Solis aleyda
1
1.2k
Stewardship and Sustainability of Urban and Community Forests
Avatar for Eric Wiseman pwiseman
0
140
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
50
14k

Transcript

  1. 1จࣈͷੈք @orisano

  2. Έͳ͞Μ จࣈΛ਺͑ΒΕ·͢ΑͶʁ

  3. a

  4. a => 1

  5. ͋

  6. ͋ => 1

  7. 佛

  8. 佛 => 1

  9. None

  10. => 1

  11. None

  12. => 1

  13. Z͑ͫ̓ͪ̂ͫ̽ ̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔ ͫ͗ ͢ L̠ͨͧͩ͘ G̴̻͈͍͔̹ ̑͗̎̅͛ ́ Ǫ̵̹̻̝̳ ͂̌

    ̌͘! ͖̬̰̙̗ ̿̋ ͥ ͥ̂ͣ̐́́͜͞

  14. Z͑ͫ̓ͪ̂ͫ̽ ̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔ ͫ͗ ͢ L̠ͨͧͩ͘ G̴̻͈͍͔̹ ̑͗̎̅͛ ́ Ǫ̵̹̻̝̳ ͂̌

    ̌͘! ͖̬̰̙̗ ̿̋ ͥ ͥ̂ͣ̐́́͜͞ => 6

  15. Έͳ͞Μ όΠτ਺Λ਺͑ΒΕ·͔͢ʁ (UTF-8)

  16. a

  17. a => 1

  18. ͋

  19. ͋ => 3

  20. 佛

  21. 佛 => 4

  22. None

  23. => 4

  24. None

  25. => 18

  26. Z͑ͫ̓ͪ̂ͫ̽ ̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔ ͫ͗ ͢ L̠ͨͧͩ͘ G̴̻͈͍͔̹ ̑͗̎̅͛ ́ Ǫ̵̹̻̝̳ ͂̌

    ̌͘! ͖̬̰̙̗ ̿̋ ͥ ͥ̂ͣ̐́́͜͞

  27. Z͑ͫ̓ͪ̂ͫ̽ ̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔ ͫ͗ ͢ L̠ͨͧͩ͘ G̴̻͈͍͔̹ ̑͗̎̅͛ ́ Ǫ̵̹̻̝̳ ͂̌

    ̌͘! ͖̬̰̙̗ ̿̋ ͥ ͥ̂ͣ̐́́͜͞ => 143

  28. ͋ͳ͕ͨࢥ͏1จࣈ͸ Ͳ͏਺͑Δ΂͖͔ʁ

  29. byte਺Ͱ͸਺͑ΒΕͳ͍

  30. Unicode͸จࣈू߹ จࣈͱ਺஋͕ରԠ͢Δ

  31. ͋ => 3042

  32. => 1F914

  33. ͜ͷ਺஋ͷ͜ͱΛ ίʔυϙΠϯτ ͱݺͿ

  34. ͜ͷίʔυϙΠϯτΛ byteྻͰදݱ͢Δํ๏Λ ΤϯίʔσΟϯάͱ͍͏

  35. UTF-8ͱ͔UTF-16ͱ͔͸ ΤϯίʔσΟϯάͷҰछ

  36. ͱΓ͋͑ͣ ίʔυϙΠϯτΛ਺͑Ε͹ ղܾʁ

  37. ͍͍͑

  38. => 1F468 + 200D + 1F469 + 200D + 1F466

  39. ࣮͸ෳ਺ͷίʔυϙΠϯτͰ ҰͭͷจࣈʹͳͬͨΓ͢Δ

  40. ਓ͕ؒೝ͍ࣝͯ͠Δ̍จࣈ͸ ॻهૉ(Grapheme cluster) ͱݺ͹Ε͍ͯΔ

  41. Ͳ͏΍Ε͹ ίʔυϙΠϯτͷྻ͔Β ॻهૉΛऔΓग़ͤΔ͔

  42. ίʔυϙΠϯτ͕ؒ ॻهૉڥքʹͳΔ͔Ͳ͏͔ͷ ݫີͳϧʔϧ͕͋Δ

  43. UAX #29 Unicode Text Segmentation

  44. None

  45. ͜ΕΛJSͰ࣮૷ͯ͠·ͨ͠ github.com/orisano/graphemesplit

  46. ৄ͘͠͸ UAX #29 Λݟͯ http://unicode.org/reports/tr29/

  47. ݟ஌Β͵ਓʹʓจࣈͱ ݴΘΕͨͱ͖ʹ͸ ͪΌΜͱ֬ೝ͠Α͏ʂ

  48. 1 byte? 1 codepoint? 1 grapheme cluster?