Upgrade to Pro — share decks privately, control downloads, hide ads and more …

In the beginning was TXT

Avatar for Markus Wein Markus Wein
October 02, 2014
Programming
0
110

In the beginning was TXT

A very short overview of the history of encodings, given at Vienna.rb on 2014-10-02

Avatar for Markus Wein

Markus Wein

October 02, 2014
Tweet

More Decks by Markus Wein

See All by Markus Wein
Command Line Productivity
Avatar for Markus Wein cypher
1
130
A crash intro to deliberate practice
Avatar for Markus Wein cypher
0
110
Keeping Your PostgreSQL Data Save
Avatar for Markus Wein cypher
0
120
Ghost in the State Machine
Avatar for Markus Wein cypher
2
310
n Things You Didn't Know About PostgreSQL (Rubyslava & PyVo 2014 Edition)
Avatar for Markus Wein cypher
1
240
How to Become a Better Developer
Avatar for Markus Wein cypher
2
1.8k
An Introduction to Rust
Avatar for Markus Wein cypher
1
8.1k
How to Become a Better Developer
Avatar for Markus Wein cypher
1
230
A Very Short Overview of Vagrant
Avatar for Markus Wein cypher
0
7.9k

Other Decks in Programming

See All in Programming
あのころの iPod を どうにか再生させたい
Avatar for orumin orumin
2
2.5k
kiroでゲームを作ってみた
Avatar for 入井 啓太 iriikeita
0
170
Comparing decimals in Swift Testing
Avatar for 417.72KI 417_72ki
0
170
サイトを作ったらNFCタグキーホルダーを爆速で作れ！
Avatar for Yuuki Shimizu yuukis
0
370
WebAssemblyインタプリタを書く ～Component Modelを添えて～
Avatar for ruccho ruccho
1
860
新世界の理解
Avatar for Akihito Koriyama koriym
0
140
STUNMESH-go: Wireguard NAT穿隧工具的源起與介紹
Avatar for Date Huang tjjh89017
0
380
マイコンでもRustのtestがしたい その２/KernelVM Tokyo 18
Avatar for Toshifumi NISHINAGA tnishinaga
2
2.3k
コーディングは技術者(エンジニア)の嗜みでして / Learning the System Development Mindset from Rock Lady
Avatar for mackey0225 mackey0225
2
520
Honoアップデート 2025年夏
Avatar for Yusuke Wada yusukebe
1
770
「リーダーは意思決定する人」って本当？～ 学びを現場で活かす、リーダー4ヶ月目の試行錯誤 ～
Avatar for Marina Nakagawa marina1017
0
230
AI時代のドメイン駆動設計-DDD実践におけるAI活用のあり方 / ddd-in-ai-era
Avatar for MinoDriven minodriven
21
8.2k

Featured

See All Featured
[RailsConf 2023 Opening Keynote] The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
30
9.6k
Fantastic passwords and where to find them - at NoRuKo
Avatar for Phil Nash philnash
51
3.4k
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
2.8k
Faster Mobile Websites
Avatar for Dean Hume deanohume
309
31k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
Avatar for Sam Stephenson sstephenson
161
15k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
48
9.6k
Sharpening the Axe: The Primacy of Toolmaking
Avatar for Bryan Cantrill bcantrill
44
2.4k
Building an army of robots
Avatar for Kyle Neath kneath
306
45k
Why Our Code Smells
Avatar for Brandon Keepers bkeepers
PRO
338
57k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
Avatar for Eileen M. Uchitelle eileencodes
139
34k
Performance Is Good for Brains [We Love Speed 2024]
Avatar for Tammy Everts tammyeverts
10
1k
Testing 201, or: Great Expectations
Avatar for Joseph Mastey jmmastey
45
7.6k

Transcript

  1. In the beginning was TXT

  2. !

  3. EBCDIC

  4. Source: http://en.wikipedia.org/wiki/EBCDIC

  5. ASCII

  6. "#$%&

  7. None

  8. ä, ö, or å and Ø?

  9. Latin-1 ISO/IEC 8859-1

  10. Latin-*

  11. Windows code pages

  12. Then came the €

  13. (

  14. None

  15. Shift-JIS

  16. This sucks

  17. Unicode!

  18. Unicode!

  19. ✈️ (planes!)

  20. Basic Multilingual Plane

  21. Code Points

  22. U+0041 (LATIN SMALL LETTER A)

  23. Source: http://codepoints.net/U+0041

  24. Grapheme

  25. a a a a a a a

  26. Composite characters

  27. U+0065 U+0301 or U+00E9

  28. e+´ => é é

  29. ´ != ´

  30. Unicode… is not an encoding

  31. UTF-32

  32. UCS-2/UTF-16

  33. UTF-8

  34. Source: http://en.wikipedia.org/wiki/File:UnicodeGrow2b.png

  35. What does it look like?

  36. Codepoint Char ASCII Latin-1 ISO-8859-15 UTF-8 UTF-16 U+0041 A 0x41

    0x41 0x41 0x41 0x00 0x41 U+00C4 Ä - 0xc4 0xc4 0xc3 0x84 0x00 0xc4 U+20AC € - - 0xa4 0xe3 0x82 0xac 0x20 0xac U+C218 ࣻ - - - 0xec 0x88 0x98 0xc2 0x18 Encoding comparison Source: http://perlgeek.de/en/article/encodings-and-unicode

  37. Remember: Just because someone claims it’s UTF-8, doesn’t mean it

    is