Upgrade to Pro — share decks privately, control downloads, hide ads and more …

In the beginning was TXT

Avatar for Markus Wein Markus Wein
October 02, 2014
Programming
140
0

In the beginning was TXT

A very short overview of the history of encodings, given at Vienna.rb on 2014-10-02

Avatar for Markus Wein

Markus Wein

October 02, 2014

More Decks by Markus Wein

See All by Markus Wein
Command Line Productivity
Avatar for Markus Wein cypher
1
180
A crash intro to deliberate practice
Avatar for Markus Wein cypher
0
140
Keeping Your PostgreSQL Data Save
Avatar for Markus Wein cypher
0
150
Ghost in the State Machine
Avatar for Markus Wein cypher
2
350
n Things You Didn't Know About PostgreSQL (Rubyslava & PyVo 2014 Edition)
Avatar for Markus Wein cypher
1
270
How to Become a Better Developer
Avatar for Markus Wein cypher
2
1.8k
An Introduction to Rust
Avatar for Markus Wein cypher
1
8.4k
How to Become a Better Developer
Avatar for Markus Wein cypher
1
270
A Very Short Overview of Vagrant
Avatar for Markus Wein cypher
0
8.1k

Other Decks in Programming

See All in Programming
Inside Stream API
Avatar for Yuichi.Sakuraba skrb
1
740
メソッドのジェネリクスでGoの夢は広がるか？ / Kyoto.go #65
Avatar for utagawa kiki utgwkk
3
850
IBM Bobを活用したレガシーアプリの最新化
Avatar for Akira Onishi (IBM) oniak3ibm
PRO
1
200
OSもどきOS
Avatar for Sora Arakawa arkw
0
570
Observability in Practice：Grafana 與 Edge Device SRE 的那些事
Avatar for Blueswen blueswen
0
170
[2026年度第1回ORセミナー] 計画最適化ベンチャーと競技プログラミング人材
Avatar for terry-u16 terryu16
0
270
The NotImplementedError Problem in Ruby
Avatar for Koichi ITO koic
1
850
Java × distroless で 軽量なコンテナイメージを / Java on Distroless
Avatar for Daisuke Garaike contour_gara
0
550
New "Type" system on PicoRuby
Avatar for pocke pocke
1
980
Creating Composable Callables in Contemporary C++
Avatar for Björn Fahller rollbear
0
150
キャリア迷子上等 ─ "ない道"は自分で作ればいい
Avatar for 16bit_idol 16bitidol
3
2.2k
AI時代のUIはどこへ行く？その2！
Avatar for Yusuke Wada yusukebe
22
7.4k

Featured

See All Featured
First, design no harm
Avatar for Per Axbom axbom
PRO
2
1.2k
Joys of Absence: A Defence of Solitary Play
Avatar for Sebastian Deterding codingconduct
1
400
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
3.5k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
56k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
37
6.5k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1370
210k
Performance Is Good for Brains [We Love Speed 2024]
Avatar for Tammy Everts tammyeverts
12
1.7k
The State of eCommerce SEO: How to Win in Today's Products SERPs - #SEOweek
Avatar for Aleyda Solis aleyda
2
11k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
10k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
Avatar for Vitaly Friedman smashingmag
254
22k
Visualization
Avatar for Eitan Lees eitanlees
152
17k
Efficient Content Optimization with Google Search Console & Apps Script
Avatar for Katarina Dahlin katarinadahlin
PRO
1
630

Transcript

  1. In the beginning was TXT

  2. !

  3. EBCDIC

  4. Source: http://en.wikipedia.org/wiki/EBCDIC

  5. ASCII

  6. "#$%&

  7. None

  8. ä, ö, or å and Ø?

  9. Latin-1 ISO/IEC 8859-1

  10. Latin-*

  11. Windows code pages

  12. Then came the €

  13. (

  14. None

  15. Shift-JIS

  16. This sucks

  17. Unicode!

  18. Unicode!

  19. ✈️ (planes!)

  20. Basic Multilingual Plane

  21. Code Points

  22. U+0041 (LATIN SMALL LETTER A)

  23. Source: http://codepoints.net/U+0041

  24. Grapheme

  25. a a a a a a a

  26. Composite characters

  27. U+0065 U+0301 or U+00E9

  28. e+´ => é é

  29. ´ != ´

  30. Unicode… is not an encoding

  31. UTF-32

  32. UCS-2/UTF-16

  33. UTF-8

  34. Source: http://en.wikipedia.org/wiki/File:UnicodeGrow2b.png

  35. What does it look like?

  36. Codepoint Char ASCII Latin-1 ISO-8859-15 UTF-8 UTF-16 U+0041 A 0x41

    0x41 0x41 0x41 0x00 0x41 U+00C4 Ä - 0xc4 0xc4 0xc3 0x84 0x00 0xc4 U+20AC € - - 0xa4 0xe3 0x82 0xac 0x20 0xac U+C218 ࣻ - - - 0xec 0x88 0x98 0xc2 0x18 Encoding comparison Source: http://perlgeek.de/en/article/encodings-and-unicode

  37. Remember: Just because someone claims it’s UTF-8, doesn’t mean it

    is