Upgrade to Pro — share decks privately, control downloads, hide ads and more …

In the beginning was TXT

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Markus Wein Markus Wein
October 02, 2014
Programming
130
0

In the beginning was TXT

A very short overview of the history of encodings, given at Vienna.rb on 2014-10-02

Avatar for Markus Wein

Markus Wein

October 02, 2014

More Decks by Markus Wein

See All by Markus Wein
Command Line Productivity
Avatar for Markus Wein cypher
1
170
A crash intro to deliberate practice
Avatar for Markus Wein cypher
0
130
Keeping Your PostgreSQL Data Save
Avatar for Markus Wein cypher
0
150
Ghost in the State Machine
Avatar for Markus Wein cypher
2
340
n Things You Didn't Know About PostgreSQL (Rubyslava & PyVo 2014 Edition)
Avatar for Markus Wein cypher
1
260
How to Become a Better Developer
Avatar for Markus Wein cypher
2
1.8k
An Introduction to Rust
Avatar for Markus Wein cypher
1
8.3k
How to Become a Better Developer
Avatar for Markus Wein cypher
1
260
A Very Short Overview of Vagrant
Avatar for Markus Wein cypher
0
8k

Other Decks in Programming

See All in Programming
From Formal Specification to Property Based Test
Avatar for Masato Ohba ohbarye
0
2.4k
How We Practice Exploratory Testing in Iterative Development( #scrumniigata ) / 反復開発の中で、探索的テストをどう実施しているか
Avatar for teyamagu teyamagu
PRO
3
780
Terraform言語の静的解析 / static analysis of Terraform language
Avatar for Kazuma Watanabe wata727
1
140
Claude CodeでETLジョブ実行テストを自動化してみた
Avatar for yoshiki kasama yoshikikasama
0
1.2k
Liberating Ruby's Parser from Lexer Hacks
Avatar for ydah ydah
2
2.7k
When benchmarks go bad - what I learned from measuring performance wrong
Avatar for Holly Cummins hollycummins
0
380
PHPでローカル環境用のSSL/TLS証明書を発行することはできるのか？ #phpconkagawa
Avatar for akase244 akase244
0
350
継続的な負荷検証を目指して
Avatar for Kazuhiko Yamashita pyama86
3
1k
cloudnative conference 2026 flyle
Avatar for azihsoyn azihsoyn
0
160
Spec Driven Development | AI Summit Vilnius
Avatar for Daniel Sogl danielsogl
PRO
1
150
Structured Concurrency, Scoped Values and Joiners in the JDK 25 26 27
Avatar for José josepaumard
1
150
ふにゃっとしない名前の付け方 〜哲学で茹で上げる、コシのあるソフトウェア設計〜
Avatar for akshimo shimomura
0
110

Featured

See All Featured
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
1.9k
What Being in a Rock Band Can Teach Us About Real World SEO
Avatar for Ade Holder 427marketing
0
230
Java REST API Framework Comparison - PWX 2021
Avatar for Matt Raible mraible
34
9.3k
Building a A Zero-Code AI SEO Workflow
Avatar for Ian Lurie portentint
PRO
0
500
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k
Future Trends and Review - Lecture 12 - Web Technologies (1019888BNR)
Avatar for Beat Signer signer
PRO
0
3.5k
Bioeconomy Workshop: Dr. Julius Ecuru, Opportunities for a Bioeconomy in West Africa
Avatar for AKADEMIYA2063 akademiya2063
PRO
1
110
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
61
9.8k
A Soul's Torment
Avatar for Nat Ma seathinner
6
2.8k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
Designing Dashboards & Data Visualisations in Web Apps
Avatar for Des Traynor destraynor
231
54k
Improving Core Web Vitals using Speculation Rules API
Avatar for Sergey Chernyshev sergeychernyshev
21
1.5k

Transcript

  1. In the beginning was TXT

  2. !

  3. EBCDIC

  4. Source: http://en.wikipedia.org/wiki/EBCDIC

  5. ASCII

  6. "#$%&

  7. None

  8. ä, ö, or å and Ø?

  9. Latin-1 ISO/IEC 8859-1

  10. Latin-*

  11. Windows code pages

  12. Then came the €

  13. (

  14. None

  15. Shift-JIS

  16. This sucks

  17. Unicode!

  18. Unicode!

  19. ✈️ (planes!)

  20. Basic Multilingual Plane

  21. Code Points

  22. U+0041 (LATIN SMALL LETTER A)

  23. Source: http://codepoints.net/U+0041

  24. Grapheme

  25. a a a a a a a

  26. Composite characters

  27. U+0065 U+0301 or U+00E9

  28. e+´ => é é

  29. ´ != ´

  30. Unicode… is not an encoding

  31. UTF-32

  32. UCS-2/UTF-16

  33. UTF-8

  34. Source: http://en.wikipedia.org/wiki/File:UnicodeGrow2b.png

  35. What does it look like?

  36. Codepoint Char ASCII Latin-1 ISO-8859-15 UTF-8 UTF-16 U+0041 A 0x41

    0x41 0x41 0x41 0x00 0x41 U+00C4 Ä - 0xc4 0xc4 0xc3 0x84 0x00 0xc4 U+20AC € - - 0xa4 0xe3 0x82 0xac 0x20 0xac U+C218 ࣻ - - - 0xec 0x88 0x98 0xc2 0x18 Encoding comparison Source: http://perlgeek.de/en/article/encodings-and-unicode

  37. Remember: Just because someone claims it’s UTF-8, doesn’t mean it

    is