Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Monadic Parsing in Python
Search
Oleksii Kachaiev
June 06, 2014
Programming
13
7k
Monadic Parsing in Python
Oleksii Kachaiev
June 06, 2014
Tweet
Share
More Decks by Oleksii Kachaiev
See All by Oleksii Kachaiev
Counting HTTP with QUIC & HTTP/3
kachayev
1
230
Talking SQL to Strangers
kachayev
3
500
Counting HTTP: 0.9...3
kachayev
1
60
Managing Data Chaos in The World of Microservices
kachayev
3
610
Deep HTTP Dive Through Aleph & Netty
kachayev
6
3.6k
Keep Your Data Safe With Refined Types
kachayev
4
1.4k
Clojure at Attendify (2nd ed)
kachayev
5
1.5k
Clojure at Attendify
kachayev
4
310
Finagle & Clojure
kachayev
6
1.3k
Other Decks in Programming
See All in Programming
CSC509 Lecture 08
javiergs
PRO
0
110
讓數據說話:用 Python、Prometheus 和 Grafana 講故事
eddie
0
350
Outline View in SwiftUI
1024jp
1
160
LLM生成文章の精度評価自動化とプロンプトチューニングの効率化について
layerx
PRO
2
140
Generative AI Use Cases JP (略称:GenU)奮闘記
hideg
0
160
プロジェクト新規参入者のリードタイム短縮の観点から見る、品質の高いコードとアーキテクチャを保つメリット
d_endo
1
1k
Content Security Policy入門 セキュリティ設定と 違反レポートのはじめ方 / Introduction to Content Security Policy Getting Started with Security Configuration and Violation Reporting
uskey512
1
430
Jakarta Concurrencyによる並行処理プログラミングの始め方 (JJUG CCC 2024 Fall)
tnagao7
1
230
GCCのプラグインを作る / I Made a GCC Plugin
shouth
1
150
飲食業界向けマルチプロダクトを実現させる開発体制とリアルな現状
hiroya0601
1
400
Googleのテストサイズを活用したテスト環境の構築
toms74209200
0
270
ECSのサービス間通信 4つの方法を比較する 〜Canary,Blue/Greenも添えて〜
tkikuc
11
2.3k
Featured
See All Featured
Java REST API Framework Comparison - PWX 2021
mraible
PRO
28
7.9k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
42
9.2k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
46
2.1k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
We Have a Design System, Now What?
morganepeng
50
7.2k
Designing the Hi-DPI Web
ddemaree
280
34k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
31
2.7k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
37
1.8k
Teambox: Starting and Learning
jrom
132
8.7k
Fireside Chat
paigeccino
32
3k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
231
17k
What's in a price? How to price your products and services
michaelherold
243
12k
Transcript
Monadic Parsing in Python Alexey Kachayev, 2014
About me • CTO at Attendify.com • Erlang, Clojure, Go,
Haskell • Fn.py library author • CPython & Storm contributor
Find me •@kachayev •github.com/kachayev •kachayev <$> gmail.com
Topic
Will talk •What is "parsing(ers)" •Approaches •Monadic parsing from scratch
•More…
Will talk •Less about theory •Much more about practice
Won’t talk •What "monad" is •Why FP is cool (*)
* you’ll understand it by yourself
Parsing
Definition •Takes grammar •Takes input string (?) •Returns tree (??)
or an error
None
For PL creators only?
Tasks • Processing information from logs • Source code analysing
• DSLs • Protocols & data formats • … and more
Approaches
Production rule S → SS|(S)|()
Grammar block = ["const" ident "=" number {"," ident "="
number} ";"] ["var" ident {"," ident} ";"] {"procedure" ident ";" block ";"} statement ! expression = ["+"|"-"] term {("+"|"-") term} ! term = factor {("*"|"/") factor} ! factor = ident | number | "(" expression ")" ! . . . .
•Top-down / bottom-up •Predictive / Backtracking •LL(k), LALR, LR, CYK
and others In theory
Manually!
@ wikipedia
Manually •Simple to understand •Hard to maintain •Really boring
Can we do better?
What we have •Context-free grammars •Formal theory •Well-defined algorithms •Standard
grammar notation(s)
So…
Parser generator •1. Parse DSL notation •2. Generate parser code
•("any" language)
Parser generator •*PEG* •*Yacc* •ANTLR •… and tens more
Parser generator •Pros •many targeted languages •formalism •performance & optimisations
Parser generator •Cons •another language •bounded in features •"compiled-time" mostly
Can we do better?
Monadic parsers & combinators
Functional Pearls Monadic Parsing in Haskell @Graham Hutton, @Erik Meijer
Parsec MPC library for Haskell
Parsec •Monadic parser combinator(s) •Works even with context- sensitive, infinite
LA grammars •Tens of ports to other langs
None
The Big Idea
Simple type Parser = String → Tree
Compose? type Parser = String → (Tree, String)
Generalize? type Parser a = String → (a, String)
Errors? type Parser a = String → Maybe (a, String)
Or better… type Parser a = String → [(a, String)]
Let’s try…
Snippets: http://goo.gl/leQIEE
None
None
None
None
… and so?
Expressiveness •[] for error •[s1] for single (predictive) •[s1..sN] for
backtracking
First-class citizen
None
Skip anything…
Recognise digit
Combinators
RegExp •and: "abc" •or: "a | b | c" •Kleene
star: "a*"
Derives •a? = "" | a •a+ = aa* •a{2,3}
= aa | aaa
None
None
laziness is cool for this do you need backtracking?
How to use it?
None
None
Cool! but..
ugly ugly not readable
Enhancements •use generators for "laziness" •"combine" function •Scala-style methods •"delay"
method
fn.py Stream
None
[1,2,3,4,5] expr →"[" digit (","digit)* "]"
None
Interesting! but..
Is it enough?
In Haskell
Can I do this in Python?
… hm
Challenge accepted!
In Python
How?
Desugaring…
What?
WAT??? even more like
unit a → Parser a
bind Parser a → (a → Parser b) → Parser
b
lift (a → b) → (a → Parser b)
lifted Parser a → (a → b) → Parser b
WAT??? ok, looks cool, but
None
None
How to use
And even more..
Haskell-style
Do-notation
None
None
(define R 2) (define diameter (lambda (r) (* 2 r)))
None
None
Looks nice!
Mutability kills backtracking :(
And more •errors handling •backtracking control •performance
Links • "funcparselib" http://goo.gl/daidQY • "Monadic parsing in Haskell" http://goo.gl/gygNlM
• "Higher-Order functions for Parsing" http://goo.gl/c8VOIZ • "Parsec" http://goo.gl/bdnDZQ • "Parcon" http://goo.gl/CT06S5 • "Pyparsing" http://goo.gl/gmr2lQ • "You Could Have Invented Monadic Parsing" http://goo.gl/h0rnOQ
Learn Haskell For Great Good
Q/A thanks for your attention,