Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python DSL
Search
Elasticsearch Inc
March 11, 2015
Technology
2
930
Python DSL
Slides for Honza's talk at Elastic{on}
Elasticsearch Inc
March 11, 2015
Tweet
Share
More Decks by Elasticsearch Inc
See All by Elasticsearch Inc
OSCON: Scaling a distributed engineering team from 50-250
elasticsearch
13
1.5k
Stuff a Search Engine Can Do
elasticsearch
17
1.7k
Using Elastic to monitor anything
elasticsearch
3
1.5k
Log all the things!
elasticsearch
4
1.2k
Why Elastic? @ 50th Vinitaly 2016
elasticsearch
5
1.9k
What's New In Elasticland?
elasticsearch
3
930
Kibana, Timelion, Graph Meetup
elasticsearch
3
790
Elastic for Time Series Data and Predictive Analytics
elasticsearch
4
3.1k
Elastic 2.0
elasticsearch
1
750
Other Decks in Technology
See All in Technology
フィンテック養成勉強会#54
finengine
0
120
UIテスト自動化サポート- Testbed for XCUIAutomation practice
notoroid
0
120
実践! AIエージェント導入記
1mono2prod
0
150
Observability infrastructure behind the trillion-messages scale Kafka platform
lycorptech_jp
PRO
0
130
Definition of Done
kawaguti
PRO
6
470
Amplifyとゼロからはじめた AIコーディング 成果と展望
mkdev10
1
370
AI技術トレンド勉強会 #1MCPの基礎と実務での応用
nisei_k
1
250
AWS アーキテクチャ作図入門/aws-architecture-diagram-101
ma2shita
29
9.5k
Oracle Cloud Infrastructure:2025年6月度サービス・アップデート
oracle4engineer
PRO
2
150
20250625 Snowflake Summit 2025活用事例 レポート / Nowcast Snowflake Summit 2025 Case Study Report
kkuv
1
240
2025/6/21 日本学術会議公開シンポジウム発表資料
keisuke198619
2
480
ローカルLLMでファインチューニング
knishioka
0
130
Featured
See All Featured
Code Reviewing Like a Champion
maltzj
524
40k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
16
940
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
How STYLIGHT went responsive
nonsquared
100
5.6k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Writing Fast Ruby
sferik
628
61k
A designer walks into a library…
pauljervisheath
206
24k
Build The Right Thing And Hit Your Dates
maggiecrowley
36
2.8k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
124
52k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.8k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
700
Transcript
Python DSL Honza Král @honzakral
{ } DSL 2
{ } DSL ? Don't you mean ORM? 3
{ } Current State { "query": { "filtered": { "query":
{ "function_score": { "query": { "bool": { "must": [ {"multi_match": { "fields": ["title^10", "body"], "query": "php"}}, {"has_child": { "child_type": "answer", "query": {"match": {"body": "python"}}}} ], "must_not": [ {"multi_match": { "fields": ["title", "body"], "query": "python"}} ] } }, "field_value_factor": {"field": "rating"} } }, "filter": {"range": {"creation_date": {"from": "2010-01-01"}}} }}, 4 "highlight": { "fields": { "title": {"fragment_size" : 50}, "body": {"fragment_size" : 50} } }, "aggs": { "tags": { "terms": {"field": "tags"}, "aggs": { "comment_avg": { "avg": {"field": "comment_count"} } } }, "frequency": { "date_histogram": { "field": "creation_date", "interval": "month" } } } } JSON DSL
{ } Now add a filter to it! 5
{ } Search Object s = Search(doc_type='question') 6
{ } Simple Query s = s.query('multi_match', fields=['title^10', 'body'], query='php')
7
{ } Compound Query s = s.query('has_child', child_type='answer', query=Q('match', body='python'))
8
{ } Q shortcut {"has_child": { "child_type": "answer', "query": {"match":
{"body": "python"}}}} Q({'has_child': { 'child_type': 'answer', 'query': {'match': {'body': 'python'}}}}) Q('has_child', child_type='answer', query=Q('match', body='python')) HasChild(child_type='answer', query=Match(body='python')) 9
{ } Query expressions Q(...) & Q(...) == Bool(must=[Q(...), Q(...)])
Q(...) | Q(...) == Bool(should=[Q(...), Q(...)]) ~Q(...) == Bool(must_not=[Q(..)]) 10
{ } Filter s = s.filter('range', creation_date={'from': date(2010, 1, 1)})
11
{ } Exclude s = s.query(~Q('multi_match', fields=['title^10', 'body'], query='python')) 12
{ } Manual query s.query = Q('function_score', query=s.query, field_value_factor={'field': 'rating'})
13
{ } Aggregations s.aggs.bucket('tags', 'terms', field='tags')\ .metric('comment_avg', 'avg', field='comment_count') s.aggs.bucket('frequency',
'date_histogram', field='creation_date', interval='month') 14
{ } Highlight ... s = s.highlight('title', 'body', fragment_size=50) 15
{ } Migration path s = Search.from_dict(my_glorious_query) s = s.filter('term',
tag='published') my_glorious_query = s.to_dict() 16 query at a time
{ } Response response = s.execute() for hit in response:
print(hit.meta.score, hit.title) for tag in response.aggregations.tags.buckets: print(tag.key, tag.avg_comments.value) 17 No more brackets!
{ } Persistence From Mapping to Model-like DocTypes 18
{ } Mapping DSL m = Mapping('article') m.field('published_from', Date()) m.field('title',
String(fields={'raw': String(index='not_analyzed')})) m.field('comments', Nested()) m['comments'].property('author', String()) m.save('index-name') m.update_from_es('index-name') 19
{ } DocType class Article(DocType): title = String() created_date =
Date() comments = Nested(properties={'author': String()}) class Meta: index = 'blog' def save(self, **kwargs): self.created_date = now() super().save(**kwargs) Article.init() Article.search()... Search(doc_type=Article) 20
{ } Configuration 21
{ } Connections connections.configure( default={'hosts': ['localhost'], 'sniff_on_start': True}, logging={ 'hosts':
['log1:9200', 'log2:9200'], 'timeout': 30, 'sniff_timeout': 120}) Search(using='logging') es = connections.get_connection() es.indices.delete(index='_all') 22
{ } Future 23
{ } More DSLs! Index Analyzers Settings ... 24
{ } FacetedSearch ? class MySiteSearch(FacetedSearch): doc_type = [Article, Comment]
fields = ['title', 'body'] published = DateHistogram( interval='week', field='published_date') category = Term(field='category') 25 Definition ???
{ } FacetedSearch ? s = MySiteSearch('python', category='blog') for hit
in s: print(h.meta.score, h.title) cat_facet = s.facets['category'] for name, count in cat_facet: mask = '%s: %d' if name == cat_facet.selected: mask = '<b>%s: %d</b>' print(mask % (name, count)) 26 Usage ????
{ } Django integration ? Model -> DocType signal handlers
to update management command to sync FacetedSearch -> Form view + template pattern 27
{ } Thank you! @honzakral
{ } This work is licensed under the Creative Commons
Attribution-NoDerivatives 4.0 International License. To view a copy of this license, visit: http://creativecommons.org/licenses/by-nd/4.0/ or send a letter to: Creative Commons PO Box 1866 Mountain View, CA 94042 USA CC-BY-ND 4.0 29