Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python DSL
Search
Elasticsearch Inc
March 11, 2015
Technology
2
930
Python DSL
Slides for Honza's talk at Elastic{on}
Elasticsearch Inc
March 11, 2015
Tweet
Share
More Decks by Elasticsearch Inc
See All by Elasticsearch Inc
OSCON: Scaling a distributed engineering team from 50-250
elasticsearch
13
1.5k
Stuff a Search Engine Can Do
elasticsearch
17
1.7k
Using Elastic to monitor anything
elasticsearch
3
1.5k
Log all the things!
elasticsearch
4
1.2k
Why Elastic? @ 50th Vinitaly 2016
elasticsearch
5
2k
What's New In Elasticland?
elasticsearch
3
940
Kibana, Timelion, Graph Meetup
elasticsearch
3
790
Elastic for Time Series Data and Predictive Analytics
elasticsearch
4
3.1k
Elastic 2.0
elasticsearch
1
750
Other Decks in Technology
See All in Technology
AIとTDDによるNext.js「隙間ツール」開発の実践
makotot
6
750
つくって納得、つかって実感! 大規模言語モデルことはじめ
recruitengineers
PRO
26
8.4k
衝突して強くなる! BLUE GIANTと アジャイルチームの共通点とは ― いきいきと活気に満ちたグルーヴあるチームを作るコツ ― / BLUE GIANT and Agile Teams
naitosatoshi
0
130
実践AIガバナンス
asei
3
160
人と組織に偏重したEMへのアンチテーゼ──なぜ、EMに設計力が必要なのか/An antithesis to the overemphasis of people and organizations in EM
dskst
6
660
Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders
kzykmyzw
0
370
Android Studio の 新しいAI機能を試してみよう / Try out the new AI features in Android Studio
yanzm
0
290
絶対に失敗できないキャンペーンページの高速かつ安全な開発、WINTICKET × microCMS の開発事例
microcms
0
180
どこで動かすか、誰が動かすか 〜 kintoneのインフラ基盤刷新と運用体制のシフト 〜
ueokande
0
200
Webアクセシビリティ入門
recruitengineers
PRO
2
900
AIエージェント就活入門 - MCPが履歴書になる未来
eltociear
0
630
攻撃と防御で実践するプロダクトセキュリティ演習~導入パート~
recruitengineers
PRO
3
950
Featured
See All Featured
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
185
54k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
Music & Morning Musume
bryan
46
6.8k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
Into the Great Unknown - MozCon
thekraken
40
2k
Fireside Chat
paigeccino
39
3.6k
Java REST API Framework Comparison - PWX 2021
mraible
33
8.8k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
30
9.6k
GitHub's CSS Performance
jonrohan
1031
460k
Making Projects Easy
brettharned
117
6.3k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.1k
Transcript
Python DSL Honza Král @honzakral
{ } DSL 2
{ } DSL ? Don't you mean ORM? 3
{ } Current State { "query": { "filtered": { "query":
{ "function_score": { "query": { "bool": { "must": [ {"multi_match": { "fields": ["title^10", "body"], "query": "php"}}, {"has_child": { "child_type": "answer", "query": {"match": {"body": "python"}}}} ], "must_not": [ {"multi_match": { "fields": ["title", "body"], "query": "python"}} ] } }, "field_value_factor": {"field": "rating"} } }, "filter": {"range": {"creation_date": {"from": "2010-01-01"}}} }}, 4 "highlight": { "fields": { "title": {"fragment_size" : 50}, "body": {"fragment_size" : 50} } }, "aggs": { "tags": { "terms": {"field": "tags"}, "aggs": { "comment_avg": { "avg": {"field": "comment_count"} } } }, "frequency": { "date_histogram": { "field": "creation_date", "interval": "month" } } } } JSON DSL
{ } Now add a filter to it! 5
{ } Search Object s = Search(doc_type='question') 6
{ } Simple Query s = s.query('multi_match', fields=['title^10', 'body'], query='php')
7
{ } Compound Query s = s.query('has_child', child_type='answer', query=Q('match', body='python'))
8
{ } Q shortcut {"has_child": { "child_type": "answer', "query": {"match":
{"body": "python"}}}} Q({'has_child': { 'child_type': 'answer', 'query': {'match': {'body': 'python'}}}}) Q('has_child', child_type='answer', query=Q('match', body='python')) HasChild(child_type='answer', query=Match(body='python')) 9
{ } Query expressions Q(...) & Q(...) == Bool(must=[Q(...), Q(...)])
Q(...) | Q(...) == Bool(should=[Q(...), Q(...)]) ~Q(...) == Bool(must_not=[Q(..)]) 10
{ } Filter s = s.filter('range', creation_date={'from': date(2010, 1, 1)})
11
{ } Exclude s = s.query(~Q('multi_match', fields=['title^10', 'body'], query='python')) 12
{ } Manual query s.query = Q('function_score', query=s.query, field_value_factor={'field': 'rating'})
13
{ } Aggregations s.aggs.bucket('tags', 'terms', field='tags')\ .metric('comment_avg', 'avg', field='comment_count') s.aggs.bucket('frequency',
'date_histogram', field='creation_date', interval='month') 14
{ } Highlight ... s = s.highlight('title', 'body', fragment_size=50) 15
{ } Migration path s = Search.from_dict(my_glorious_query) s = s.filter('term',
tag='published') my_glorious_query = s.to_dict() 16 query at a time
{ } Response response = s.execute() for hit in response:
print(hit.meta.score, hit.title) for tag in response.aggregations.tags.buckets: print(tag.key, tag.avg_comments.value) 17 No more brackets!
{ } Persistence From Mapping to Model-like DocTypes 18
{ } Mapping DSL m = Mapping('article') m.field('published_from', Date()) m.field('title',
String(fields={'raw': String(index='not_analyzed')})) m.field('comments', Nested()) m['comments'].property('author', String()) m.save('index-name') m.update_from_es('index-name') 19
{ } DocType class Article(DocType): title = String() created_date =
Date() comments = Nested(properties={'author': String()}) class Meta: index = 'blog' def save(self, **kwargs): self.created_date = now() super().save(**kwargs) Article.init() Article.search()... Search(doc_type=Article) 20
{ } Configuration 21
{ } Connections connections.configure( default={'hosts': ['localhost'], 'sniff_on_start': True}, logging={ 'hosts':
['log1:9200', 'log2:9200'], 'timeout': 30, 'sniff_timeout': 120}) Search(using='logging') es = connections.get_connection() es.indices.delete(index='_all') 22
{ } Future 23
{ } More DSLs! Index Analyzers Settings ... 24
{ } FacetedSearch ? class MySiteSearch(FacetedSearch): doc_type = [Article, Comment]
fields = ['title', 'body'] published = DateHistogram( interval='week', field='published_date') category = Term(field='category') 25 Definition ???
{ } FacetedSearch ? s = MySiteSearch('python', category='blog') for hit
in s: print(h.meta.score, h.title) cat_facet = s.facets['category'] for name, count in cat_facet: mask = '%s: %d' if name == cat_facet.selected: mask = '<b>%s: %d</b>' print(mask % (name, count)) 26 Usage ????
{ } Django integration ? Model -> DocType signal handlers
to update management command to sync FacetedSearch -> Form view + template pattern 27
{ } Thank you! @honzakral
{ } This work is licensed under the Creative Commons
Attribution-NoDerivatives 4.0 International License. To view a copy of this license, visit: http://creativecommons.org/licenses/by-nd/4.0/ or send a letter to: Creative Commons PO Box 1866 Mountain View, CA 94042 USA CC-BY-ND 4.0 29