Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python DSL
Search
Elasticsearch Inc
March 11, 2015
Technology
2
940
Python DSL
Slides for Honza's talk at Elastic{on}
Elasticsearch Inc
March 11, 2015
Tweet
Share
More Decks by Elasticsearch Inc
See All by Elasticsearch Inc
OSCON: Scaling a distributed engineering team from 50-250
elasticsearch
13
1.5k
Stuff a Search Engine Can Do
elasticsearch
17
1.7k
Using Elastic to monitor anything
elasticsearch
3
1.5k
Log all the things!
elasticsearch
4
1.2k
Why Elastic? @ 50th Vinitaly 2016
elasticsearch
5
2k
What's New In Elasticland?
elasticsearch
3
960
Kibana, Timelion, Graph Meetup
elasticsearch
3
800
Elastic for Time Series Data and Predictive Analytics
elasticsearch
4
3.1k
Elastic 2.0
elasticsearch
1
750
Other Decks in Technology
See All in Technology
GoでもGUIアプリを作りたい!
kworkdev
PRO
0
160
Data Hubグループ 紹介資料
sansan33
PRO
0
2.2k
新規事業におけるGORM+SQLx併用アーキテクチャ
hacomono
PRO
0
420
ガバメントクラウドの概要と自治体事例(名古屋市)
techniczna
3
240
『バイトル』CTOが語る! AIネイティブ世代と切り拓くモノづくり組織
dip_tech
PRO
1
130
ニッポンの人に知ってもらいたいGISスポット
sakaik
0
170
AIエージェント入門 〜基礎からMCP・A2Aまで〜
shukob
0
110
AI-Readyを目指した非構造化データのメダリオンアーキテクチャ
r_miura
1
180
Node.js 2025: What's new and what's next
ruyadorno
0
620
[VPoE Global Summit] サービスレベル目標による信頼性への投資最適化
satos
0
130
AIツールでどこまでデザインを忠実に実装できるのか
oikon48
6
3.5k
ソフトウェアエンジニアの生成AI活用と、これから
lycorptech_jp
PRO
0
560
Featured
See All Featured
The Pragmatic Product Professional
lauravandoore
36
6.9k
Practical Orchestrator
shlominoach
190
11k
Building Better People: How to give real-time feedback that sticks.
wjessup
369
20k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.2k
A Tale of Four Properties
chriscoyier
161
23k
[RailsConf 2023] Rails as a piece of cake
palkan
57
5.9k
Fireside Chat
paigeccino
40
3.7k
Agile that works and the tools we love
rasmusluckow
331
21k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
How GitHub (no longer) Works
holman
315
140k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
GitHub's CSS Performance
jonrohan
1032
470k
Transcript
Python DSL Honza Král @honzakral
{ } DSL 2
{ } DSL ? Don't you mean ORM? 3
{ } Current State { "query": { "filtered": { "query":
{ "function_score": { "query": { "bool": { "must": [ {"multi_match": { "fields": ["title^10", "body"], "query": "php"}}, {"has_child": { "child_type": "answer", "query": {"match": {"body": "python"}}}} ], "must_not": [ {"multi_match": { "fields": ["title", "body"], "query": "python"}} ] } }, "field_value_factor": {"field": "rating"} } }, "filter": {"range": {"creation_date": {"from": "2010-01-01"}}} }}, 4 "highlight": { "fields": { "title": {"fragment_size" : 50}, "body": {"fragment_size" : 50} } }, "aggs": { "tags": { "terms": {"field": "tags"}, "aggs": { "comment_avg": { "avg": {"field": "comment_count"} } } }, "frequency": { "date_histogram": { "field": "creation_date", "interval": "month" } } } } JSON DSL
{ } Now add a filter to it! 5
{ } Search Object s = Search(doc_type='question') 6
{ } Simple Query s = s.query('multi_match', fields=['title^10', 'body'], query='php')
7
{ } Compound Query s = s.query('has_child', child_type='answer', query=Q('match', body='python'))
8
{ } Q shortcut {"has_child": { "child_type": "answer', "query": {"match":
{"body": "python"}}}} Q({'has_child': { 'child_type': 'answer', 'query': {'match': {'body': 'python'}}}}) Q('has_child', child_type='answer', query=Q('match', body='python')) HasChild(child_type='answer', query=Match(body='python')) 9
{ } Query expressions Q(...) & Q(...) == Bool(must=[Q(...), Q(...)])
Q(...) | Q(...) == Bool(should=[Q(...), Q(...)]) ~Q(...) == Bool(must_not=[Q(..)]) 10
{ } Filter s = s.filter('range', creation_date={'from': date(2010, 1, 1)})
11
{ } Exclude s = s.query(~Q('multi_match', fields=['title^10', 'body'], query='python')) 12
{ } Manual query s.query = Q('function_score', query=s.query, field_value_factor={'field': 'rating'})
13
{ } Aggregations s.aggs.bucket('tags', 'terms', field='tags')\ .metric('comment_avg', 'avg', field='comment_count') s.aggs.bucket('frequency',
'date_histogram', field='creation_date', interval='month') 14
{ } Highlight ... s = s.highlight('title', 'body', fragment_size=50) 15
{ } Migration path s = Search.from_dict(my_glorious_query) s = s.filter('term',
tag='published') my_glorious_query = s.to_dict() 16 query at a time
{ } Response response = s.execute() for hit in response:
print(hit.meta.score, hit.title) for tag in response.aggregations.tags.buckets: print(tag.key, tag.avg_comments.value) 17 No more brackets!
{ } Persistence From Mapping to Model-like DocTypes 18
{ } Mapping DSL m = Mapping('article') m.field('published_from', Date()) m.field('title',
String(fields={'raw': String(index='not_analyzed')})) m.field('comments', Nested()) m['comments'].property('author', String()) m.save('index-name') m.update_from_es('index-name') 19
{ } DocType class Article(DocType): title = String() created_date =
Date() comments = Nested(properties={'author': String()}) class Meta: index = 'blog' def save(self, **kwargs): self.created_date = now() super().save(**kwargs) Article.init() Article.search()... Search(doc_type=Article) 20
{ } Configuration 21
{ } Connections connections.configure( default={'hosts': ['localhost'], 'sniff_on_start': True}, logging={ 'hosts':
['log1:9200', 'log2:9200'], 'timeout': 30, 'sniff_timeout': 120}) Search(using='logging') es = connections.get_connection() es.indices.delete(index='_all') 22
{ } Future 23
{ } More DSLs! Index Analyzers Settings ... 24
{ } FacetedSearch ? class MySiteSearch(FacetedSearch): doc_type = [Article, Comment]
fields = ['title', 'body'] published = DateHistogram( interval='week', field='published_date') category = Term(field='category') 25 Definition ???
{ } FacetedSearch ? s = MySiteSearch('python', category='blog') for hit
in s: print(h.meta.score, h.title) cat_facet = s.facets['category'] for name, count in cat_facet: mask = '%s: %d' if name == cat_facet.selected: mask = '<b>%s: %d</b>' print(mask % (name, count)) 26 Usage ????
{ } Django integration ? Model -> DocType signal handlers
to update management command to sync FacetedSearch -> Form view + template pattern 27
{ } Thank you! @honzakral
{ } This work is licensed under the Creative Commons
Attribution-NoDerivatives 4.0 International License. To view a copy of this license, visit: http://creativecommons.org/licenses/by-nd/4.0/ or send a letter to: Creative Commons PO Box 1866 Mountain View, CA 94042 USA CC-BY-ND 4.0 29