Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Faceting analyzed fields with some sprinkles of...

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Boaz Leskes Boaz Leskes
June 04, 2013
Technology
100
0

Faceting analyzed fields with some sprinkles of probability theory

Talk given at Berlin buzzwords 2013

Avatar for Boaz Leskes

Boaz Leskes

June 04, 2013

More Decks by Boaz Leskes

See All by Boaz Leskes
Every Shard Deserves a Home - Shard Allocation in Elasticsearch
Avatar for Boaz Leskes bleskes
0
360
Life of a Document in Elasticsearch
Avatar for Boaz Leskes bleskes
3
3.4k
Resiliency in Elasticsearch & Lucene
Avatar for Boaz Leskes bleskes
0
560
Resiliency in Elasticsearch & Lucene
Avatar for Boaz Leskes bleskes
0
290
Designing Concurrent Distributed Sequence Numbers for Elasticsearch
Avatar for Boaz Leskes bleskes
2
760
Not all Nodes are Created Equal - Scaling Elasticsearch
Avatar for Boaz Leskes bleskes
1
400
Not all Nodes are Created Equal - Scaling Elasticsearch
Avatar for Boaz Leskes bleskes
6
750
The ELK Stack: For Real-Time Enlightenment
Avatar for Boaz Leskes bleskes
1
1.8k
Staying Ahead of Users And Time - two use cases of scaling data with Elasticsearch
Avatar for Boaz Leskes bleskes
1
360

Other Decks in Technology

See All in Technology
Tofu-Controller lightning talk @ Containerday 2026
Avatar for Giovanni Toraldo gionn
0
140
PHPカンファレンス関西の招待状 / Invitation to PHP Conference Kansai
Avatar for Y-KANOH ykanoh
0
220
JAWSUG TOHOKU 青森支部 「LoGoチャット」のAWS移行で 乗り越えたこと
Avatar for たけだ taketakekaho
0
360
Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces
Avatar for Preferred Networks pfn
PRO
0
150
What Is JetBrains Air
Avatar for Daisuke Masuda x5gtrn
PRO
0
130
Let's Change Our World—Again
Avatar for Koji SHIMADA snoozer05
PRO
2
200
NDIでApple Vision Proにリアルタイム映像送信
Avatar for jyouryuusui jyouryuusui
0
140
React改造してみた
Avatar for Mooncake Sugar tsukina_7mochi
0
140
AI コーディングと人間の認知スループット / CTO Night & Day 2026 Morning Session / AI Coding and Human Cognitive Throughput
Avatar for Tamirlan 893 Torgayev prog893
5
1.4k
フロントエンドカンファレンス関西2026 - 速すぎるUIに、待ち時間を
Avatar for teamLab teamlab
PRO
0
220
Kiro Crewで「みのるん式」仕事術やってみた
Avatar for みのるん minorun365
PRO
3
960
S3 Filesで挑む！ 八戸せんべい汁の食べ頃問題
Avatar for usanchuu usanchuu
0
130

Featured

See All Featured
JAMstack: Web Apps at Ludicrous Speed - All Things Open 2022
Avatar for David Neal reverentgeek
1
640
Data-driven link building: lessons from a $708K investment (BrightonSEO talk)
Avatar for Szymon Słowik szymonslowik
1
1.4k
Ruling the World: When Life Gets Gamed
Avatar for Sebastian Deterding codingconduct
0
410
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
Avatar for Ines Montani inesmontani
PRO
0
760
Build your cross-platform service in a week with App Engine
Avatar for Jose L jlugia
234
19k
Faster Mobile Websites
Avatar for Dean Hume deanohume
310
32k
A Modern Web Designer's Workflow
Avatar for Chris Coyier chriscoyier
699
190k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
Avatar for Rebecca Miller-Webster rmw
35
3.9k
Kristin Tynski - Automating Marketing Tasks With AI
Avatar for Tech SEO Connect techseoconnect
PRO
0
550
What’s in a name? Adding method to the madness
Avatar for The Alliance productmarketing
PRO
24
4.2k
How to Talk to Developers About Accessibility
Avatar for Jason CranfordTeague jct
2
590
New Earth Scene 8
Avatar for Sydney Stone popppiees
4
2.7k

Transcript

  1. Faceting analyzed fields with some sprinkles of probability theory conjures

    trending topic analysis and other interesting insights Boaz Leskes Elasticsearch @bleskes work done for Buzzcapture

  2. Trending?

  3. © Buzzcapture

  4. © Buzzcapture

  5. reference reference topic © Buzzcapture

  6. topic reference ≠

  7. topic reference P(w|T) = kDt |w 2 Dt k kDt

    k
  8. None

  9. topic reference P(w|T) = kDt |w 2 Dt k kDt

    k

  10. P(w|T) = kDt |w 2 Dt k kDt k

  11. brown dog fox quick 2 5 10 12 5 6

    12 13 2 5 6 10 12 13 brown dog fox quick

  12. In our index. • Terms = 12GB • “Arrows” =

    41GB

  13. { tweet: { type: "string", analyzer: "whitespace" fielddata: { filter:

    { regex: "^#.*", frequency: { min: 10 } } } } } Drop terms which occur too little

  14. Drop docs with too many terms

  15. reference reference topic © Buzzcapture

  16. iculture 10,122 floor 8,998 cover 6,874 toy 4,402 ground 3,841

    4.0 7,878 4.1 4,292 rtacties 4,078 jelly 2,905 bean 2,857