Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Faceting analyzed fields with some sprinkles of...

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Boaz Leskes Boaz Leskes
June 04, 2013
Technology
0
70

Faceting analyzed fields with some sprinkles of probability theory

Talk given at Berlin buzzwords 2013

Avatar for Boaz Leskes

Boaz Leskes

June 04, 2013
Tweet

More Decks by Boaz Leskes

See All by Boaz Leskes
Every Shard Deserves a Home - Shard Allocation in Elasticsearch
Avatar for Boaz Leskes bleskes
0
330
Life of a Document in Elasticsearch
Avatar for Boaz Leskes bleskes
3
3.3k
Resiliency in Elasticsearch & Lucene
Avatar for Boaz Leskes bleskes
0
530
Resiliency in Elasticsearch & Lucene
Avatar for Boaz Leskes bleskes
0
240
Designing Concurrent Distributed Sequence Numbers for Elasticsearch
Avatar for Boaz Leskes bleskes
2
730
Not all Nodes are Created Equal - Scaling Elasticsearch
Avatar for Boaz Leskes bleskes
1
370
Not all Nodes are Created Equal - Scaling Elasticsearch
Avatar for Boaz Leskes bleskes
6
690
The ELK Stack: For Real-Time Enlightenment
Avatar for Boaz Leskes bleskes
1
1.7k
Staying Ahead of Users And Time - two use cases of scaling data with Elasticsearch
Avatar for Boaz Leskes bleskes
1
330

Other Decks in Technology

See All in Technology
AI時代にあわせたQA組織戦略
Avatar for Masami Yajiri masamiyajiri
6
2.7k
BPaaSオペレーション・kubell社内 n8n活用による効率化検証事例紹介
Avatar for kubell kubell_hr
0
280
Vivre en Bitcoin : le tutoriel que votre banquier ne veut pas que vous voyiez
Avatar for Renaud Lifchitz rlifchitz
0
370
Claude in Chromeで始める自律的フロントエンド開発
Avatar for morimorikochan diggymo
1
270
それぞれのペースでやっていく Bet AI / Bet AI at Your Own Pace
Avatar for Yuya Takeyama yuyatakeyama
1
610
SOC2は、取った瞬間よりその後が面白い
Avatar for hima 3flower
1
250
SMTP完全に理解した ✉️
Avatar for yamatai12 yamatai1212
0
110
「全社導入」は結果。1人の熱狂が組織に伝播したmikanのn8n活用
Avatar for Sota Mikami sota_mikami
0
520
新規事業における「一部だけどコア」な
AI精度改善の優先順位づけ
Avatar for Higuchi kokoro zerebom
0
330
VRTと真面目に向き合う
Avatar for Yuya Hirayama hiragram
1
490
新規事業 toitta におけるAI 機能評価の話 / AI Feature Evaluation in toitta
Avatar for pokutuna pokutuna
0
280
re:Inventで見つけた「運用を捨てる」技術。
Avatar for shinji ezaki ezaki
1
150

Featured

See All Featured
Building Experiences: Design Systems, User Experience, and Full Site Editing
Avatar for Michelle Schulp Hunt marktimemedia
0
400
Leadership Guide Workshop - DevTernity 2021
Avatar for David Neal reverentgeek
1
190
A Guide to Academic Writing Using Generative AI - A Workshop
Avatar for Kenji Saito ks91
PRO
0
180
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
Avatar for soudai sone soudai
PRO
61
49k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
51k
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
63
The Impact of AI in SEO - AI Overviews June 2024 Edition
Avatar for Aleyda Solis aleyda
5
710
Sharpening the Axe: The Primacy of Toolmaking
Avatar for Bryan Cantrill bcantrill
46
2.7k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
Avatar for Joe Casabona jcasabona
34
2.6k
AI in Enterprises - Java and Open Source to the Rescue
Avatar for ivargrimstad ivargrimstad
0
1.1k
Navigating Weather and Climate Data
Avatar for Ryan Abernathey rabernat
0
77
Site-Speed That Sticks
Avatar for Harry Roberts csswizardry
13
1k

Transcript

  1. Faceting analyzed fields with some sprinkles of probability theory conjures

    trending topic analysis and other interesting insights Boaz Leskes Elasticsearch @bleskes work done for Buzzcapture

  2. Trending?

  3. © Buzzcapture

  4. © Buzzcapture

  5. reference reference topic © Buzzcapture

  6. topic reference ≠

  7. topic reference P(w|T) = kDt |w 2 Dt k kDt

    k
  8. None

  9. topic reference P(w|T) = kDt |w 2 Dt k kDt

    k

  10. P(w|T) = kDt |w 2 Dt k kDt k

  11. brown dog fox quick 2 5 10 12 5 6

    12 13 2 5 6 10 12 13 brown dog fox quick

  12. In our index. • Terms = 12GB • “Arrows” =

    41GB

  13. { tweet: { type: "string", analyzer: "whitespace" fielddata: { filter:

    { regex: "^#.*", frequency: { min: 10 } } } } } Drop terms which occur too little

  14. Drop docs with too many terms

  15. reference reference topic © Buzzcapture

  16. iculture 10,122 floor 8,998 cover 6,874 toy 4,402 ground 3,841

    4.0 7,878 4.1 4,292 rtacties 4,078 jelly 2,905 bean 2,857