Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Faceting analyzed fields with some sprinkles of...

Avatar for Boaz Leskes Boaz Leskes
June 04, 2013
Technology
0
63

Faceting analyzed fields with some sprinkles of probability theory

Talk given at Berlin buzzwords 2013

Avatar for Boaz Leskes

Boaz Leskes

June 04, 2013
Tweet

More Decks by Boaz Leskes

See All by Boaz Leskes
Every Shard Deserves a Home - Shard Allocation in Elasticsearch
Avatar for Boaz Leskes bleskes
0
320
Life of a Document in Elasticsearch
Avatar for Boaz Leskes bleskes
3
3.3k
Resiliency in Elasticsearch & Lucene
Avatar for Boaz Leskes bleskes
0
510
Resiliency in Elasticsearch & Lucene
Avatar for Boaz Leskes bleskes
0
230
Designing Concurrent Distributed Sequence Numbers for Elasticsearch
Avatar for Boaz Leskes bleskes
2
720
Not all Nodes are Created Equal - Scaling Elasticsearch
Avatar for Boaz Leskes bleskes
1
370
Not all Nodes are Created Equal - Scaling Elasticsearch
Avatar for Boaz Leskes bleskes
6
680
The ELK Stack: For Real-Time Enlightenment
Avatar for Boaz Leskes bleskes
1
1.7k
Staying Ahead of Users And Time - two use cases of scaling data with Elasticsearch
Avatar for Boaz Leskes bleskes
1
310

Other Decks in Technology

See All in Technology
複数サービスを支える マルチテナント型 Batch MLプラットフォーム
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
980
Unlocking the Power of AI Agents with LINE Bot MCP Server
Avatar for LINE Developers Thailand linedevth
0
120
COVESA VSSによる車両データモデルの標準化とAWS IoT FleetWiseの活用
Avatar for Yuto Osawa osawa
1
400
AIエージェント開発用SDKとローカルLLMをLINE Botと組み合わせてみた / LINEを使ったLT大会 #14
Avatar for you（@youtoy） you
PRO
0
130
テストを軸にした生き残り術
Avatar for KNOWLEDGE WORK / 株式会社ナレッジワーク kworkdev
PRO
0
220
プラットフォーム転換期におけるGitHub Copilot活用〜Coding agentがそれを加速するか〜 / Leveraging GitHub Copilot During Platform Transition Periods
Avatar for AEON aeonpeople
1
240
バイブスに「型」を！Kent Beckに学ぶ、AI時代のテスト駆動開発
Avatar for amixedcolor amixedcolor
3
590
[ JAWS-UG 東京 CommunityBuilders Night #2 ]SlackとAmazon Q Developerで 運用効率化を模索する
Avatar for sh_fk2 sh_fk2
3
460
Automating Web Accessibility Testing with AI Agents
Avatar for South maminami373
0
1.3k
Snowflake Intelligenceにはこうやって立ち向かう！クラシルが考えるAI Readyなデータ基盤と活用のためのDataOps
Avatar for harry gappy50
0
280
AI時代を生き抜くエンジニアキャリアの築き方 (AI-Native 時代、エンジニアという道は 「最大の挑戦の場」となる) / Building an Engineering Career to Thrive in the Age of AI (In the AI-Native Era, the Path of Engineering Becomes the Ultimate Arena of Challenge)
Avatar for JaeSoon Jeong jeongjaesoon
0
260
「Linux」という言葉が指すもの
Avatar for Satoru Takeuchi sat
PRO
4
140

Featured

See All Featured
Fantastic passwords and where to find them - at NoRuKo
Avatar for Phil Nash philnash
52
3.4k
KATA
Avatar for Megan Lloyd mclloyd
32
14k
VelocityConf: Rendering Performance Case Studies
Avatar for Addy Osmani addyosmani
332
24k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
36
2.5k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
8
930
Building Applications with DynamoDB
Avatar for Matt Wood mza
96
6.6k
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
77
6k
Optimizing for Happiness
Avatar for Tom Preston-Werner mojombo
379
70k
[RailsConf 2023 Opening Keynote] The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
30
9.7k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
37
2.9k
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.4k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
Avatar for Michelle Schulp Hunt marktimemedia
31
2.5k

Transcript

  1. Faceting analyzed fields with some sprinkles of probability theory conjures

    trending topic analysis and other interesting insights Boaz Leskes Elasticsearch @bleskes work done for Buzzcapture

  2. Trending?

  3. © Buzzcapture

  4. © Buzzcapture

  5. reference reference topic © Buzzcapture

  6. topic reference ≠

  7. topic reference P(w|T) = kDt |w 2 Dt k kDt

    k
  8. None

  9. topic reference P(w|T) = kDt |w 2 Dt k kDt

    k

  10. P(w|T) = kDt |w 2 Dt k kDt k

  11. brown dog fox quick 2 5 10 12 5 6

    12 13 2 5 6 10 12 13 brown dog fox quick

  12. In our index. • Terms = 12GB • “Arrows” =

    41GB

  13. { tweet: { type: "string", analyzer: "whitespace" fielddata: { filter:

    { regex: "^#.*", frequency: { min: 10 } } } } } Drop terms which occur too little

  14. Drop docs with too many terms

  15. reference reference topic © Buzzcapture

  16. iculture 10,122 floor 8,998 cover 6,874 toy 4,402 ground 3,841

    4.0 7,878 4.1 4,292 rtacties 4,078 jelly 2,905 bean 2,857