permission is strictly prohibited 3 Kibana • Democratize your data • Create "Visualizations" and "Dashboards" • Slide and dice log data using Elasticsearch Aggregations • Plugins to extend functionality • Timelion • Graph UI Plugin • Marvel • Sense • Community…
permission is strictly prohibited 4 Timelion • Kibana 4.2 plugin • "Do more with time series data" • Easy query language based on "chaining" functions together • Simple functions such as add and subtract as well as moving averages, cumulative sums and derivatives • Custom styling • Connects to outside data sources as well as Elasticsearch in the same query • Worldbank's Data API • More to be added - community too!
permission is strictly prohibited 6 Graph Plugin • Why Graph? • Find connections between terms • Suggestion engines • Related terms • Connect user queries with products & departments • Domain specific research • Why Elasticsearch graphs are different • Generated "on-the-fly" • Avoids "super nodes" • Use sampling and diversity settings to generate personalized (and useful) graphs
permission is strictly prohibited 7 Graph Terms • Graph: The actual data structure returned by your query • Vertex: Represent terms in your indices • As your data set new cities will become part of your graph • Edge: The connection between vertices • These connections are generated on-the-fly and can change as your data changes • Significant Terms Aggregation • Graph API relies on this aggregation to generate edges between "strongly connected" vertices. • Avoids super nodes
permission is strictly prohibited 10 Faster & Better Results From Sampling • "Sampler Aggregation" allows results to be calculated over a particular subset - in this case users from around the world, not just the USA { "vertex_fields": ["artists.raw"], "options": { "sampleSize": 1000, "diversity": { "field": "country.raw" "maxDocsPerValue": 200 } } "query": { "terms": { "artists.raw": [ "faith no more", "peeping tom", "the beatles" ] } } }
permission is strictly prohibited 11 FAQ • When? • Scheduled for 2.2 release - soon • Will Kibana have a UI plugin? • Yes, still working on something • How is this different from graph DB's? • Relevance: Graph DBs have no relevance ranking algos so always get tangled up in the super-popular nodes in networks (Twitter=Bieber, Wikipedia=US, Music=coldplay...). In search-world we are very familiar with Zipf's law and super-popular terms like "the" so know how to apply ranking. • Aggregate views: We don't need to return millions of individual docs as nodes. We can use a summary representation of the connection between 2 bank accounts as a single link and use aggs to summarize potentially millions of transactions as properties of that single link at high speed using date histogram, max, sum aggs etc