Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Elasticsearch

Introduction to Elasticsearch

New England Search Technologies (NEST) Group Meetup

Igor Motov

March 25, 2015
Tweet

More Decks by Igor Motov

Other Decks in Programming

Transcript

  1. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 2 Elastic • Founded in 2012 • By the people behind the Elasticsearch and Apache Lucene • Renamed to Elastic in 2015 • Headquarters: Amsterdam and Mountain View, CA • We provide • Training (public & onsite) • Development support • Production support • Hosted elasticsearch
  2. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 3 Enough about me, let’s talk about you for a bit!
  3. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 4 Have you used Elasticsearch recently?
  4. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 5 Have you used elasticsearch recently?
  5. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 6 Some Elasticsearch users
  6. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 7 What’s Elasticsearch • Real time search and analytics engine • JSON-oriented, Apache Lucene-based • Automatic Schema Detection • With full control of schema when needed • Distributed • Scales Up+Out, Highly Available • Multi-tenancy • Dynamically create/delete indices • API centric • Most functionality is exposed through an API
  7. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 8 Elasticsearch Documents • Identified by index/type/id • JSON Document: { "author": "imotov", "title": "Introduction to Elasticsearch", "post_date": "2015-03-25", "size": 30 }
  8. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 10 Elasticsearch distribution . ├── LICENSE.txt ├── NOTICE.txt ├── README.textile ├── bin │ ├── elasticsearch │ ├── elasticsearch.in.sh │ └── plugin ├── config │ ├── elasticsearch.yml │ └── logging.yml ├── data │ └── elasticsearch ├── lib │ ├── elasticsearch-x.y.z.jar │ ├── ... │ └── └── logs ├── elasticsearch.log └── elasticsearch_index_search_slowlog.log
  9. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 12 Communicating with Elasticsearch • REST Protocol • Binary Protocol
  10. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 13 Communicating with Elasticsearch • REST API • JavaScript • Perl • PHP • Python • Ruby • .NET • Binary Protocol • Java • Groovy
  11. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 14 It’s not just search
  12. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 17 Aggregations • Group by • values • ranges • histogram • time intervals (years/months/days…) • IP ranges • geo location • …
  13. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 18 • Calculate • counts • max • min • avg • sum • standard deviation (with bounds) • variance • sum of squares • percentile rank
  14. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 22 Let’s talk about scale and high availability.
  15. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 23 Scale and High Availability node1 0 1 2 3 4 hash(id)%5 M
  16. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 24 High Availability node1 0 1 2 3 4 node2 M
  17. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 25 High Availability node1 0 1 2 3 4 node2 0 1 2 3 4 M
  18. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 26 High Availability node1 0 1 2 3 4 hash(id%5) node1 0 1 2 3 4 M
  19. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 27 Scale node1 0 2 4 node2 1 2 3 node3 0 1 3 4 M
  20. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 28 Scale node1 0 2 4 node2 1 2 3 node3 0 1 3 4 M
  21. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 29 Scale node1 0 2 node2 1 2 3 node3 1 3 4 0 4 M
  22. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 30 Scale node1 0 2 node2 1 2 3 node3 1 3 4 0 4 M
  23. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 31 Master is gone node2 1 2 3 node3 3 4 0 4
  24. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 32 Master is elected node2 1 2 3 node3 3 4 0 4 M
  25. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 33 Primary are promoted node2 1 2 3 node3 3 4 0 4 M
  26. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 34 High Availability node2 1 2 3 node3 3 4 0 4 M
  27. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited node4 35 High Availability is restored node2 1 2 3 node3 3 4 0 4 M 1 2 0
  28. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 36 Elasticsearch ecosystem
  29. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 37 Elasticsearch Ecosystem
  30. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 38 Elasticsearch Ecosystem logstash
  31. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 39 Elasticsearch Ecosystem logstash
  32. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited Shield 40 Elasticsearch Ecosystem logstash
  33. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 41 Elasticsearch Resources
  34. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 42 Elasticsearch - The Definitive Guide http://www.elastic.co/guide/en/elasticsearch/guide/current/index.html
  35. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 44 http://www.meetup.com/Elasticsearch-Boston/ Boston Elasticsearch Meetup
  36. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written

    permission is strictly prohibited 46 Elasticsearch Training http://purchases.elastic.co/