Share
Embed
Transcript
113 slides
-
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Jun Ohtani @johtani 2014/Sep. Elasticsearch & Kibana -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ΞδΣϯμ • Elasticsearchͷհ • ElasticsearchͷσϞ • KibanaͷσϞ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited about • Me, Jun Ohtani / Technical Adovocate lucene-gosenίϛολʔ ElasticSearch Serverຊޠ൛ͷ༁ elasticsearch-extended-analysisͷ։ൃ http://blog.johtani.info • Elasticsearch, founded in 2012 Products: Elasticsearch, Logstash, Kibana, Marvel Professional services: Support & development subscriptions Trainings -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited શจݕࡧͱʁ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited શจݕࡧͱʁ • શจݕࡧʢFull text searchʣͱɺίϯϐϡʔλʹ ͓͍ͯɺෳͷจॻʢϑΝΠϧʣ͔Βಛఆͷจࣈ ྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕࡧʯʮ୯Ұ ϑΝΠϧͷจࣈྻݕࡧʯͱҟͳΓɺʮෳจॻ ʹ·͕ͨͬͯɺจॻʹؚ·ΕΔશจΛରͱͨ͠ ݕࡧʯͱ͍͏ҙຯͰ༻͞ΕΔɻ ʢWikipediaΑΓʣ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ༻ޠ • ΠϯσοΫε ݕࡧΤϯδϯ͕ݕࡧʹ༻͢Δσʔλͷอଘઌ • υΩϡϝϯτʢจॻʣ ݕࡧΤϯδϯʹอଘ͞Εͨσʔλ • ϑΟʔϧυ υΩϡϝϯτʹؚ·ΕΔଐੑ • ΫΤϦ ݕࡧ݅ɺݕࡧࣜ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ༻ޠ • εΩʔϚ υΩϡϝϯτͷߏΛఆٛ͢Δͷ • λʔϜʢTermʣɺτʔΫϯʢTokenʣ ΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ จষΛҰఆͷ๏ଇͰ۠ͬͨ୯ޠ ୯ޠ͚ͩͰͳ͘ɺ୯ޠͷҐஔͳͲؚΉ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited υΩϡϝϯτͷొ 1 2 ΧπΦαβΤͷఋ αβΤϫΧϝͷ࢞ υΩϡϝϯτͷొ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited υΩϡϝϯτͷొ 1 2 ΧπΦαβΤͷఋ αβΤϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ υΩϡϝϯτͷొ ୯ޠʹׂ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited υΩϡϝϯτͷొ 1 2 ΧπΦαβΤͷఋ αβΤϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 υΩϡϝϯτͷొ ୯ޠʹׂ ୯ޠ͔Βidͷྻ͕ Ҿ͚ΔΑ͏ʹ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧ݅ೖྗ ΧπΦɹαβΤ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ୯ޠͷ۠Γํ • ӳޠͷ߹ I am speaking Introduction Elasticsearch. ɹ ɹ • ຊޠͷ߹ ࢲೖElasticsearchʹ͍͍ͭͯͯ͠Δɻ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ୯ޠͷ۠Γํ • ӳޠͷ߹ I am speaking Introduction Elasticsearch. εϖʔε͕ΕͱΘ͔Δ • ຊޠͷ߹ ࢲೖElasticsearchʹ͍͍ͭͯͯ͠Δɻ Ͳ͜Ͱ۠ΕΑ͍ʁ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited N-Gramͱܗଶૉղੳ • సஔΠϯσοΫεͷΩʔͷ࡞Γํ ຊޠ୯ޠͷΕ͕Θ͔Βͳ͍ͷͰɺసஔΠϯσοΫεͷ Ωʔओʹ࣍ͷ̎ͭͷख๏Ͱ࡞ • N-Gram NจࣈͣͭจষΛ۠Δ • ܗଶૉղੳ ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠Δ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ܗଶૉղੳ ! ! • ϝϦοτɿ ҙຯͷ͋Δ୯ޠͷΕ ࢺใΛݩʹՃॲཧ͕ՄೳʢޠװมͳͲʣ • σϝϦοτɿ ৽ޠʢະޠʣʹऑ͍ˠࣙॻϕʔεͷ߹ɺࣙॻʹͳ͍୯ޠݕ ग़ෆೳɻ ΧπΦαβΤͷఋ ΧπΦ ͷ αβΤ ఋ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited N-Gram ! ! • ϝϦοτɿ ະޠʹରԠՄೳ • σϝϦοτɿ ΠϯσοΫεංେԽ ࢺใʹجͮ͘ॲཧ͕ෆՄೳ ΧπΦαβΤͷఋ Χπ πΦ Φ α αβ βΤ Τͷ ͷఋ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Elasticsearchͱʁ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϑϦʔϫʔυݕࡧ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ߜΓࠐΈ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϋΠϥΠτ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ιʔτ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϖʔδϯά -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ूܭ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited αδΣετ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Elasticsearch in 10 seconds • εΩʔϚϑϦʔɺࢄυΩϡϝϯτετΞɺREST & JSON • Φʔϓϯιʔε: Apache License 2.0 • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ • JavaͰ࣮ɻ֦ு༰қ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Πϯετʔϧ͔Βݕࡧ·Ͱ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited μϯϩʔυͱىಈ $ wget https://download.elasticsearch.org/... $ tar -xf elasticsearch-1.3.2.tar.gz $ ./elasticsearch-1.3.2/bin/elasticsearch ... [2014-08-29 13:53:07,356][INFO ][node] [Araña] started ... Also puppet modules and RPM/DEB -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ىಈͷ֬ೝ » curl localhost:9200 { "status" : 200, "name" : "Araña", "version" : { "number" : "1.3.2", "build_hash" : "dee175dbe2f254f3f26992f5d7591939aaefd12f", "build_timestamp" : "2014-08-13T14:29:30Z", "build_snapshot" : false, "lucene_version" : "4.9" }, "tagline" : "You Know, for Search" } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited » curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : "Clinton Gormley", "started" : "2013-02-04", "pages" : 230 }' σʔλొ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited » curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : "2013-02-04", "pages" : 230 }' σʔλߋ৽ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited σʔλআ » curl -X DELETE localhost:9200/books/book/1 σʔλͷऔಘ » curl —X GET localhost:9200/books/book/1 » curl —X GET localhost:9200/books/book/1/_source -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧ » curl -XGET localhost:9200/books/_search?q=elasticsearch { "took" : 2, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.076713204, "hits" : [ { "_index" : “books", "_type" : “book", "_id" : "1", "_score" : 0.076713204, "_source" : { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : “2013-02-04", "pages" : 230 } } ] } } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited » curl -XGET ‘localhost:9200/books/book/_search' -d '{ "query": { "filtered" : { "query" : { "match": { "text" : { "query" : “To Be Or Not To Be", "cutoff_frequency" : 0.01 } } }, "filter" : { "range": { "price": { "gte": 20.0 "lte": 50.0 ... } }' ݕࡧ - Query DSL -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ࢄߏɺεέʔϧ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Basic terms • ΠϯσοΫε σʔλͷཧతͳू߹ɻ RDBͷσʔλϕʔεͷΑ͏ͳͷLogical • ϨϓϦέʔγϣϯ ಡΈࠐΈͷεέʔϥϏϦςΟ্ SPOFͷղফ • γϟʔσΟϯά ෳϚγϯσʔλΛׂ ॻ͖ࠐΈͷεέʔϥϏϦςΟ্ σʔλϑϩʔ੍ޚ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited γϟʔυͱϨϓϦΧ node 1 orders products 1 4 1 2 2 2 curl -‐X PUT localhost:9200/orders -‐d '{ "settings.index.number_of_shards" : 4 "settings.index.number_of_replicas" : 1 }' curl -‐X PUT localhost:9200/products -‐d '{ "settings.index.number_of_shards" : 2 "settings.index.number_of_replicas" : 0 }' -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited γϟʔυͱϨϓϦΧ node 1 orders products 1 4 1 node 2 orders products 2 2 3 4 1 2 3 -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ࣗಈతͳࢄ node 1 orders products 2 1 4 1 node 2 orders products 2 2 node 3 orders products 3 4 1 3 -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Ϋϥελཧ • ୯ҰϚελ Ϋϥελͷঢ়ଶʢϊʔυใɺϚοϐϯάʣΛཧ • ϚϧνΩϟετʹΑΔϊʔυ୳ࡧ ɹ (ϢχΩϟετՄೳ) • ඞཁͳઃఆ Ϋϥελ໊ ࠷খϚελϊʔυ • Tips: 3ͭͷϚελϊʔυΛ༻ҙ͠ɺσʔλΛอ࣋ ͠ͳ͍ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ΫϥελɺϊʔυͷαΠδϯά • σʔλྔͱϦΫΤετʹґଘ υΩϡϝϯτʁϑΟʔϧυʁΫΤϦͷׂ߹ʁ FacetAggregationɺιʔτɺΧελϜείΞͷར༻ʁ ॻ͖ࠐΈͷׂ߹ʁυΩϡϝϯτআ/ߋ৽͢Δʁ ࣌ؒʹґଘͨ͠σʔλʁ • 1ϊʔυɺϨϓϦΧແ͠Ͱςετ γϟʔυαΠζɺJVMώʔϓ༻ྔɺGCසɺγϟʔυ͓Α ͼϊʔυɺγϟʔυ͋ͨΓͷυΩϡϝϯτɺCPU༻ ɺσΟεΫ༻ɺΠϯσοΫεͷύλʔϯ • Tips: 30 GB heap -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Ϋϥελ͓ΑͼΞϓϦͷߏ • ϩʔΧϧσΟεΫ; SAN OK • Tips: ϨΠςϯγͷߴ͍WAN্ͷΫϥελඇਪ ɻCross-zone in EC2 is OK. ! Your app ES Data 1 ES Data N ... ES Master; no data ! Your app ... ES Master; no data ES Master; no data High Speed Network ES Node Client ES Node Client -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ΠϯσοΫεͷొͱݕࡧͷ ෦ಈ࡞ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited υΩϡϝϯτొγʔέϯε • υΩϡϝϯτΛొ doc1 node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited υΩϡϝϯτొγʔέϯε • υΩϡϝϯτͷIDͷϋογϡΛܭࢉͯ͠γϟʔ υͰׂΓɺొ͢ΔγϟʔυΛܾΊΔ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 doc1 -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited υΩϡϝϯτొγʔέϯε • γϟʔυ͕ܾ·ͬͨΒɺϓϥΠϚϦγϟʔυ͕͋ Δϊʔυ͕Ͳ͔͜ΛௐΔ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 doc1 -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited υΩϡϝϯτొγʔέϯε • ϓϥΠϚϦ͔ΒϨϓϦΧ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 doc1 -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧγʔέϯε • ݕࡧΫΤϦΛ࣮ߦ Query node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧγʔέϯε • ϊʔυ͕̍ड͚औͬͨΫΤϦΛશγϟʔυʹૹ৴ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 Query -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧγʔέϯε • ϊʔυ͕̍ड͚औͬͨΫΤϦΛશγϟʔυʹૹ৴ • γϟʔυຖʹݕࡧ࣮ߦ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 Query Query Query Query Query -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧγʔέϯε • શγϟʔυ͔Βͷ݁ՌΛϚʔδ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 Query ݁Ռ ݁Ռ ݁Ռ ݁Ռ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧγʔέϯε • શγϟʔυ͔Βͷ݁ՌΛϚʔδ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 ݁Ռ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ݕࡧγʔέϯε • ݁ՌΛૹ৴ ݁Ռ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Analysis & Mappings -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Analysisͱʁ • సஔΠϯσοΫεͷ୯ޠʢTermʣΛυΩϡϝϯτ ͔Βநग़͢Δॲཧ • ϑΟʔϧυຖʹࢦఆ͞ΕͨΞφϥΠβ͕ॲཧ • ΞφϥΠβCharFilterɺTokenizerɺTokenFilter͔ Βߏ • ΠϯσοΫε࣌ɺݕࡧ࣌ʹ࣮ߦ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Analysisͷ֓ཁ Analysis υΩϡϝϯτ Term Id ΧπΦ 1 αβΤ 1ɺ2 ϫΧϝ 2 … ... ΠϯσοΫε -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Analysisͷ֓ཁ Analysis ݕࡧ Ωʔϫʔυ ΠϯσοΫε ΠϯσοΫε ݕࡧॲཧ ݕࡧ݁Ռ Term Id ΧπΦ 1 αβΤ 1ɺ2 ϫΧϝ 2 … ... -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Analyzer Analysisͷߏ char_filter tokenizer char_filter token_ filter token_ filter Text Tokens -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Analyzerͷઃఆ {"index":{ “analysis":{ "analyzer" : { "my_analyzer" : { "type" : "custom", "tokenizer" : “kuromoji_tokenizer”, “char_filter" : [“char_filter1”,…], “filter" : [“token_filter1”,…] } } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Char Filter • ೖྗจࣈྻΛจࣈ୯ҐͰॲཧ ! • ྫɿhtml_strip ೖྗɿ ! ग़ྗɿ <title>Elasticsearch is not a service of AWS</title> Elasticsearch is not a service of AWS -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Tokenizer • ೖྗจࣈྻΛτʔΫϯྻʹׂ • τʔΫϯͷׂ͢ΔϩδοΫτʔΫφΠβʹ ґଘ • ྫɿstandard ೖྗɿ ! ग़ྗɿ Elasticsearch is not a service of AWS Elasticsearch is not a service of AWS -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Tokenizer • ྫɿkeyword ग़ྗɿ • ྫɿkuromoji_tokenizer ೖྗɿ ! ग़ྗɿ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ Elasticsearch is not a service of AWS ण͕࢘ඒຯ͔ͬͨ͠ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited TokenFilter • TokenizerʹΑΓग़ྗ͞ΕͨTokenྻʹରͯ͠ॲཧ • ྫɿlowercase ೖྗɿ ! ग़ྗɿ • ྫɿstop ೖྗɿ ! ग़ྗɿ Elasticsearch is not a service of AWS elasticsearch is not a service of aws Elasticsearch is not a service of AWS Elasticsearch service AWS -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited TokenFilter • ྫɿkuromoji_baseform ೖྗɿ ! ग़ྗɿ • ྫɿkuromoji_readingform ೖྗɿ ! ग़ྗɿ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ ण࢘ ͕ ඒຯ͍͠ ͨ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ sushi ga oishika ta -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Mapping • ΠϯσοΫεͷߏʢschemaʣΛఆٛ • Schema less!͚ͩͲ… • ܗࣜΛࢦఆʢanalyze͢Δ/͠ͳ͍ɺ/গɺ ͳͲʣ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿWikipediaͷMappingͷҰ෦ "mappings": { "page": { "properties": { "link": { "type": "string", "fields": { "raw": {"type": "string", "index": "not_analyzed"} } }, "text": { "type": "string"}, "title": { "type": "string", "fields": { "raw": {"type": "string", "index": "not_analyzed"} }… -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited aggregation -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Aggregationͱ • 1.0͔Βಋೖ • FacetΑΓڧྗͳूܭͳͲ͕Մೳ ֊తͳूܭɺάϧʔϓԽ ಈతͳूܭɺάϧʔϓԽ • େ͖͘2छྨ BucketɹυΩϡϝϯτΛ͝ͱʹ݁ՌΛάϧʔϐϯά Metricɹ υΩϡϝϯτͷ࣋ͭΛूܭ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿݴޠ͝ͱͷϑΝηοτ » curl -XGET twitter-2014.08.22/_search -d ' GET twitter-2014.08.22/_search { "facets": { "lang_facet": { "terms": {"field": "lang"} } } } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿݴޠ͝ͱͷϑΝηοτ "facets": { "lang_facet": { "_type": "terms", "missing": 0, "total": 4521075, "other": 320228, "terms": [ { "term": "en", "count": 1471819 }, { "term": "ja", "count": 980145 }, -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿݴޠ͓ΑͼҬͷूܭ » curl -XGET twitter-2014.08.22/_search -d ' { "aggs": { "lang": { "terms": {"field": "lang" }, "aggs": { "place": { "terms": { "field": “place.full_name", "size": 10 } } } } } } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿݴޠ͓ΑͼҬͷूܭ "aggregations": { "lang": { "buckets": [{…}, { "key": "ja", "doc_count": 980145, "place": { "buckets": [ { "key": "ژࢢ෬ݟ۠, ژ", "doc_count":252 }, { "key": "ઍా۠, ౦ژ", "doc_count": 39 },… -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited bucket • filter • filters (>=1.4.0) • missing • nested • reverse nested (>=1.2.0) • children • terms • significant terms (>=1.1.0) • range • date range • ipv4 range • histogram • date histogram • geo distance • geohash grid -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited metric • min • max • sum • avg • stats • extended_stats • value count • percentiles (>=1.1.0) • percentile ranks (>=1.3) • cardinality (>=1.1.0) • geo bounds (>=1.3.0) • top_hits (>=1.3.0) • scripted metric (>= 1.4.0) -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ͦͷଞͷػೳ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Geo • ͞·͟·ͳܗࣜͷσʔλͰGeoݕࡧՄೳ • σʔλɿ ҢܦɺGeoHashɺGeoShape • ݕࡧɿ geo shape queryɺgeo shape filterɺgeo distance filterɺ geo distance range filterɺgeo bounding box filterɺ geo polygon filter -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿGeoσʔλొ ## Ңܦ { "pin" : { "location" : "41.12,-71.34" } } ! ## GeoHash { "pin" : { "location" : "drm3btev3e86" } } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿGeoσʔλొ ## GeoShape(Polygon) { "location" : { "type" : "polygon", "coordinates" : [[ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ]] } } ## GeoShape(Circle) { "location" : { "type" : "circle", "coordinates" : [-45.0, 45.0], "radius" : "100m" } } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿGeoσʔλݕࡧ { "query": { "geo_shape": { "location": { "shape": { "type": "envelope", "coordinates": [[13, 53],[14, 52]] } } } } } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿڑʹΑΔϑΟϧλ { "filtered" : { "query" : {"match_all" : {}}, "filter" : { "geo_distance" : { "distance" : "200km", "pin.location" : { "lat" : 40, "lon" : -70 } } } } } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Percolator • υΩϡϝϯτͰͳ͘ΫΤϦΛొ • _percolate APIʹυΩϡϝϯτΛૹ৴ • ొ͞ΕͨΫΤϦʹϚονͨ͠߹ɺ Ϛονͨ͠ΫΤϦΛฦ͢ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿΫΤϦొ curl -XPUT 'localhost:9200/my-index/.percolator/1' -d '{ "query" : { "match" : { "message" : "bonsai tree" } } }' -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿσʔλνΣοΫ curl -XGET 'localhost:9200/my-index/message/_percolate' -d '{ "doc" : { "message" : "A new bonsai tree in the office" } }' -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿ݁Ռ { "took" : 19, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "total" : 1, "matches" : [ { "_index" : "my-index", "_id" : "1" } ] } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Snapshot/Restore • 1.0͔Βಋೖ • ΠϯσοΫε୯ҐͰSnapshot/RestoreՄೳ • อଘઌʢRepositoryʣ Shard FSɺS3ɺHDFSɺAzure… -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿϦϙδτϦ࡞ $ curl -XPUT 'http://localhost:9200/_snapshot/ my_backup' -d '{ "type": "fs", "settings": { "location": "/mount/backups/my_backup", "compress": true } }' -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ྫɿSnapshotͱϦετΞ ## Snapshot $ curl -XPUT “localhost:9200/_snapshot/my_backup/ snapshot_1?wait_for_completion=true" ! ! ## Restore $ curl -XPOST "localhost:9200/_snapshot/my_backup/ snapshot_1/_restore" -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Ecosystem • Plugins ϓϥάΠϯʹΑΔػೳͷՃ • ΫϥΠΞϯτϥΠϒϥϦ Ruby, python, php, perl, javascript, .NET Scala, clojure, go • Kibana • Logstash -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Hadoop࿈ܞ&Kibana • ʮαʔόʗΠϯϑϥΤϯδχΞཆಡຊ ϩάऩ ूʙՄࢹԽฤʯൃചத • Hadoop࿈ܞͷհ • Kibanaͷৄࡉ -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Tools for sys admins -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited REST-based management • Elasticsearch is full of monitoring APIs Everything is returned as JSON • Humans are not the world’s best JSON parsers • What if elasticsearch had an easy to use interface from the commandline? -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Which node is the master? $ curl "localhost:9200/_cluster/state?pretty&filter_metadata=true&! filter_routing_table=true"! {! "cluster_name" : "elasticsearch",! "master_node" : "GNf0hEXlTfaBvQXKBF300A",! "blocks" : { },! "nodes" : {! "ObdRqLHGQ6CMI5rOEstA5A" : {! "name" : "Triton",! "transport_address" : “inet[/10.0.1.11:9300]”,! "attributes" : { }! },! "4C7pKbfhTvu0slcSy_G4_w" : {! "name" : "Kid Colt",! "transport_address" : "inet[/10.0.1.12:9300]",! "attributes" : { }! },! "GNf0hEXlTfaBvQXKBF300A" : {! "name" : "Lang, Steven",! "transport_address" : "inet[/10.0.1.13:9300]",! "attributes" : { }! }! }! } -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Which one is the master? (v1.0) $ curl localhost:9200/_cat/master GNf0hEXlTfaBvQXKBF300A 10.0.1.13 Lang, Steven -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited _cat/* api • /_cat/allocation • /_cat/count • /_cat/health • /_cat/master • /_cat/aliases • /_cat/nodes • /_cat/recovery • /_cat/shards • /_cat/indices • /_cat/thread_pool -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Monitor your cluster with Marvel • Point in time views are a start • Marvel shows historical trends • Visualize cluster behavior, act before problems ! • Free for development, $500/year for up to 5 nodes -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Overview -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Node statistics -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Index statistics -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Cluster Pulse -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Sense -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited σϞ for Elasticsearch ຊޠWikipedia -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited σϞ for Elasticsearch Twitter Stream -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited σϞ for Kibana Twitter Stream -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited σϞ for Kibana Git log -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited ࢀߟจݙ • Elasticsearch - The Definitive guide http://www.elasticsearch.com/guide/en/elasticsearch/guide/ current/index.html • ॻ੶ ElasticSearchServerຊޠ൛ αʔό/ΠϯϑϥΤϯδχΞ ɹཆಡຊɹϩάऩू -
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Q&A • Training http://training.elasticsearch.com • Support http://www.elasticsearch.com/support/ • Github https://github.com/elasticsearch