Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic Stackでマイクロサービス運用を
楽にするには? / Monitoring ...

Jun Ohtani
January 25, 2019

Elastic Stackでマイクロサービス運用を
楽にするには? / Monitoring Microservices with Elastic Stack

OSC 大阪 2019でのセッションで使用した資料です。
「マイクロサービス」とありますが、アプリケーションの監視の話がメインですので、アプリケーションの監視やログ解析などに興味のある方にも読んでいただける内容となっています。

Jun Ohtani

January 25, 2019
Tweet

More Decks by Jun Ohtani

Other Decks in Technology

Transcript

  1. !2 about • Me, Jun Ohtani / Community Engineer ‒

    lucene-gosenίϛολʔ ‒ σʔλ෼ੳج൫ߏஙೖ໳ ڞஶ ‒ http://blog.johtani.info
 • Elastic, founded in 2012 ‒ Products: Elasticsearch, Logstash, Kibana, Beats 
 Elastic APM, 
 Elastic Cloud, Swiftype 
 Professional services: Support & development subscriptions
 Trainings, Consulting, SaaS
  2. Metrics Logging APM Site
 Search Application Search Business
 Analytics Enterprise


    Search Security
 Analytics Future ιϦϡʔγϣϯ อଘɺݕࡧɺ෼ੳ ՄࢹԽɺ؅ཧ ΠϯδΣετ Kibana Elasticsearch Beats Logstash Elastic Stack
  3. Metrics Logging APM Site
 Search App
 Search Business
 Analytics Enterprise


    Search Security
 Analytics Future ιϦϡʔγϣϯ SaaS Elastic Cloud Self Managed Elastic Cloud
 Enterprise Standalone σϓϩΠ อଘɺݕࡧɺ෼ੳ ՄࢹԽɺ؅ཧ ΠϯδΣετ Kibana Elasticsearch Beats Logstash Elastic Stack
  4. อଘɺݕࡧɺ෼ੳ Elasticsearch ՄࢹԽɺ؅ཧ Kibana Beats ΠϯδΣετ Logstash Metrics Logging APM

    Site
 Search Application Search Business
 Analytics Enterprise
 Search Security
 Analytics Future ιϦϡʔγϣϯ SaaS Elastic Cloud Self Managed Elastic Cloud
 Enterprise Standalone σϓϩΠ Elastic Stack
  5. !13

  6. The Beats family Heartbeat Uptime monitoring Filebeat Log files Winlogbeat

    Windows Event Logs Packetbeat Network data +40 community Beats Metricbeat Metrics Auditbeat Audit data
  7. !16

  8. !18

  9. 19 Elasticsearch Heart of the Elastic Stack ෼ࢄܕɺεέʔϥϒϧ ߴՄ༻ੑ Ϛϧνςφϯτ

    ։ൃऀϑϨϯυϦʔ ϦΞϧλΠϜɺશจݕࡧ ΞάϦήʔγϣϯ
  10. Elasticsearch in 10 seconds • εΩʔϚϑϦʔɺ෼ࢄυΩϡϝϯτετΞɺREST & JSON • Φʔϓϯιʔε:

    Apache License 2.0 • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ • JavaͰ࣮૷ɻ֦ு΋༰қ !28
  11. σʔλొ࿥ 30 curl -XPUT localhost:9200/books/book/1 -d ' { "title" :

    "Elasticsearch - The definitive guide", "authors" : "Clinton Gormley", "started" : "2013-02-04", "pages" : 230 }'
  12. σʔλߋ৽ 31 curl -XPUT localhost:9200/books/book/1 -d ' { "title" :

    "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : "2013-02-04", "pages" : 230 }'
  13. σʔλ࡟আ !32 curl -X DELETE localhost:9200/books/book/1 σʔλͷऔಘ curl —X GET

    localhost:9200/books/book/1 curl —X GET localhost:9200/books/book/1/_source
  14. ݕࡧ - Query DSL !33 curl -XGET ‘localhost:9200/books/doc/_search' -d '{

    "query": { "bool": { "must": [ { "match": { "title": "Search" }}, { "match": { "content": "Elasticsearch" }} ], "filter": [ { "term": { "status": "published" }}, { "range": { "publish_date": { "gte": "2015-01-01" }}} ] } } }'
  15. γϟʔυͱϨϓϦΧ !35 node 1 orders products 1 4 1 2

    2 3 curl -X PUT localhost:9200/orders -d '{ "settings.index.number_of_shards" : 4 "settings.index.number_of_replicas" : 1 }' curl -X PUT localhost:9200/products -d '{ "settings.index.number_of_shards" : 2 "settings.index.number_of_replicas" : 0 }'
  16. ࣗಈతͳ෼ࢄ !37 node 1 orders products 2 1 4 1

    node 2 orders products 2 2 node 3 orders products 3 4 1 3
  17. !38

  18. 39 Kibana Window into the Elastic Stack ՄࢹԽͱ෼ੳ ஍ཧۭؒ ΧελϚΠζͱ

    Ϩϙʔτͷڞ༗ άϥϑ୳ࡧ Elastic Stack΁ͷ ηΩϡΞͳΞΫηεͱ؅ཧ ΧελϜAppsͷ࡞੒
  19. !41 Elastic Stackͷߏ੒ Beats Log Files Metrics Wire Data your{beat}

    Kibana Instances Kafka Distributed Message Queue Notification Queues Storage Metrics Data Store Web APIs Social Sensors Elasticsearch Nodes Logstash Nodes
  20. !49 Metricbeat Ϟδϡʔϧ • Aerospike module • Apache module •

    Ceph module • Couchbase module • Docker module • Dropwizard module • Elasticsearch module • Etcd module • Golang module • Graphite module • HAProxy module • HTTP module • Jolokia module • Kafka module • Kibana module • Kubernetes module • kvm module • Logstash module • Memcached module • MongoDB module • Munin module • MySQL module • Nginx module • • PHP_FPM module • PostgreSQL module • Prometheus module • RabbitMQ module • Redis module • System module • uwsgi module • vSphere module • Windows module • ZooKeeper module

  21. Filebeat modules - v6.4.2 • Apache2 module • Auditd module

    • Icinga module • IIS module • Kafka module • Logstash module • MongoDB module • MySQL module • Nginx module • Osquery module • PostgreSQL module • Redis module • System module • Traefik module
  22. !60

  23. !66

  24. !67

  25. !68

  26. !70

  27. !71

  28. !73 σʔλͷొ࿥ํ๏ • Kibanaͷαϯϓϧσʔλʢ6.4͔Βʣ • LogstashͰJDBC input • LogstashͰCSV •

    FilebeatͰΞΫηεϩά • MetricbeatͰϝτϦοΫ • PacketbeatͰMySQL/PostgreSQLͷύέοτղੳ
  29. !85 ࢀߟจݙ • Elasticsearch - The Definitive guide ‒ http://www.elastic.co/guide/en/elasticsearch/guide/current/

    index.html • ॻ੶ʢ೔ຊޠʣ ‒ σʔλ෼ੳج൫ߏஙೖ໳ ‒ Elasticsearch࣮ફΨΠυ
  30. !87 ࢀߟαΠτ • Ϣʔεέʔε • https://www.elastic.co/use-cases • DiscussʢWebϑΥʔϥϜʣ • https://discuss.elastic.co

    • Elastic{ON}ͷϏσΦͱࢿྉ • https://www.elastic.co/elasticon/videos • αϙʔτϝχϡʔ • https://www.elastic.co/subscriptions