Upgrade to Pro — share decks privately, control downloads, hide ads and more …

La Suite ELK au service des développeurs et de ...

La Suite ELK au service des développeurs et de la production

Comment la Suite ELK facilite la centralisation et la corrélation d'indicateurs hétérogènes, pour une meilleure prise de décision.

Jérémy Lecour

October 14, 2016
Tweet

More Decks by Jérémy Lecour

Other Decks in Technology

Transcript

  1. metrics • utilisateurs connectés • nombre de commandes • panier

    moyen • état du stock • temps de chargement • taux d’échec • réponse de la BDD • disponibilité des services métier système
  2. xeonys-fr-01 HAProxy Apache/FPM A1 Apache/FPM A2 BDD-A Queues Varnish Apache/FPM

    B1 Apache/FPM B2 BDD-B Queues Apache/FPM B3 xeonys-uk HAProxy Varnish Apache/FPM 1 Apache/FPM 2 BDD Queues xeonys-de HAProxy Varnish Apache/FPM 1 Apache/FPM 2 BDD Queues Apache/FPM 3
  3. ELK

  4. Logstash Extract Transform Load inputs: beats: port: 5044 gelf: port:

    12201 use_udp: true filters: grok: pattern: httpd date: match: dd-MMM-YYYY HH:mm:ss mutate: convert: [line: integer] geoip: src: remote_ip target: geo
  5. Logstash Extract Transform Load inputs: beats: port: 5044 gelf: port:

    12201 use_udp: true filters: grok: pattern: httpd date: match: dd-MMM-YYYY HH:mm:ss mutate: convert: [line: integer] geoip: src: remote_ip target: geo outputs: elasticsearch: host: 127.0.0.1:9200 index: logs-%{type}-%{+YYYY.MM.dd} stdout: codec: rubydebug
  6. administration facile paquets à installer très peu de config de

    départ cluster trivial sauvegardes par snapshot
  7. $ curl -XGET 'http://localhost:9200/_search' -d '{ "query": { "bool" :

    { "must" : { "query_string" : { "query" : "some query string here" } }, "filter" : { "term" : { "user" : "kimchy" } } } } } ' documents
  8. $ curl -XGET 'http://localhost:9200/_search' -d '{ "query": { "bool" :

    { "must" : { "query_string" : { "query" : "some query string here" } }, "filter" : { "term" : { "user" : "kimchy" } } } } } ' documents { "_shards":{ "total" : 5, "successful" : 5, "failed" : 0 }, "hits":{ "total" : 1, "hits" : [ { "_index" : "twitter", "_type" : "tweet", "_id" : "1", "_source" : { "user" : "kimchy", "postDate" : "2009-11-15T14:12:12", "message" : "hello Elasticsearch" } } ] } }
  9. agrégations GET /cars/transactions/_search { "size" : 0, "aggs": { "colors":

    { "terms": { "field": "color" }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } }
  10. agrégations GET /cars/transactions/_search { "size" : 0, "aggs": { "colors":

    { "terms": { "field": "color" }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } { ... "aggregations": { "colors": { "buckets": [ { "key": "red", "doc_count": 4, "avg_price": { "value": 32500 } }, { "key": "blue", "doc_count": 2, "avg_price": { "value": 20000 } }, { "key": "green", "doc_count": 2, "avg_price": { "value": 21000 } } ] } } ... }
  11. éléments de corrélation Queues PHP HTTP Proxy date request method

    referrer user-agent host stack date file line class function message severity app_name app_env host stack
  12. éléments de corrélation Queues PHP HTTP Proxy date request method

    referrer user-agent host stack date file line class function message severity app_name app_env host stack date request method frontend backend host stack
  13. éléments de corrélation Queues PHP HTTP Proxy date request method

    referrer user-agent host stack date file line class function message severity app_name app_env host stack date request method frontend backend host stack date message severity app_name app_env host stack
  14. collecte inputs: beats: port: 5044 gelf: port: 12201 use_udp: true

    Logstash + lib/gelf.php Icons by Chameleon Design from the Noun Project
  15. collecte inputs: beats: port: 5044 gelf: port: 12201 use_udp: true

    Logstash + lib/gelf.php Icons by Chameleon Design from the Noun Project
  16. collecte inputs: beats: port: 5044 gelf: port: 12201 use_udp: true

    Logstash + lib/gelf.php prospectors: paths: /var/log/*.log input_type: log output: logstash: hosts: logstash:5044 Filebeat Icons by Chameleon Design from the Noun Project
  17. collecte inputs: beats: port: 5044 gelf: port: 12201 use_udp: true

    Logstash + lib/gelf.php prospectors: paths: /var/log/*.log input_type: log output: logstash: hosts: logstash:5044 Filebeat Icons by Chameleon Design from the Noun Project
  18. modèle économique Libre, gratuit • Elasticsearch • Logstash • Kibana

    • Beat • Curator • … Services • formation • assistance • consulting • X-Pack
  19. modèle économique Libre, gratuit • Elasticsearch • Logstash • Kibana

    • Beat • Curator • … Services • formation • assistance • consulting • X-Pack Cloud • As A Service • Hébergé
  20. forge.evolix.org www.evolix.fr @evolix – @jlecour [email protected] @ wiki.evolix.org Git logo

    from git-scm.org Chain by Bohdan Burmich from the Noun Project Paw Print by Mattijs Dekkers from the Noun Project Twitter logo from twitter.com