Elastic{ON} 2018 - Upgrade to 6.0: Leading with Empathy

Transcript

1. archana sriram kibana engineer elastic eldnahevitaerc upgrade to 6.0 leading with empathy archanid ^ change
2.  who am I
3. 黄 花 き ば な
5.  who is Elastic
7.  who are Elastic
8. Elasticsearch: Core/Infra, Distributed, Search & Aggs, Lucene, Security Documentation Kibana Operations Kibana Platform Monitoring (Kibana & Elasticsearch) Beats Logstash Cloud Support Kibana QA
9.  who are you
10. www.elastic.c o
11. How to ruin your elasticsearch installation
12. How to ruin your elasticsearch installation A heart-rending tragicomedy
13. How to ruin your elasticsearch installation A heart-rending tragicomedy A recurring nightmare
14. How to ruin your elasticsearch installation A heart-rending tragicomedy A recurring nightmare A true story
15. 5.6 5.6 5.6
16. 5.6 5.6 5.6
17. 5.6 5.6
18. 5.6 5.6 6.2
19. 5.6 5.6 6.2
20. [2018-01-27T01:06:02,635][INFO ][o.e.d.z.ZenDiscovery ] [chmgl9ap15] failed to send join request to master [{chmgl9ap16}{- enENLKCQnaYPJW7IL80YA}{sK8- LS1OQhe9zOCTXJpdFg}{10.212.81.133}{10.212.81.133:9300}{ml.ma x_open_jobs=10, ml.enabled=true}], reason [RemoteTransportException[[chmgl9ap16][10.212.81.133:9300][intern al:discovery/zen/join]]; nested: IllegalStateException[failure when sending a validation request to node]; nested: RemoteTransportException[[chmgl9ap15][10.212.81.53:9300][internal: discovery/zen/join/validate]]; nested: IllegalStateException[Security index is not on the current version [6] - The Upgrade API must be run for 6.x nodes to join the cluster]; ]
21. [2018-01-27T01:06:02,635][INFO ][o.e.d.z.ZenDiscovery ] [chmgl9ap15] failed to send join request to master [{chmgl9ap16}{- enENLKCQnaYPJW7IL80YA}{sK8- LS1OQhe9zOCTXJpdFg}{10.212.81.133}{10.212.81.133:9300}{ml.ma x_open_jobs=10, ml.enabled=true}], reason [RemoteTransportException[[chmgl9ap16][10.212.81.133:9300][intern al:discovery/zen/join]]; nested: IllegalStateException[failure when sending a validation request to node]; nested: RemoteTransportException[[chmgl9ap15][10.212.81.53:9300][internal: discovery/zen/join/validate]]; nested: IllegalStateException[Security index is not on the current version [6] - The Upgrade API must be run for 6.x nodes to join the cluster]; ]
22. 5.6 5.6 6.2 .security
23. 5.6 5.6 .security
24. 5.6 5.6 5.6 .security
25. [2018-01-27T03:13:53,725][ERROR][o.e.g.GatewayMetaState ] [chmgl9ap15] failed to read local state, exiting… org.elasticsearch.ElasticsearchException: java.io.IOException: failed to read [id:176, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/global-176.st]
26. [2018-01-27T03:13:53,725][ERROR][o.e.g.GatewayMetaState ] [chmgl9ap15] failed to read local state, exiting... org.elasticsearch.ElasticsearchException: java.io.IOException: failed to read [id:176, legacy:false, file:/usr/share/elasticsearch/data/nodes/0/_state/global-176.st]
27. 5.6 5.6 5.6 .security cluster state
28. 5.6 5.6 5.6 .security cluster state
29. 5.6 5.6 5.6 .security cluster state
30. 5.6 5.6 .security cluster state
31. 5.6 5.6 5.6 .security cluster state
32. 5.6 5.6 .security cluster state 5.6
33. 5.6 5.6 .security cluster state 5.6
34. 5.6 5.6 .security cluster state 6.2
35. .security cluster state 6.2 6.2 5.6
36. .security cluster state 6.2 6.2 6.2
37. bin/x-pack/users useradd yasmeen -p ***** -r superuser
38. • First, congratulations on using your literacy skills right now. People who read instructions are rare. • 30% of users read. 15% both read and understand what they read. • You have no idea how much time went into all this documentation. • I bet you will not click the link to the documentation in the previous bullet point. • Also, those statistics about users are made up. • The next bullet point is really not worth reading, I promise. Just skip over it. • Are you frustrated because you cannot both read this slide and hear what I’m saying? • If I said nonsense right now, would you even notice? How to upgrade without borking your system 38 All the steps you need to follow to do it right! Here are the steps:
39.  the 3 empathies
40.  how it feels to use the product today
41.  what’s best for the product
42.  how it feels in the community  as we change
43. structure flexibility
44. www.elastic.c o
45. innovate   replace legacy features   remove fragility   improve user experience   keep our users onboard Development Constitution
46. agility stability
47. upgrade assistant
48. upgrade guide
49. upgrade webinar
50. breaking changes docs
51. solutions for removal of multiple mapping types
54. Kibana’s solution for removal of multiple mapping types
56. convert multiple types automatically*
57. www.elastic.c o
58.  response-ability
59. { }
60. www.elastic.c o
61. www.elastic.c o
62.  the future
63. www.elastic.c o
64. Questions, Stories, Hugs, Tears Visit me at the AMA
65. www.elastic.c o
66. Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/ Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 66 Please attribute Elastic with a link to elastic.co
67. Appendix
68. Mapping types Analogy to SQL gone awry
69. user_name name email content tweeted_at user tweet
70. user_name name email content tweeted_at deleted user tweet
71. user_name name email content tweeted_at deleted user boolean tweet date
72. user_name name email content tweeted_at deleted user boolean tweet date
73. Split types into new indices
74. user_name name email content tweeted_at user tweet
75. user_name name email content tweeted_at user tweet user_name name email user user_name content tweeted_at tweet
76. PUT twitter { "mappings": { "user": { "properties": { "name": { "type": "text" }, "user_name": { "type": "keyword" }, "email": { "type": "keyword" } } }, "tweet": { "properties": { "content": { "type": "text" }, "user_name": { "type": "keyword" }, "tweeted_at": { "type": "date" } } } } }
77. PUT users { "settings": { "index.mapping.single_type": true }, "mappings": { "user": { "properties": { "name": { "type": "text" }, "user_name": { "type": "keyword" }, "email": { "type": "keyword" } } } } } PUT twitter { "mappings": { "user": { "properties": { "name": { "type": "text" }, "user_name": { "type": "keyword" }, "email": { "type": "keyword" } } }, "tweet": { "properties": { "content": { "type": "text" }, "user_name": { "type": "keyword" }, "tweeted_at": { "type": "date" } } } } }
78. PUT tweets { "settings": { "index.mapping.single_type": true }, "mappings": { "tweet": { "properties": { "content": { "type": "text" }, "user_name": { "type": "keyword" }, "tweeted_at": { "type": "date" } } } } } PUT users { "settings": { "index.mapping.single_type": true }, "mappings": { "user": { "properties": { "name": { "type": "text" }, "user_name": { "type": "keyword" }, "email": { "type": "keyword" } } } } } PUT twitter { "mappings": { "user": { "properties": { "name": { "type": "text" }, "user_name": { "type": "keyword" }, "email": { "type": "keyword" } } }, "tweet": { "properties": { "content": { "type": "text" }, "user_name": { "type": "keyword" }, "tweeted_at": { "type": "date" } } } } }
79. POST _reindex { "source": { "index": "twitter", "type": "tweet" }, "dest": { "index": "tweets" } } POST _reindex { "source": { "index": "twitter", "type": "user" }, "dest": { "index": "users" } }
80. GET users/doc/5 { "_index": "users", "_type": "doc", "_id": "5", "_version": 1, "found": true, "_source": { "user_name": "kimchy", "name": "Shay Banon", "email": "shay@elastic.co" } } GET twitter/user/5 { "_index": "twitter", "_type": "user", "_id": "5", "_version": 1, "found": true, "_source": { "user_name": "kimchy", "name": "Shay Banon", "email": "shay@elastic.co" } } OLD WAY TO GET A DOCUMENT NEW WAY TO GET A DOCUMENT
81. GET users/doc/5 { "_index": "users", "_type": "doc", "_id": "5", "_version": 1, "found": true, "_source": { "user_name": "kimchy", "name": "Shay Banon", "email": "shay@elastic.co" } } GET twitter/user/5 { "_index": "twitter", "_type": "user", "_id": "5", "_version": 1, "found": true, "_source": { "user_name": "kimchy", "name": "Shay Banon", "email": "shay@elastic.co" } } OLD WAY TO GET A DOCUMENT NEW WAY TO GET A DOCUMENT
82. POST users/doc/5 { "user_name": "kimchy", "name": "Shay Banon", "email": "shay@elastic.co" } POST twitter/user/5 { "user_name": "kimchy", "name": "Shay Banon", "email": "shay@elastic.co" } OLD WAY TO POST A DOCUMENT NEW WAY TO POST A DOCUMENT
83. Merge fields into single type
84. user_name name email content tweeted_at user tweet
85. user_name name email content tweeted_at user tweet user_name name email content tweeted_at doc
86. user_name name email content tweeted_at user tweet type user_name name email content tweeted_at doc
87. user_name name email content tweeted_at user tweet type user_name name email content tweeted_at doc user tweet
89. PUT new_twitter { "settings": { "index.mapping.single_type": true }, "mappings": { "doc": { "properties": { "type": { "type": "keyword" }, "name": { "type": "text" }, "user_name": { "type": "keyword" }, "email": { "type": "keyword" }, "content": { "type": "text" }, "tweeted_at": { "type": "date" } } } } } PUT twitter { "mappings": { "user": { "properties": { "name": { "type": "text" }, "user_name": { "type": "keyword" }, "email": { "type": "keyword" } } }, "tweet": { "properties": { "content": { "type": "text" }, "user_name": { "type": "keyword" }, "tweeted_at": { "type": "date" } } } } }
90. POST _reindex { "source": { "index": "twitter", }, "dest": { "index": "new_twitter" }, "script": { "source": """ ctx._source.type = ctx._type; ctx._id = ctx._type + '-' + ctx._id; ctx._type = 'doc'; """ } }
91. GET twitter/user/5 { "_index": "twitter", "_type": "user", "_id": "5", "_version": 1, "found": true, "_source": { "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } } GET new_twitter/doc/user-5 { "_index": "new_twitter", "_type": "doc", "_id": "user-5", "_version": 2, "found": true, "_source": { “type”: “user”, "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } } OLD WAY TO GET A DOCUMENT NEW WAY TO GET A DOCUMENT
92. GET new_twitter/doc/user-5 { "_index": "new_twitter", "_type": "doc", "_id": "user-5", "_version": 2, "found": true, "_source": { “type”: “user”, "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } } GET twitter/user/5 { "_index": "twitter", "_type": "user", "_id": "5", "_version": 1, "found": true, "_source": { "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } } OLD WAY TO GET A DOCUMENT NEW WAY TO GET A DOCUMENT
93. GET new_twitter/doc/user-5 { "_index": "new_twitter", "_type": "doc", "_id": "user-5", "_version": 2, "found": true, "_source": { “type”: “user”, "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } } GET twitter/user/5 { "_index": "twitter", "_type": "user", "_id": "5", "_version": 1, "found": true, "_source": { "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } } OLD WAY TO GET A DOCUMENT NEW WAY TO GET A DOCUMENT
94. GET new_twitter/doc/user-5 { "_index": "new_twitter", "_type": "doc", "_id": "user-5", "_version": 2, "found": true, "_source": { “type”: “user”, "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } } GET twitter/user/5 { "_index": "twitter", "_type": "user", "_id": "5", "_version": 1, "found": true, "_source": { "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } } OLD WAY TO GET A DOCUMENT NEW WAY TO GET A DOCUMENT
95. POST new_twitter/doc/user-5 { "type": "user", "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } POST twitter/user/5 { "user_name": "eldnahevitaerc", "name": "Archana Sriram", "email": "archana@elastic.co" } OLD WAY TO POST A DOCUMENT NEW WAY TO POST A DOCUMENT
96. Nest multiple types under single top-level type
97. user_name name email content tweeted_at user tweet
98. user tweet doc type user_name name email type user_name content tweeted_at user_name name email content tweeted_at user tweet