Reaching Zen in Your Cluster Coordination Layer Philipp Krenn @xeraa
Slide 1
Slide 2
Developer
Slide 3
Slide 4
Concepts Cluster, Node, Index, Shard, Document, ID
Slide 5
Cluster Coordination?
Slide 6
Cluster State?
Slide 7
== Cluster Metadata Cluster Settings Index Metadata Lots more
Slide 8
GET _cluster/state Only move forward Do not lose data
Slide 9
Demo Cluster Cluster State
Slide 10
3 Main Components Discovery Master Election Cluster State Publication
Slide 11
Zen Zen to Zen2 Not pluggable
Slide 12
Slide 13
Discovery Where are master-eligible nodes? Is there a master already?
Slide 14
Slide 15
Settings discovery.zen.ping.unicast.hosts -> discovery.seed_hosts static discovery.zen.hosts_provider -> discovery.seed_providers dynamic (File, EC2, GCE,…)
Slide 16
Master Election Agree which node should be master Form a cluster
Slide 17
Slide 18
Slide 19
Slide 20
Slide 21
Slide 22
Slide 23
Slide 24
discovery.zen. minimum_master_node s Trust users?
Slide 25
Demo Upgrade 6.7 -> 7.0, 6.8 -> 7.1
Slide 26
Demo Full Cluster Restart [CTRL] + [C]
Slide 27
cluster. initial_master_node s Nodes for the very first election
Slide 28
List of node names OK to set on multiple nodes as long as they are all consistent Ignored once node has joined a cluster even if restarted Unnecessary when joining new node to existing cluster
Slide 29
Upgrade 6 to 7 Full cluster restart: Set cluster.initial_master_nodes Rolling upgrade: cluster.initial_master_nodes not required
Slide 30
Demo Cluster Rebuild
Slide 31
elasticsearch2 | {“type”: “server”, “timestamp”: “2019-05-24T14:02:51,173+0000”, “level”: “WARN”, “component”: “o.e.c.c.ClusterFormationFailureHelper”, “cluster.name”: “docker-cluster”, “node.name”: “elasticsearch2”, “message”:
Slide 32
“master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and [cluster.initial_master_nodes] is empty on this node: have discovered [ {elasticsearch1}{pSUJ60tSRWSrcWkRevLfyA}{_jIaabgyTQOHAOjcwUruIQ} {192.168.112.3}{192.168.112.3:9300} {ml.machine_memory=1073741824, ml.max_open_jobs=20, xpack.installed=true}, {elasticsearch3}{ngaTCze8QHSHydCXsttXyw}{mbIad-A4SLOJvP7Ava5dEw} {192.168.112.4}{192.168.112.4:9300} {ml.machine_memory=1073741824, ml.max_open_jobs=20, xpack.installed=true} ]; discovery will continue using [192.168.112.3:9300, 192.168.112.4:9300] from hosts providers and [ {elasticsearch2}{iANt64LESxqjJv8tHV5KKw}{K0bYEuQ2TnamsiOefTUXgQ} {192.168.112.2}{192.168.112.2:9300} {ml.machine_memory=1073741824, xpack.installed=true, ml.max_open_jobs=20} ] from last-known cluster state; node term 0, last-accepted version 0 in term 0”
Slide 33
Result: Safer Cluster Operations
Slide 34
Dynamic Cluster Scaling Master-ineligible: same as before Adding master-eligible: Just do it Removing master-eligible: Just do it, as long as you remove less than half
Slide 35
Cluster State Publication Agree on cluster state updates Broadcast updates to all nodes
Slide 36
Slide 37
Slide 38
Conclusion
Slide 39
Zen to Zen2 Faster, safer, more debuggable
Slide 40
Reaching Zen in Your Cluster Coordination Layer Philipp Krenn @xeraa
