Providing and Supporting Docker Images Philipp Krenn 4444 @xeraa
Slide 1
Slide 2
Infrastructure | Developer !
Slide 3
Who uses Docker?
Slide 4
Who uses Docker in production?
Slide 5
Who uses stateful Docker images?
Slide 6
Who uses our images?
Slide 7
Who uses our stack with other images?
Slide 8
Docker: the world's most heavily funded college project Internal quote from Slack
Slide 9
Slide 10
Content "Official" Base images Release policy Security Support Orchestration
Slide 11
Docker Hub "official"
Slide 12
What do you get? docker pull elasticsearch
Slide 13
Slide 14
Slide 15
Slide 16
Custom registry docker.elastic.co
Slide 17
I’m surprised more people don’t just host their own container registries since doing that is faster than every cloud offering and docker hub https://twitter.com/jessfraz/status/978449365261082625
Slide 18
Our Motivation Download statistics Speed & reliability
Slide 19
Slide 20
https://www.docker.elastic.co
Slide 21
Problems Some broken tooling like automated builds, Kitematic,... China IPv6
Slide 22
Slide 23
https://hub.docker.com/r/elastic/ elasticsearch/ kibana/ *beat/ logstash/ apm-server/
Slide 24
Quick Quiz Our image or not?
Slide 25
docker pull logstash:alpine
Slide 26
docker pull elastic/elasticsearch
Slide 27
docker pull kibana
Slide 28
docker pull docker.elastic.co/beats/filebeat:6.2.3
Slide 29
Base Images
Slide 30
Elasticsearch Alpine
Slide 31
Kibana 4 Beats 4 Logstash Ubuntu
Slide 32
Common base image in 5.4+ CentOS 7
Slide 33
https://github.com/elastic/elasticsearch-docker/blob/master/templates/Dockerfile.j2
FROM centos:7
LABEL maintainer "Elastic Docker Team docker@elastic.co"
ENV ELASTIC_CONTAINER true
ENV PATH /usr/share/elasticsearch/bin:$PATH
ENV JAVA_HOME /usr/lib/jvm/jre-1.8.0-openjdk
RUN yum update -y &&
yum install -y nc java-1.8.0-openjdk-headless unzip wget which &&
yum clean all
Slide 34
Upside Similar setup Shared layers JVM images largish anyway
Slide 35
Slide 36
Downside Size
Slide 37
$ docker images REPOSITORY TAG IMAGE ID SIZE docker.elastic.co/kibana/kibana 6.3.0 128b6bfb9645 729MB docker.elastic.co/elasticsearch/elasticsearch 6.3.0 7e188ef97fe0 783MB docker.elastic.co/beats/filebeat 6.3.0 f90fd7f32529 318MB docker.elastic.co/kibana/kibana 6.2.4 327c6538ba4c 933MB docker.elastic.co/elasticsearch/elasticsearch 6.2.4 7cb69da7148d 515MB docker.elastic.co/beats/filebeat 6.2.4 26a00abcde82 319MB docker.elastic.co/kibana/kibana 5.6.7 0f8347a118b3 697MB docker.elastic.co/elasticsearch/elasticsearch 5.6.7 f9bc1b5416b9 574MB docker.elastic.co/beats/filebeat 5.6.7 02648e71cced 334MB docker.elastic.co/kibana/kibana 5.3.3 ffe778f7e489 679MB docker.elastic.co/elasticsearch/elasticsearch 5.3.3 5857f98b5920 165MB docker.elastic.co/beats/filebeat 5.3.3 c01be8a8f630 232MB
Slide 38
5.3 5.6 6.2 6.3 ES 165MB 574MB 515MB 783MB Kibana 679MB 697MB 933MB 729MB Filebeat 232MB 334MB 319MB 318MB
Slide 39
Does it matter? stateful vs stateless
Slide 40
What to include?
Slide 41
Single image for 5.x Platinum trial
Slide 42
Three flavors 6.0 to 6.2 Basic * , OSS, Platinum trial * Default
Slide 43
Slide 44
Two flavors 6.3+ Basic / Platinum trial * , OSS 783MB vs 673MB * Default
Slide 45
Future
Slide 46
Multiple JDK versions? 6.2: OpenJDK 8 6.3 BC: OpenJDK 10.0.1
Slide 47
Multiple base images?
Slide 48
Windows?!
Slide 49
Release Policy
Slide 50
No :latest
Slide 51
Zombies ideas that should have been killed by evidence, but keep shambling along
Slide 52
5 and 5.6 ?
Slide 53
What's in a tag? docker.elastic.co/elasticsearch/elasticsearch 5.3.3 5857f98b5920 4 months ago docker.elastic.co/beats/filebeat 5.3.3 c01be8a8f630 5 months ago docker.elastic.co/kibana/kibana 5.3.3 ffe778f7e489 5 months ago
Slide 54
Currently Overwrite tag
Slide 55
Label Schema
LABEL org.label-schema.schema-version="1.0"
org.label-schema.vendor="Elastic"
org.label-schema.name="elasticsearch"
org.label-schema.version="{{ elastic_version }}"
org.label-schema.url="https://www.elastic.co/products/elasticsearch"
org.label-schema.vcs-url="https://github.com/elastic/elasticsearch-docker"
{% if image_flavor == 'oss' -%}
license="Apache-2.0"
{% else -%}
license="Elastic License"
{% endif -%}
Slide 56
Base image & JVM direct dependencies
Slide 57
Future Add image version?
Slide 58
Security
Slide 59
Run Elasticsearch as root
Slide 60
Cockroaches claims that disappear for a while when proved wrong, but just keep on coming back
Slide 61
Mode Production: Clusterable Development: Local network
Slide 62
"Docker" mode discovery.type=single-node
Slide 63
Bootstrap checks are here to stay
Slide 64
6.0+ no more default credentials
Slide 65
6.0+ X-Pack non-trial requires certificates
Slide 66
Support
Slide 67
The container runs Elasticsearch as user elasticsearch using uid:gid 1000:1000 . https://www.elastic.co/guide/en/elasticsearch/reference/current/ docker.html
Slide 68
Slide 69
Slide 70
Slide 71
Don't mutate the bind mounted local directory
Slide 72
Zombies ideas that should have been killed by evidence, but keep shambling along
Slide 73
Those who do not understand Unix are condemned to reinvent it, poorly. — Henry Spencer
Slide 74
Docker default value? LimitNOFILE & LimitNPROC
Slide 75
infinity https://github.com/moby/moby/commit/ 8db61095a3d0bcb0733580734ba5d54bc27a614d (July 2016)
Slide 76
Test $ docker run --rm centos:7 /bin/bash -c \
'ulimit -Hn && ulimit -Sn && ulimit -Hu && ulimit -Su' 1048576 1048576 unlimited unlimited
Slide 77
Combine two new systems to get chaos & despair
Slide 78
Docker is a leaky abstraction
Slide 79
Slide 80
Orchestration
Slide 81
Who uses Kubernetes?
Slide 82
Who uses Swarm?
Slide 83
Who uses Mesos?
Slide 84
Who uses Nomad?
Slide 85
No orchestration yet
Slide 86
Testing images for orchestration
Slide 87
Kubernetes 1.8 allows dots in env vars https://github.com/kubernetes/kubernetes/issues/2707
Slide 88
Kubernetes has made huge improvements in the ability to run stateful workloads including databases and message queues, but I still prefer not to run them on Kubernetes. https://twitter.com/kelseyhightower/status/963413508300812295
Slide 89
Kubernetes makes it easier to deploy stateful services not manage them. Stateful services must meet Kubernetes half way and manage their own cluster membership, failover, and replication. CockroachDB and Consul are two great examples, but far from perfect. https://twitter.com/kelseyhightower/status/963415653930553345
Slide 90
Slide 91
Conclusion
Slide 92
"Docker is disrupting the industry"
Slide 93
"Can I run Elasticsearch on Docker?"
Slide 94
"Should I run Elasticsearch on Docker?"
Slide 95
Even when stateful services do the right things managing state is still hard. Mixing stateful and stateless applications on the same cluster elevates the complexity of the entire cluster. Cluster security and upgrades become much harder. https://twitter.com/kelseyhightower/status/963417215608369153
Slide 96
https://www.meetup.com/Elasticsearch-Berlin/ Thursday June 14, 19:00 Springer Nature
Slide 97
Questions & Discussion Philipp Krenn 444 @xeraa
