Centralized Logging Patterns Philipp Krenn 44444 @xeraa

Developer Advocate

Apache License 2.0

Disclaimer I build highly monitored Hello World apps

Example: Java

.NET: NLog PHP: Monolog JavaScript: Winston ...

Parse ! 44

Pro: No change Con: RegEx

Send ✉ 44

Pro: No more files Con: Outages & coupling

Structure ! 44

Pro: Right format Con: Special log appender

Containerize ! 44

https://turnoff.us/geek/before- devops-after-devops/

Where to put Filebeat? Sidecar

Default JSON log filebeat.prospectors:

  • type:

log paths:

"/var/lib/docker/containers//.log"

json.message_key:

log

json.keys_under_root:

true processors:

  • add_docker_metadata:

~

Metadata {

"host" : "10.4.15.9" ,

"port" : 6379 ,

"docker" : {

"container" : {

"id" : "382184ecdb385cfd5d1f1a65f78911054c8511ae009635300ac28b4fc357ce51" ,

"name" : "my-java" ,

"image" : "my-java:1.0.0" ,

"labels" : {

"app" : "java" } } } }

Mount log path my-java: container_name:

my-java hostname:

my-java build:

${PWD}/config/my-java networks:

['stack'] command:

java

-jar

my-java.jar volumes: -

./logs/my-java/:/opt/my-java/logs/ filebeat: container_name:

filebeat hostname:

filebeat image:

"docker.elastic.co/beats/filebeat:${ELASTIC_VERSION}" volumes: -

./logs/my-java/:/var/log/my-java/ -

./docker-compose/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro command:

filebeat

-e networks:

['stack']

Configuration templates filebeat.autodiscover: providers: - type:

docker templates: - condition: equals:

docker.container.image:

redis config: - type:

docker

containers.ids: -

"${data.docker.container.id}" exclude_lines:

["^\s+[\-`('.|_]"]

Drop asciiart lines

Pro: Hot ! Con: Complexity

Orchestrate ! 44

Where to put Filebeat? DaemonSet

Metadata processors:

  • add_kubernetes_metadata: in_cluster:

true

Metadata {

"host" : "172.17.0.21" ,

"port" : 9090 ,

"kubernetes" : {

"container" : {

"id" : "382184ecdb385cfd5d1f1a65f78911054c8511ae009635300ac28b4fc357ce51" ,

"image" : "my-java:1.0.0" ,

"name" : "my-java" },

"labels" : {

"app" : "java" , },

"namespace" : "default" ,

"node" : {

"name" : "minikube" },

"pod" : {

"name" : "java-2657348378-k1pnh" } }, }

Configuration templates filebeat.autodiscover: providers: - type:

kubernetes templates: - condition: equals:

kubernetes.namespace:

kube-system config: - type:

docker

containers.ids: -

"${data.kubernetes.container.id}" exclude_lines:

["^\s+[\-`('.|_]"]

Drop asciiart lines

Customize indices output.elasticsearch: index:

"%{[kubernetes.namespace]:filebeat}-%{[beat.version]}-%{+yyyy.MM.dd}"

Pro: Hot ! ! ! Con: Complexity++

Conclusion 44

Examples https://github.com/xeraa/java-logging

Parse ! Send ✉ Structure ! Containerize ! Orchestrate !

Questions? 44 Philipp Krenn 44444 @xeraa