how to dynamicly use pod lables in filebeat to send logs to specific logstash pipeline?

Question

I am running filebeat on kubernetes as daemonset and I have an up an running ELK on kubernetes.I am using rook ceph filesystem as my storage solution.application logs stored in for example :
/var/lib/kubelet/pods/e9e8fb85-8557-455c-8ea8-4d4bf95ebd62/volumes/kubernetes.io~csi/pvc-2e990ca3-0640-49d7-aea6-ce0400c83bef/mount/example.log
by my research I already konw to mount this path in filebeat container and already mounted.so these pods are ephemeral.so my questions are :
1.do I need to use autodisscover feature?
2.do I need to add kubernetes metadata and which one is needed ?
3.how to dynamicly use pod lables in filebeat to send logs to specific logstash pipeline(use autodiscover maybe) ? thanks so much

Answer 1

I think you can use filebeat.autodiscover so you can parse logs from your workloads running on Kubernetes.

Use pod annotations on your workloads to take effect the Hint Based autodiscover

annotations:
    co.elastic.logs.json-logging/json.keys_under_root: "true"
    co.elastic.logs.json-logging/json.add_error_key: "true"
    co.elastic.logs.json-logging/json.message_key: "message"

You can read more details on the docs.