I am currently trying to understand how RocksDB works in flink. We currently have an application that is running in flink inside AWS EKS.
We currently have the state.backend.rocksdb.localdir commented out in the flink config file. Does that mean RocksDB is not being used? In the flink UI I can see keys getting added to RocksDB does that mean it’s only storing in memory and not on disk? Not sure if it still uses disk when using Kubernetes without any local storage enabled in flink.
I am currently trying to add ebs persistent storage and I am able to do it with running a basic test pod in EKS. When I uncomment the state.backend.rocksdb.localdir in the flink config file it throws an error that it can not find the path. Below I attached a snippet of the task manager manifest files. Would also add I have added the PVC file and the storage class file. Any help would be appreciated.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ebs-sc
provisioner: kubernetes.io/aws-ebs
volumeBindingMode: WaitForFirstConsumer
parameters:
type: gp2
iopsPerGB: "10"
fsType: ext4
reclaimPolicy: Retain
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ebs-claim
spec:
accessModes:
- ReadWriteOnce
storageClassName: ebs-sc
resources:
requests:
storage: 50Gi
containers:
- name: taskmanager
image: 1234566.dkr.ecr.us-east-1.amazonaws.com/test123:latest
imagePullPolicy: Always
resources:
requests:
cpu: 3
memory: "6144Mi"
limits:
cpu: 3
#memory: "4096Mi"
memory: "6144Mi"
#memory: "8192Mi"
env:
- name: STATE_BACKEND
value: "rocksdb"
- name: STATE_BACKEND_ROCKSDB_LOCALDIR
value: "/rocksdb/data"
args: ["taskmanager"]
ports:
- containerPort: 6122
name: rpc
- containerPort: 6125
name: query-state
livenessProbe:
tcpSocket:
port: 6122
initialDelaySeconds: 30
periodSeconds: 60
volumeMounts:
- name: persistent-storage
mountPath: /rocksdb/data
- name: flink-config-volume
mountPath: /opt/flink/conf/
# - name: rocksdb-local-store
# mountPath: /mnt/flink/staleness/rocksdb
securityContext:
runAsUser: 9999 # refers to user _flink_ from official flink image, change if necessary
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: ebs-claim
- name: flink-config-volume
configMap:
name: inventory-staleness-stg-config
items:
- key: flink-conf.yaml
path: flink-conf.yaml
- key: log4j-console.properties
path: log4j-console.properties
- name: job-artifacts-volume
hostPath:
path: /host/path/to/job/artifacts