Rook ceph block storage on bare metal - how to export a file from a pod?

Question

I'm trying to work out how to transfer a file from ceph block storage, e.g. a database backup.

I've followed the example here: https://github.com/rook/rook/blob/master/Documentation/ceph-block.md so that I have Wordpress and mysql working on rook-ceph-block.

How can I then transfer a file from the running pods. For example if I wanted to download a database backup onto another host?

Answer 1

Simply do not use block storage.

At the moment I use NFS with Ubuntu+Autofs (outside of Kubernetes) with the relatively new Kubernetes local storage. See at the bottom. One advantage compared to e.g. a default Longhorn is RWX instead of RWO.

With sudo mc you can easily copy stuff back and forth.

You can also easily copy things out of GlusterFS, but in version 7, which I tested a few weeks ago, it is not yet suitable for databases or Redis with many small write operations.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: <PV_NAME>
spec:
  capacity:
    storage: 20Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: <AUTO_FS_PATH>
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - <VM_NAME>
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: <PVC_NAME>
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: local-storage
  resources:
    requests:
      storage: 20Gi

Answer 2

There are a number of approaches that could possibly work. If you know which specific container has the file then it's always possible to configure a node-port, ssh into the container with the file, install and run sshd, then use scp from the target machine using the node-port you allocated.

Another approach would be to create a simple HTTP server in a new pod that mounts the same filesystem. This could expose the file with an HTTP get command. You could do this quite easily with nodejs, express, and fs in a handful of lines of code. This is better than the first solution because you don't have to know which container has the file (something that can be challenging to discover).

However, you are probably doing the wrong thing. The right solution is to configure rook to do this for you using a volume snapshot.