Roles with multiple namespaces in K8s

Question

I have a operator which has multinamespaced controllers, I am using ClusterRole and RoleBinding as shown below:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: null
  name: app-role
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - secrets
  - pods/exec
  verbs:
  - create
  - delete
  - get
  - list
  - watch
- apiGroups:
  - app.group.com
  resources:
  - JavaApps
  verbs:
  - create
  - delete
  - get
  - list
- apiGroups:
  - app.group.com
  resources:
  - DBApps
  verbs:
 - create
  - delete
  - get
  - list

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: app-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: app-role
subjects:
- kind: ServiceAccount
  name: default
  namespace: appns

I am using WATCH_NAMESPACE with multiple namespaces as mentioned in this link (https://sdk.operatorframework.io/docs/building-operators/golang/operator-scope/): If I change ClusterRole to Role and create it in appns then the controller kept on failing with the following error:

reflector.go:147] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:229: Failed to watch *v1alpha1.DbApps: failed to list *v1alpha1.DbApps: DbApps.app.group.com is forbidden: User "system:serviceaccount:appns:default" cannot list resource "DbApps" in API group "app.group.com" in the namespace "appns1"

I tired setting up extra binding but it did not work:

   apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      name: app-rolebinding1
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: app-role
    subjects:
    - kind: ServiceAccount
      name: default
      namespace: appns1

Is it possible to use Role with Multinamespace controllers?

Answer 1

The plain and somewhat misleading answer is no, you cannot. You bind roles either to a namespace or to a cluster - per RoleBinding. However, you can bind a ClusterRole to multiple namespaces with multiple role bindings. We will look into that later, but first a word of caution.

For the use case scenario of an operator, especially one with so far reaching permissions, I'd go with namespaced permissions. Read your ClusterRole carefully: This permission set allows reading and manipulation of all secrets in your cluster - which should be treated as your Crown Jewels. Yes, Kubernetes makes handling secrets look easy as a cluster "root", but cluster wide access to all secrets is something would fail one of the basic principles of cyber security:

Access to confidential information is to be granted strictly on a 'need to know' basis.

That being said, what you can do is to apply a ClusterRole to a namespace. If you do that, a ClusterRole is basically just a template for the permissions you want to grant to several namespaces. You would define a ClusterRole like this:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # "namespace" omitted since ClusterRoles are not namespaced
  name: secret-reader
rules:
- apiGroups: [""]
  #
  # at the HTTP level, the name of the resource for accessing Secret
  # objects is "secrets"
  resources: ["secrets"]
  verbs: ["get", "watch", "list"]

Now, if you did a ClusterRoleBinding, the according roles could read secrets in all namespaces.

However, you can bind the ClusterRole to specific namespaces:

apiVersion: rbac.authorization.k8s.io/v1
# This role binding allows "dave" to read secrets in the "development" namespace.
kind: RoleBinding
metadata:
  name: read-secrets
  #
  # The namespace of the RoleBinding determines where the permissions are granted.
  # This only grants permissions within the "development" namespace.
  namespace: development
subjects:
- kind: User
  name: dave # Name is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

Note metadata.namespace. Now, this would allow dave to read secrets in the namespace development. You would need to repeat this for all namespaces dave is supposed to access.

Answer 2

@Markus, thanks for the detailed reply and conforming with multi-namespace controller we need clusterrole.

Since we are dealing with service accounts and not with users, I tested with following changes and it worked for me:

apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      name: app-rolebinding1
      namespace: appns1
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: app-role
    subjects:
    - kind: ServiceAccount
      name: default
      namespace: appns

appns is the place where my controllers are running.