Kubernetes - RBAC And Admission Controllers

What is RBAC?

From Wikipedia:

"In computer systems security, role-based access control (RBAC) or role-based security is an approach to restricting system access to authorized users. It is an approach to implement mandatory access control (MAC) or discretionary access control (DAC)."

From official Kubernetes doc:

“Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within your organization.”

RBAC (Role-Based Access Control) is a critical concept to understand when working with a Kubernetes cluster, as it determines which users can perform which actions within the cluster. In a multi-tenant environment, you may want to grant access to different teams within their own namespace, with each team having different roles (such as developer or admin). You may also need to define different RBAC roles for different job roles within a namespace and for accessing different resources within a namespace.

Simply put, RBAC allows you to protect access to resources within your cluster(s).

In addition to RBAC, you may also want to use Admission Controllers, which are a set of features in Kubernetes that allow you to control what is allowed into your cluster. With Admission Controllers, you can validate and manipulate incoming requests, and there may be various use cases for doing so. Most Kubernetes distributions come with several out-of-the-box admission controller plugins that are enabled by default, which you should be aware of.

---

❗In this article, we will combine information from the official Kubernetes documentation with our own insights and experience to provide a more comprehensive and understandable overview of the topic. We will clearly distinguish between content from the official documentation and our own additions by highlighting the former in the text

---

This article aims to provide a thorough understanding of the RBAC concept in Kubernetes, rather than step-by-step instructions for setting it up. The concepts discussed in this article are applicable to all Kubernetes distributions, though specific implementations, such as the integration of AWS IAM with Kubernetes RBAC in AWS EKS, may vary. By the end of this article, you should have a solid understanding of how RBAC works in Kubernetes.

For more in-depth instructions on setting up Kubernetes RBAC, keep an eye out for a full Kubernetes course to be released soon at devoriales.com.

RBAC Objects

In this paragraph, we are listing the objects that are related to the Kubernetes RBAC model and where they fit in the cluster. Later on, we will learn how they depend on each other.

• Entities: These are users, groups, or service accounts that need permissions to perform actions in the cluster, either within a namespace or cluster-wide.
• Resources: These are objects within the cluster, such as pods, secrets, config maps, services, and PVCs, that the entities need access to.
• Roles: These define the actions that entities can perform on resources within a namespace.
• ClusterRoles: These work similarly to roles, but are used to define actions that exist at the cluster level (such as nodes or namespaced resources) and can be granted for all namespaces.
• Bindings: RoleBindings and ClusterRoleBindings link an entity (e.g. a user) to a set of permissions (a role or cluster role).
• Namespaces: These provide a security boundary that allows different users or teams to work with their own resources within the cluster. They are useful for creating a multitenant environment within the cluster.

The following represents the components related to Kubernetes RBAC and where they fit in the cluster:

 

Authentication and Authorization

Analogy - authentication

When you pass through different country borders, you need to have a passport to identify yourself.

You need to show the passport to the border authorities as a way of authenticating yourself to them. This proves that you are who you claim to be.

Analogy - Authorization

We can continue from the previous example. Now that you have authenticated yourself to the border authorities, you may want to visit different places within the country. For example, you may want to visit the town hall or claim your social rights.

The question is whether you are authorized to do these things as a foreigner or if you have additional rights as a citizen.

Your authorization level determines what you are allowed to do within the country's borders.

Kubernetes Authentication and Authorization Process

As explained with analogies, identity and authorization are not the same things.

A user may have access to the cluster, but it doesn't mean they have permission to do everything in the cluster.

We will break down the process to understand it fully. It's also important to distinguish between user accounts and service accounts, as they are managed differently.

User Accounts

First, the authentication has to happen before getting to the authorization phase:

In the authentication step, the user is authenticated by an Identity Provider.

Kubernetes does not manage users directly. It assumes that authentication is performed by a third-party Identity Provider.

To quote the official Kubernetes documentation: "Normal users cannot be added to a cluster through an API call."
The user needs to present a valid certificate assigned by the cluster's certificate authority (CA). 

Kubernetes looks at the common name field in the subject of the cert:

(e.g., "/CN=anna")

After the user is authenticated, Kubernetes creates a temporary user object and sends it to the RBAC authorization submodule.

Without going into the details of authentication, the user must always have a valid certificate to enter the cluster.

Only after this step will the user be allowed to perform actions that are permitted by the RBAC rules.

---

❗There are different authentication methods that can be used in Kubernetes that are beyond the scope of this article.

In the following diagram, four authentication methods are listed that can be used  (some will be covered in a future video):

1. X.509 Certificates: pretty simple method. You can generate and validate your self-signed certificate with Kubernetes's CA key.
2. OIDC: used with third-party Identity Provider. Most Kubernetes cloud distributions use this method (EKS, GKE etc)
3. Service Account Authentication Token: Everything is done inside the cluster, you create the SA and use a token from the generated secret.
4. Password files; Not very secure. Very simple to create, basically just a file containing password, username and user id.  

The cloud providers make this implementation pretty straightforward. For instance, AWS EKS has a pretty nice integration between AWS IAM and AWS EKS. You can create roles, users, and groups in AWS IAM and apply those to your EKS cluster. 

---

Service Accounts

Service accounts, in contrast to user accounts, are managed directly by Kubernetes. You can create service accounts directly in Kubernetes and those can then be assigned to e.g. Pods to get access to different resources, like registries, other services, other pods, etc. 

With RBAC, you control what's granted for users to perform in the cluster, like:

• Creating and modifying  deployments and pods
• read secrets in a particular namespace 
• read secrets in all namespaces
• Create RoleBindings in a particular namespace

Service accounts, in contrast to user accounts, are managed directly by Kubernetes. You can create service accounts directly in Kubernetes, which can then be assigned to, for example, pods to access different resources such as registries, other services, and other pods.

With RBAC, you control what actions users are allowed to perform in the cluster, such as creating and modifying deployments and pods, reading secrets in a particular namespace or in all namespaces, and creating role bindings in a particular namespace.

The Authentication and Authorization Process

The authentication and authorization process in Kubernetes works as follows:

1. The subject (user, user group, or service account) presents its token to the Kubernetes API for authentication and authorization.
2. The Kubernetes API does not handle authentication itself, but rather passes the token to the identity provider for authentication, which then returns an approval or denial response.
3. If the subject is authenticated, the Kubernetes API authorizes it to access different resources in the cluster based on the permissions granted to that particular user or service account.
4. The approval or denial is returned to the subject. If approved, the subject is now authorized to perform certain actions in the cluster.

 

Kubernetes Authorization

When a user sends a request to apply a deployment manifest to the cluster, the following occurs:

1. The user sends the request to the kube-apiserver with their token.
2. The kube-apiserver passes the token to the identity provider for authentication.
3. The identity provider authenticates the user and returns an approval or denial response to the kube-apiserver.
4. If the user is authenticated, the kube-apiserver authorizes the request using RBAC to determine what permissions the user has within the cluster.
5. If the user has the necessary permissions, the kube-apiserver applies the deployment manifest to the cluster and stores it in etcd. If the user does not have the necessary permissions, the kube-apiserver returns an error 401 Unauthorized.

Example - Applying a deployment manifest to the cluster

example - deployment manifest file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

The following will occur (as conceptually explained here):

The kube-apiserverwill check for authorization by checking the RBAC permissions that have been granted to the user If the user is authorized, the kube-apiserverwill store the request in etcd The API server will return a response to the client indicating the success or failure of the request If the request was successful, the changes will be applied to the cluster

This is the general process that occurs when a request is made to the kube-apiserver. The specific details may vary depending on the type of request and the resources being accessed.

Example - Grant Permissions to a Service Account

Use-case description: we want to create a service account that also needs to have read permissions to the following Kubernetes objects: services, pods, and configmaps.

To accomplish this task, we need to create the following:

• Service Account: my-app-sa
• Role: viewer
• RoleBinding: my-app-viewer

Here we have manifest files that will create the desired state.

Service Account manifest

apiVersion: v1
kind: ServiceAccount # kind of object
metadata:
  name: serviceaccount:my-app-sa # name of the service account
  namespace: my-namespace

Role manifest

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: role:viewer # name of the role
  namespace: my-namespace
rules:          # list of rules
  - apiGroups:  
      - ''      # empty string means the first built-in group
    resources: # list of resources to which the role applies
      - services 
      - pods
      - configmaps
    verbs: # list of verbs to which the role applies
      - get 
      - list 

RoleBinding manifest

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: rolebinding:my-app-viewer # name of the role binding
  namespace: my-namespace # namespace of the role binding
roleRef:
  apiGroup: rbac.authorization.k8s.io # group of the role
  kind: Role 
  name: role:viewer # name of the role
subjects: # list of subjects to which the role binding applies
  - kind: ServiceAccount
    name: serviceaccount:my-app-sa # name of the service account
    namespace: my-namespace

What is happening when the user submits the manifest files above?

Given that the user has permissions to perform actions required (create SA, create Role, and RoleBindings), the service account will be able to send requests to the following API endpoints:

/api/v1/namespaces/{namespace}/services
/api/v1/namespaces/{namespace}/pods
/api/v1/namespaces/{namespace}/configmaps

---

❗What does an empty string mean in the apiGroups as part of the Role manifest?

The /api an endpoint is legacy and used only for core resources (namespaces, pods, secrets, configmaps, etc.).

/apis/<group-name> is used for the rest of the resources, like custom resources.

When writing an empty string, we mean the core group which is /api endpoint:

rules:          # list of rules
  - apiGroups:  
      - ''      # empty string means the first built-in Core group

/api/v1/namespaces/{namespace}/services
/api/v1/namespaces/{namespace}/pods
/api/v1/namespaces/{namespace}/configmaps

The Core Group:

The API extended resources (you can extend the Kubernetes API):

/apis/myextension.mycompany.io/v1/

Please Note! Another post is planned to explain the Kubernetes API structure.

---

Roles and ClusterRoles

Introduction

In Kubernetes, there are two RBAC resources called Role and ClusterRole.

• Role is used to set permissions within a specific namespace. You need to specify the namespace when creating a Role.
• ClusterRole is used to set cluster-wide permissions and does not belong to a specific namespace.

---

From Kubernetes docs:

❗If you want to define a role within a namespace, use a Role; if you want to define a role that is cluster-wide, use a ClusterRole.

---

Diagram - Role and ClusterRole scopes

The permissions you define are additive, which means you only specify what is allowed and do not specify any deny permissions.

This is true for both Role and ClusterRole types. However, when a ClusterRole is linked to a Service Account through a RoleBinding, the ClusterRole permissions only apply to the namespace in which the RoleBinding was created.

Role

When you define a Role, you need to specify which namespace it belongs to. 

Here is an example of a Role manifest coming from the Kubernetes official doc:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

Here we have a pod-reader a role that grants read access to the entity. It grants read permissions for the pod resources. 

For example, the user can read the pod resources by having the pod-reader role. 

ClusterRole

ClusterRole can be defined in a similar way as a Role that is assigned to a namespace.

The significant difference is, that it applies to all resources in all namespaces or to a resource that is cluster specific such as a node. 

In the following example, we're defining a ClusterRole named secret-reader that grants access to all secrets in all namespaces within the cluster:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # "namespace" omitted since ClusterRoles are not namespaced
  name: secret-reader
rules:
- apiGroups: [""]
  #
  # at the HTTP level, the name of the resource for accessing Secret
  # objects is "secrets"
  resources: ["secrets"]
  verbs: ["get", "watch", "list"]

RoleBinding and ClusterRoleBinding

RoleBinding and ClusterRoleBinding are objects that are used to connect subjects (users, groups, or service accounts) to roles or cluster roles, respectively. They allow you to grant permissions to subjects, defining what actions they can take in the cluster

The following diagram illustrates how the entities are connected to the RoleBinding and ClusterRoleBinding which in turn are connected to a Role or ClusterRole.

RoleBinding example

RoleBindings belong to a particular namespace, just like the Role object. Inside the RoleBinding, you must also reference a Role or ClusterRole.

For example, let's create a RoleBinding called "read-pods" that connects a subject (a user called Tom) to a Role called "pod-reader" that we created in the default namespace:

apiVersion: rbac.authorization.k8s.io/v1
# This role binding allows "tom" to read pods in the "default" namespace.
# You need to already have a Role named "pod-reader" in that namespace.
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
# You can specify more than one "subject"
- kind: User
  name: tom # "name" is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  # "roleRef" specifies the binding to a Role / ClusterRole
  kind: Role #this must be Role or ClusterRole
  name: pod-reader # this must match the name of the Role or ClusterRole you wish to bind to
  apiGroup: rbac.authorization.k8s.io

ClusterRoleBinding example

ClusterRoleBinding is used for cluster-wide access, where you define a subject tied to a ClusterRole. 

The group "manager" can read all secrets across all namespaces within the cluster. The following diagram shows how it could be configured:

The following manifest example comes from the official Kubernetes doc and applies the ClusterRoleBinding explained above:

apiVersion: rbac.authorization.k8s.io/v1
# This cluster role binding allows anyone in the "manager" group to read secrets in any namespace.
kind: ClusterRoleBinding
metadata:
  name: read-secrets-global
subjects:
- kind: Group
  name: manager # Name is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

RoleBinding can refer to a ClusterRole

In a RoleBinding object, you can define a ClusterRole in the roleRef and also define a namespace that will give the permissions to objects only in a specific namespace.

For example, we have a Rolebinding in the "development" namespace. The user "dave" is the subject of the RoleBinding.

In the roleRef, we define a ClusterRole named secret-reader. One might think that the user dave has permission to read all secrets within the cluster since we have defined a ClusterRole in the roleRef. However, this is not the case. The permissions granted to the user dave are limited to the "development" namespace, as specified in the RoleBinding manifest. We will see how this can be defined in a manifest.

Diagram - Relationship between RoleBinding and ClusterRole

In a RoleBinding manifest, we can refer to a ClusterRole instead of a Role as we have seen earlier:

roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

The following RoleBinding manifest grants the user "dave" to only read secrets in the namespace named "development".

apiVersion: rbac.authorization.k8s.io/v1
# This role binding allows "dave" to read secrets in the "development" namespace.
# You need to already have a ClusterRole named "secret-reader".
kind: RoleBinding
metadata:
  name: read-secrets
  #
  # The namespace of the RoleBinding determines where the permissions are granted.
  # This only grants permissions within the "development" namespace.
  namespace: development
subjects:
- kind: User
  name: dave # Name is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io

---

❗You cannot change the roleRef field after you have defined and applied the binding, regardless of whether it is a RoleBinding or ClusterRoleBinding. If you need to change the roleRef, you must re-create the binding.

---

The following ClusterRole grants permissions to read secrets in the namespace 'development'. The RoleBinding that we have written above refers to this ClusterRole.

The subject we have defined (the user "dave") will only be able to read secrets in the namespace "development" after we apply the ClusterRoleBinding and the ClusterRole manifests.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # "namespace" omitted since ClusterRoles are not namespaced
  name: secret-reader
rules:
- apiGroups: [""]
  #
  # at the HTTP level, the name of the resource for accessing Secret
  # objects is "secrets"
  resources: ["secrets"]
  verbs: ["get", "watch", "list"]

Why refer to a ClusterRole from a RoleBinding

One simple reason is that you can define your roles in a few ClusterRole objects. Then, for each namespace you create, you can create a RoleBinding that points to a specific ClusterRole, which can save you the effort of creating separate Role objects for each namespace and repeating the same permissions.

Admission Controllers

What are Admission Controllers In Kubernetes

In this article, we will introduce Kubernetes admission controllers, a powerful native feature that allows us to control what runs on the cluster. I cannot emphasize enough how powerful this feature is and how it should be considered by anyone running a fairly large cluster(s).

Admission controllers can help secure the cluster and application deployments, enforcing security and compliance, but not in a way that requires manual effort. Instead, the level of control required can be automated as part of the user's submission of requests to the cluster.

 

The admission controller can monitor resource usage, such as CPU and memory, to ensure that users do not request too many resources. It can also check that users do not use non-approved image registries and enforce the pod security policy. It can also check that users do not use non-approved image registries, and enforce the pod security policy to help ensure the security and compliance of the cluster and the applications running on it

I have been in discussions with people who claim that they can build this kind of validation into their CI/CD pipeline, for example by running GitHub actions to validate Kubernetes manifest files. However, the problem with this approach is that users may apply Kubernetes manifest files manually, bypassing the validation process in the CI/CD pipeline. My recommendation is to have both types of validation in place to ensure that nothing slips through.

---

❗There is a great tool called contest that can be used to write tests against Kubernetes manifests

---

You can think of the admission controller as a guard that validates the configuration before it is persisted in the etcd store, but only after the request has been authorized. Since admission controllers are part of Kubernetes, they can validate requests regardless of how the Kubernetes manifest files are applied, whether through automation or manually. Admission controllers will always take action when a request is made to the kube-apiserver.

RBAC with Admission Controllers

In this article, we have learned how Kubernetes RBAC works and to some degree, how to set it up. We have learned that we can create Roles and ClusterRoles, how to create bindings and grant permissions that allow different operations on Kubernetes objects, such as read, update, delete, and create.

However, RBAC is not fine-grained enough for some use cases. For example, you may want to grant a subject permission to create a deployment, but only allow them to create a certain number of replicas. Or you may want to grant a subject permission to create a service, but not allow them to create a certain type of service, such as a nodeport service.

To achieve this level of fine-grained control, you can use admission controllers in conjunction with RBAC. Admission controllers can perform the following operations when interacting with the Kubernetes API:

• create (CREATE)
• delete (DELETE)
• modify objects (UPDATE)
• connect to proxy. (CONNECT)

Admission controllers are compiled binaries that are part of the kube-apiserver and can only be enabled by cluster administrators.

The following diagram shows different phases when the admission controller manages and validates a request:

Let's break it up a little bit so we understand the whole process.

There are different admission controllers that you can enable on the cluster, but only cluster administrators can perform this action. 

There are two phases when processing a request, which are based on two types of admission controllers:

 

• Mutating admission controllers: modify a request.
• Validating admission controllers:  run in the validation phase and decide whether to accept or reject a request.

These two types of admission controllers can be used separately or in combination to control access to the cluster and the resources within it.

Validating Admission Controller

The validating admission controller checks requests made to the Kubernetes API server and validates them against predefined rules. If the request is invalid, the admission controller will reject it and send an HTTP status message back to the client with details about the error. This message can include custom text specified by the admission controller. The request can be validated against multiple admission policy controls.

The admission controller should provide a detailed reason for any failures. It is recommended to read the error messages to understand why a request is failing and will not be applied.

Mutating Admission Controller

The difference between the validating admission controller and the mutating admission controller is that the latter allows changing an object, whereas the former only allows rejection.

Mutating admission controllers can:

• Change an object
• Run both mutation (change) and validation of the request in one step

It is recommended to separate validating and mutating admission controllers because the validating admission controller could cover the validation of mutations performed by other admission controllers. You can have multiple admission controllers.

Dynamic Admission Control

In addition to the built-in admission controller plugins, you can also develop your own plugins as extensions. These custom plugins will run as webhooks, which we will explain soon.

For your admission controller, you can use MutatingAdmissionWebhook and ValidatingAdmissionWebhook.

Before implementing dynamic admission controllers, you must enable them using the --enable-admission-plugins flag.

--enable-admission-plugins=...,MutatingAdmissionWebhook,\
 ValidatingAdmissionWebhook

How To Enable Admission Controllers

Only cluster administrators can enable admission controllers.

To enable admission controllers, use the --enable-admission-plugins flag followed by a comma-delimited list of admission control plugins.

For example, to enable some built-in plugins as well as MutatingAdmissionWebhook and ValidatingAdmissionWebhook, you can use the following command:

--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,Priority,ResourceQuota,PodSecurityPolicy

---

❗If you run AWS EKS as I do, as of writing I run Kubernetes version 1.22, the following admission controllers are enabled by default:

NamespaceLifecycle, For example, LimitRanger validates that none of the objects in a Kubernetes deployment violate the constraints specified in the LimitRange object of a Namespace. It can also mutate the request to assign default resource limits and requests to Pods that don’t specify any., ServiceAccount, DefaultStorageClass, ResourceQuota, DefaultTolerationSeconds, NodeRestriction, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, PodSecurityPolicy, TaintNodesByCondition, StorageObjectInUseProtection, PersistentVolumeClaimResize, ExtendedResourceToleration, CertificateApproval, PodPriority, CertificateSigning, CertificateSubjectRestriction, RuntimeClass, DefaultIngressClass

Full list can be found here

---

How To Disable Admission Controllers

You can disable admission controller plugins in a similar way to how you enable them, using the --disable-admission-plugins flag.

Example:

kube-apiserver --disable-admission-plugins=PodNodeSelector,AlwaysDeny

Check Enabled Admission Controllers 

With the following command, you can check which admission controllers have been enabled in your cluster:

kube-apiserver -h | grep enable-admission-plugins

Use Cases for Admission Controllers

One use case for admission controllers is to ensure the secure execution of workloads on the cluster. There are several security concerns that you may need to address, such as:

• Using vulnerable images
• Using non-approved base images or registries
• Running containers as the root user
• Running privileged pods
• Not setting resource limits for containers
• Improperly configuring RBAC

With admission controllers, you can control what can be run on your cluster. For example, you can use admission controllers to check security constraints, such as only allowing approved images from approved registries, and disallowing the use of containers as the root user. You can also use admission controllers to enforce a policy requiring limits to be set on containers.

Dynamic Admission Controllers

So far, we have seen how to enable pre-configured admission controllers, but we can also build our own admission controllers to meet specific business requirements. Dynamic admission control allows us to inject custom logic into the admission control feature in Kubernetes.

---

From Official Kubernetes doc:

Admission webhooks are HTTP callbacks that receive admission requests and do something with them. You can define two types of admission webhooks, validating admission webhook and mutating admission webhook. Mutating admission webhooks are invoked first, and can modify objects sent to the API server to enforce custom defaults. After all object modifications are complete, and after the incoming object is validated by the API server, validating admission webhooks are invoked and can reject requests to enforce custom policies.

---

In the following section, we will look at a use case showing how that can be accomplished. 

Use Case Example - Allow Certain Registry

The requirement is to only allow images to be pulled from a local, company-hosted registry due to security compliance imposed by the cybersec team.

To meet this requirement, we want to ensure that images are only pulled from a secured local registry: <your-company.io>.

 

To meet this requirement, we will need to build a validating admission controller and register it with the Kubernetes API server.

There is a specific object called AdmissionReview that is responsible for sending requests to our application and receiving responses from it (the admission controller).

We can build either a mutating admission controller or a validating admission controller (or both).

To solve this challenge, we will define a validating admission controller that will approve or deny requests based on the registry specified.

Here are the steps:

1. A deployment or pod manifest is applied to the Kubernetes cluster.
2. A registered validating admission controller picks up the request from step 1 and sends it to an app that will process the request.
3. Since the pod manifest specifies the local registry in the image name, our application sends true as the response to the validating admission controller.
4. The client sees a successful status (e.g. "Running") or an error like: "Error from server: error when creating "pod-spec.yaml": admission webhook "com.name.whitelist-registry" denied the request: the image comes from an untrusted registry!

 

In the case of a custom admission controller, you need to build and register the validating admission controller with the Kubernetes API server (which will be covered in a future course).

---

Example - Here is an example of a Validating  Admission Controller that is running in the default namespace:

apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
  name: angry-check
webhooks:
  - name: angry-check
    clientConfig:
      service:
        name: angry-check
        namespace: default
        path: "/check"
      caBundle: "${CA_BUNDLE}"
    rules:
      - operations: ["CREATE"]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]

In the clientConfig part, we have specified the location of the service.

The /path is the service endpoint that listens for incoming requests.

The caBundle is a strong recommendation for good security in the cluster. In our case, we only allow HTTPS traffic to our validating controller app.

In the rules part, we define which requests should be forwarded to our app.

In our particular case, we have defined that only CREATE requests for pod resources should be forwarded to our controller.

The actual app, angry-check, will be deployed to the default namespace.

---

In the Kubernetes API, there is an object of the type AdmissionReview that is used for both sending requests and receiving responses.

For requests, there is a request field that holds various information. The most important piece of information we are interested in is the kind of object, such as Pod or Deployment.

Example: AdmissionReview (Validating Admission Controller) - Request

{
  "apiVersion": "admission.k8s.io/v1",
  "kind": "AdmissionReview",
  "request": {
    # this uid gets randomly generated to identify a unique admission call
    "uid": <random uid>,
    ...
    "object": {"apiVersion":"v1","kind":"Pod"},
    ...
    }
}  

Example - AdmissionReview - Validating Admission Controller - Response

{
  "apiVersion": "admission.k8s.io/v1",
  "kind": "AdmissionReview",
  "response": {
    "uid": "<request.uid>",
    "allowed": true,
    "status": {
      "code": <optional http status>,
      "message": "optional message"
    }
  }
}

In the case of our own admission controller, angry-check, it would need to create a response like:

// Response that will be sent to Kubernetes API
ar := v1beta1.AdmissionReview{
  Response: &v1beta1.AdmissionResponse{
    Allowed: true,
    Result: &metav1.Status{
      Message: "Approved to be applied to the cluster!",
    },
  },
}
resp, err := json.Marshal(ar)

If we would build a Mutating Admission Controller, a JSONPatch type object will be part of the response in the AdmissionReview object and the original request would be modified. 

Example - Mutating Admission Controller - response

{
  "apiVersion": "admission.k8s.io/v1",
  "kind": "AdmissionReview",
  "response": {
    "uid": "<request.uid>",
    "allowed": true,
    "status": {
      "code": <optional http status>,
      "message": "optional message"
    },
     "patchType": "JSONPatch",
    "patch": <base64 encoded JSON patch>
  }
}

---

❗One caveat with mutating admission controllers is that the object created by the user may not be the same as the one that is applied to the cluster and persisted in the etcd store. For example, the end result may not be what the user expects.

---

As we have seen, admission controllers are useful for maintaining compliance and enforcing policies within an organization. They can be used in both staging and production environments, and the webhooks and interfaces of the admission controllers can be easily monitored.

TL;DR

In this article, you have been introduced to the following:

• Role-based access control (RBAC) 
  
  • Role
  • ClusterRole
  • Bindings
    • RoleBinding
    • ClusterRoleBinding
• How the Authentication concept works in Kubernetes
• How the Authorization concept works in Kubernetes
• Admission Controllers
  • The purpose of Admission Controllers
  • How admission controllers operate
  • Webhooks
  • The request and response object - AdmissionReview
  • Different types of admission controllers
    
    • Mutating admission controller
    • Validating admission controller

References

Kubernetes official docs:

• access-authn-authz
• admission-controllers

 

About the Author

Aleksandro Matejic, Cloud Architect, began working in IT Industry over 20y ago as a technical consultant at Lindahl Rothoff in southern Sweden. Since then, he has worked in various companies and industries like Atea, Baxter, and IKEA having various architect roles. In his spare time, Aleksandro is developing and running devoriales.com, a blog and learning platform launched in 2022. In addition, he likes to read and write technical articles about software development and DevOps methods and tools. You can contact Aleksandro by paying a visit to his LinkedIn Profile.