Evolution of security strategies in K8s environments

In the latest versions of K8s there has been an evolution regarding the definition of security strategies at the level of access policies to the cluster by users and developers. The security contexts (securityContext) allow you to define the configurations at the level of access control and privileges for a pod or container in a simple way using keywords in the configuration files.
To facilitate the implementation of these security strategies throughout the cluster, new strategies have emerged such as the Pod Security Policy (PSP) where the cluster administrator is in charge of defining these policies at the cluster level with the aim that developers can follow these policies.
Other interesting projects include Open Policy Agent (OPA) as the main cloud-native authorization policy agent for creating policies and managing user permissions for access to applications.
The objective of this talk is to present the evolution that has occurred in security strategies and how we could use them together, as well as analyze their behavior in accessing resources. Among the points to be discussed we can highlight:
*Introduction to security strategies in K8s environments
*Pod Security Admission(PSA) vs Open Policy Agent (OPA)
*Combination of different security strategies together
*Access to resources in privileged and non-privileged mode

Transcript

1. www.containerdays.io #CDS23 Evolution of security strategies in K8s environments
2. www.containerdays.io #CDS23 Agenda • Introduction to security strategies in K8s environments • Pod Security Admission(PSA) vs Open Policy Agent (OPA) • Combination of different security strategies together • Access to resources in privileged and non-privileged mode
3. www.containerdays.io #CDS23 Introduction to security strategies in K8s environments • Cluster Hardening: Implement best practices for securing the Kubernetes cluster itself, including securing access to the API server, enabling RBAC (Role-Based Access Control), and using network policies to control communication between pods. • Pod Security Policies (PSP): Enforce security policies that define what a pod can and cannot do, including limiting privilege levels, host access, and running as non-root users.
4. www.containerdays.io #CDS23 Introduction to security strategies in K8s environments • Secrets Management: Use Kubernetes Secrets to store sensitive information securely, such as API keys, passwords, or certificates. • Role-Based Access Control (RBAC): Define fine-grained access controls for users and service accounts to limit the scope of actions they can perform within the cluster.
5. www.containerdays.io #CDS23 Introduction to security strategies in K8s environments • Limit Resource Consumption: Set resource quotas to limit the amount of CPU, memory, and other resources that can be consumed by pods, preventing resource exhaustion and potential denial-of-service attacks. • Pod Security Context: Use pod security context to define security settings at the pod level, such as user and group IDs, SELinux, and file system permissions.
6. www.containerdays.io #CDS23 Introduction to security strategies in K8s environments • PodSecurityPolicy has been deprecated from Kubernetes 1.21.
7. www.containerdays.io #CDS23 Introduction to security strategies in K8s environments • PodSecurityContext, the Kubernetes tool which allows users to specify security contexts and how the pod will be execute.
8. www.containerdays.io #CDS23 Introduction to security strategies in K8s environments Security Contexts RBAC (Role-Based Access Control) Resource scope Pods Pods, Nodes, cluster Actions Predefined capabilities RBAC policies Extensibility Via integrations with external frameworks, including SELinux and AppArmor Can’t use external tools to define policies.
9. www.containerdays.io #CDS23 Security Context spec: securityContext: runAsUser: 1000 fsGroup: 2000 allowPrivilegeEscalation: false
10. www.containerdays.io #CDS23 Security Context apiVersion: v1 kind: Pod metadata: name: scd-3 spec: containers: - name: scd-3 image: nginx securityContext: capabilities: add: ["NET_ADMIN","SYS_TIME"]
11. www.containerdays.io #CDS23 KubeAudit https://github.com/Shopify/kubeaudit
12. www.containerdays.io #CDS23 • New form of admission control is created with the understanding that Kubernetes users are probably going to seek external authorization. • It can be deactivated partially or entirely to coexist with external admission controllers like OPA. • KEP-2579: Pod Security Admission Control • https://github.com/kubernetes/enhancements/blob/mast er/keps/sig-auth/2579-psp-replacement/README.md Pod Security Admission(PSA)
13. www.containerdays.io #CDS23 Pod Security Admission(PSA) • Setting Default Security Constraints • Fine-Grained Control over Policy Definition • Sub-Namespace Policy Granularity
14. www.containerdays.io #CDS23 Pod Security Admission(PSA)
15. www.containerdays.io #CDS23 Pod Security Admission(PSA) • Pod Security admission places requirements on a Pod's Security Context and other related fields according to the three levels defined by the Pod Security Standards: privileged, baseline, and restricted. • spec.containers[*].ports • spec.volumes[*].hostPath • spec.securityContext • spec.containers[*].securityContext
16. www.containerdays.io #CDS23 Pod Security Admission(PSA) kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 featureGates: PodSecurity: true nodes: - role: control-plane - role: worker
17. www.containerdays.io #CDS23 Pod Security Admission(PSA) Mode Description enforce Policy violations will cause the pod to be rejected. audit Policy violations will trigger the addition of an audit annotation to the event recorded in the audit log, but are otherwise allowed. warn Policy violations will trigger a user-facing warning, but are otherwise allowed.
18. www.containerdays.io #CDS23 Pod Security Admission(PSA)
19. www.containerdays.io #CDS23 Pod Security Admission(PSA) kubectl label --overwrite ns test-ns \ pod-security.kubernetes.io/warn=baseline \ pod-security.kubernetes.io/warn-version=v1.22
20. www.containerdays.io #CDS23 Pod Security Admission(PSA) • It is consistent in deploying the security levels on namespaces by labels which helps with testing, troubleshooting and maintaining. • Ability to perform dry runs using --dry-run=server before applying pod-security on namespace labels • Provides validations for compliance with policies and will not change the pods to enforce compliance.
21. www.containerdays.io #CDS23 Pod Security Admission(PSA) $ kubectl label --dry-run=server --overwrite ns --all \ pod-security.kubernetes.io/enforce=baseline Warning: kuard: privileged namespace/default labeled namespace/kube-node-lease labeled namespace/kube-public labeled Warning: kube-proxy-vxjwb: host namespaces, hostPath volumes, privileged Warning: kube-proxy-zxqzz: host namespaces, hostPath volumes, privileged Warning: kube-apiserver-kind-control-plane: host namespaces, hostPath volumes Warning: etcd-kind-control-plane: host namespaces, hostPath volumes Warning: kube-controller-manager-kind-control-plane: host namespaces, hostPath volumes Warning: kindnet-cl5ln: non-default capabilities, host namespaces, hostPath volumes Warning: kube-scheduler-kind-control-plane: host namespaces, hostPath volumes Warning: kindnet-6ptww: non-default capabilities, host namespaces, hostPath volumes namespace/kube-system labeled namespace/local-path-storage labeled
22. www.containerdays.io #CDS23 Pod Security Admission(PSA) apiVersion: v1 kind: Namespace metadata: name: test-ns labels: pod-security.kubernetes.io/enforce: baseline pod-security.kubernetes.io/audit: restricted pod-security.kubernetes.io/warn: restricted
23. www.containerdays.io #CDS23 Pod Security Admission(PSA) apiVersion: v1 kind: Pod metadata: name: nginx spec: containers: - image: nginx name: nginx ports: - containerPort: 80
24. www.containerdays.io #CDS23 Pod Security Admission(PSA) $ kubectl apply -f pod.yaml Warning: would violate "latest" version of "restricted" PodSecurity profile: allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") pod/nginx created $ kubectl get pods NAME READY STATUS RESTARTS AGE nginx 1/1 Running 0 6s
25. www.containerdays.io #CDS23 Pod Security Admission(PSA) {"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"808ca159-914c-43fa- b4c8-dee5cb2fc440","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/default/p ods?fieldManager=kubectl-create","verb":"create","user":{"username":"kubernetes-admin","grou ps":["system:masters","system:authenticated"]},"sourceIPs":["172.18.0.1"],"userAgent":"kubectl/ v1.22.0 (darwin/amd64) kubernetes/c2b5237","objectRef":{"resource":"pods","namespace":"default","name":"nginx","api Version":"v1"},"responseStatus":{"metadata":{},"code":201},"requestReceivedTimestamp":"2023 -08-21T03:30:26.605589Z","stageTimestamp":"2023-08-21T03:30:26.627123Z","annotations":{" authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"","pod-security.kubernetes .io/audit":"allowPrivilegeEscalation != false (container \"nginx\" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container \"nginx\" must set securityContext.capabilities.drop=[\"ALL\"]), runAsNonRoot != true (pod or container \"nginx\" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container \"nginx\" must set securityContext.seccompProfile.type to \"RuntimeDefault\" or \"Localhost\")"}}
26. www.containerdays.io #CDS23 Pod Security Admission(PSA) apiVersion: apiserver.config.k8s.io/v1 kind: AdmissionConfiguration plugins: - name: PodSecurity configuration: apiVersion: pod-security.admission.config.k8s.io/v1alpha1 kind: PodSecurityConfiguration defaults: enforce: "baseline" enforce-version: "latest" audit: "restricted" audit-version: "latest" warn: "restricted" warn-version: "latest" exemptions: usernames: [] runtimeClassNames: [] namespaces: [kube-system]
27. www.containerdays.io #CDS23 Policy-as-code (PAC) solutions
28. www.containerdays.io #CDS23 • Policy agent for cloud-native authorization • It provides a means of standardizing policy definition and management throughout the cloud-native technology stack. • When combined with Kubernetes, OPA has the capability to enforce guardrails upon an entire system, requiring users’ permissions to match policy at all times.
29. www.containerdays.io #CDS23
30. www.containerdays.io #CDS23 • Require specific labels on all resources. • Require container images from the corporate image registry. • Require all Pods specify resource requests and limits. • Prevent conflicting Ingress objects from being created.
31. www.containerdays.io #CDS23
32. www.containerdays.io #CDS23
33. www.containerdays.io #CDS23
34. www.containerdays.io #CDS23 Pod Security Admission(PSA) vs Open Policy Agent(OPA) Pod Security Admission (PSA) Open Policy Agent (OPA) Simplicity Flexibility Native Integration Customization Performance External Control Limited Attack Surface Compliance
35. www.containerdays.io #CDS23 Pod Security Admission(PSA) vs Open Policy Agent(OPA) • Which users can access which resources? • Which subnets egress traffic is allowed to? • Which clusters a workload must be deployed to? • Which registries images can be downloaded from? • Which capabilities a container can execute with? • Which times of day the system can be accessed at?
36. www.containerdays.io #CDS23 Combination of different security strategies • RBAC (Role-Based Access Control) • PodSecurity Admission Controllers • Network Policies • Secrets Management • Security Contexts • Runtime Security
37. www.containerdays.io #CDS23 Access to resources in privileged and non-privileged mode • Privileged Mode • Non-Privileged Mode*
38. www.containerdays.io #CDS23 Access to resources in privileged and non-privileged mode • Privileged Mode
39. www.containerdays.io #CDS23 Access to resources in privileged and non-privileged mode • Non-Privileged Mode
40. www.containerdays.io #CDS23 Conclusions • Security • Functionality • Isolation • Attack Surface
41. www.containerdays.io #CDS23 ¡Thank you! @jmortegac https://www.linkedin.com/in/jmortega1 https://jmortega.github.io https://josemanuelortegablog.com