Kubernetes 101 for Penetration Testers - null Mumbai
This hands-on (demo) driven talk intends to get application / network security analysts get started with Kubernetes Cluster Penetration Testing. The slides are meant to be used for hands-on learning using locally setup cluster.
the session • DO NOT do hands-on during the session - 1 hour is too less • Use the slides as a reference to try out hands-on after the session ◦ Slides are built specifically as a reference material • Use additional reference material provided for further learning • Ping me for doubts & questions (@abh1sek on Twitter)
willing to learn new things • Familiar with Linux err… I mean GNU/Linux • Familiar with network or application security • Familiar with basic vulnerability and exploit terminology • Familiar with vulnerability assessment & penetration testing
the (compiled) code for an application along with the dependencies it needs at run time. Each container that you run is repeatable; the standardization from having dependencies included means that you get the same behavior wherever you run it. Think of container as “Code + Config + Runtime” packaged in an archive stored locally or in a Git like remote repository, called Container Registry
for managing containerized workloads and services, that facilitates both declarative configuration and automation. A container orchestrator really - Refer to Illustrated Children’s Guide to Kubernetes :) https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/
-n 3 \ --enable-default-cni=false --network-plugin=cni kubectl cluster-info kubectl get nodes -o wide ❗ Multi-node clusters are currently experimental and might exhibit unintended behavior. To track progress on multi-node clusters, see https://github.com/kubernetes/minikube/issues/7538. Try out the online playground at Katacoda https://www.katacoda.com/courses/kubernetes/playground
a set of Pods as a network service. There are multiple service types such as ClusterIP, NodePort, LoadBalancer kubectl expose pod nginx-1 \ --port=8888 --target-port=80 --type=NodePort export NODE_PORT=$(kubectl get svc nginx-1 -o jsonpath='{.spec.ports[0].nodePort}') curl http://<NodeIP>:$NODE_PORT/ NodePort may be risky to use
https://github.com/kubernetes/community/tree/master/wg-security-audit How can they attack? 03 • Leverage configuration weaknesses • Exploit vulnerabilities • Exploit trust across components • Lack of appropriate AuthZ controls • Lack of security hardening of the cluster What can they attack? 02 • Cluster state storage (etcd) • Secrets • Volumes (Data Breach) • Container Image (Private Repository) • Compute Resources (Example: Crypto Mining) Who are the attackers? 01 • External (From internet) • Internal (Attacker in a Pod) • Developer (User with some access in the cluster) • Malicious Administrator • End User
2. Vulnerability Testing a. You must do a conventional VA/PT for the infrastructure (OS) running Master and Node components in additional to Kubernete specific testing 3. Exploitation a. Privilege Escalation b. Lateral Movement 4. Persistence
https://$API_SERVER_ENDPOINT/api/v1/namespaces curl -sk https://$API_SERVER_ENDPOINT/api/v1/namespaces/default/pods kubectl auth can-i list namespaces kubectl auth can-i list pods kubectl auth can-i create pod Testing with unprivileged credential like Pod default service account
in Minikube cluster (as it should be) docker run -it --rm \ --network host \ --env ETCDCTL_API=3 \ --env ALLOW_NONE_AUTHENTICATION=yes \ bitnami/etcd:latest -- \ etcdctl --endpoints https://$ETCD_IP:2379 get / Should fail as client-cert auth is enabled by default, but you may be lucky :)
\ --image appsecco/k8s-security-tools \ -- bash Simulating an attacker in a Pod with required security tools printenv ifconfig host -v kubernetes.default kubectl auth can-i create pod ls -al /var/run/secrets/kubernetes.io/serviceaccount/
Pods on a node can communicate with all pods on all nodes without NAT using the Pod Network i.e. anyone can talk to anyone by default Service Discovery through DNS assigns unique IP address to services in a dedicated Service Network CIDR All of this is facilitated by the CNI Plugin ifconfig ping kubernetes
\ kube-hunter As external attacker to scan Master IP(s) for known issues kube-hunter --pod --cidr $POD_CIDR As internal attacker from attacker tools container https://github.com/aquasecurity/kube-hunter
and gain access to other Pods (and resources) ◦ Finally gain access to the cluster as cluster-admin • How? ◦ Known vulnerable components in the control plane ◦ Open or vulnerable service in Pod/Service network ▪ Example: Helm Tiller Privilege Escalation ◦ Abusing privilege ▪ Example: Privilege Escalation Abusing hostPath Volume Mount
a file or directory from the host node's filesystem into your Pod. This is not something that most Pods will need, but it offers a powerful escape hatch for some applications. https://blog.appsecco.com/kubernetes-names pace-breakout-using-insecure-host-path-volu me-part-1-b382f2a6e216
of Helm is running inside the cluster without authentication (default in Helm 2, removed in Helm 3) 2. We connect to tiller on predictable service name, namespace and port a. Alternatively, we can scan Service CIDR and discover tiller as well 3. We connect to tiller and ask it to install a chart that binds cluster-admin like privilege to namespace default service account 4. Our Pod, or for that matter, any Pod in running in default namespace now owns the cluster https://engineering.bitnami.com/articles/helm-security.html https://v2.helm.sh/docs/securing_installation/
Work in Progress ◦ https://owasp.org/www-project-kubernetes-security-testing-guide/ ◦ https://github.com/owasp/kstg • Aims to be the reference guide for Kubernetes Cluster Penetration Testing • Me (@abh1sek) and Madhu Akula (@madhuakula) working on it for now, looking for your contribution :)
and open source training material including hands-on lab for Docker & Kubernetes security for you to try out. https://github.com/appsecco/atta cking-and-auditing-docker-contai ners-and-kubernetes-clusters
• Hacking and Hardening Kubernetes Clusters by Example [I] - Brad Geesaman, Symantec • Advanced Persistent Threats: The Future of Kubernetes Attacks • Kubernetes From an Attacker's Perspective — OWASP Bay Area Meetup • CIS Benchmark for Kubernetes • aquasecurity/kube-hunter: Hunt for security weaknesses in Kubernetes clusters • aquasecurity/kube-bench: Checks whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark • kelseyhightower/kubernetes-the-hard-way: Bootstrap Kubernetes the hard way on Google Cloud Platform. No scripts.