The Complexity of Container and Kubernetes Security
Organizations worldwide are rapidly adopting containerization and Kubernetes for deploying modern cloud-native applications. The scalability, efficiency, and automation capabilities of Kubernetes have made it the de facto orchestration platform for containerized workloads. However, this shift to container-based infrastructure introduces significant security challenges that many organizations are still learning to address. Traditional penetration testing methodologies don't directly translate to containerized environments, and many security teams lack the specialized knowledge needed to properly test container and Kubernetes deployments for vulnerabilities.
For more details, see our guides on cloud penetration testing.Containers and Kubernetes introduce new attack surfaces that don't exist in traditional infrastructure environments. The ephemeral nature of containers means systems constantly spin up and spin down. Container images often contain embedded secrets and credentials. Kubernetes clusters expose APIs that can be exploited if not properly secured. Container runtimes like Docker provide interfaces that can be leveraged for privilege escalation. Network policies in Kubernetes may be incorrectly configured, allowing unauthorized communication between pods. These vulnerabilities require specialized testing approaches that penetration testers must understand to effectively assess container security.
The container and Kubernetes ecosystem continues to evolve rapidly. New vulnerability classes emerge regularly. Best practices evolve as the community learns from security incidents. Organizations that don't regularly test their container deployments for security vulnerabilities risk deploying applications that attackers can compromise through well-known attack techniques. Penetration testing specifically focused on containers and Kubernetes has become essential for organizations operating cloud-native infrastructure at scale.
Understanding Container Escape Vulnerabilities
Container escape vulnerabilities represent one of the most critical security risks in containerized environments. A container escape occurs when an attacker breaks out of the container isolation boundaries and gains access to the host operating system or other containers running on the same host. Once an attacker escapes a container, they can potentially access sensitive data from other containers, compromise the Kubernetes node itself, and move laterally throughout the cluster.
Several classes of vulnerabilities enable container escapes. Kernel vulnerabilities in the host operating system can be exploited from within a container to break out of container isolation. While containers use Linux namespaces and cgroups to provide isolation, these isolation mechanisms rely on kernel security. A vulnerability in the kernel can undermine the entire isolation model. Penetration testers should research known kernel vulnerabilities and test whether containers can exploit them to escape isolation.
Container runtime vulnerabilities represent another significant risk. Docker and other container runtimes provide interfaces and functionality that could be exploited. For example, if the Docker daemon socket is accessible from within a container, an attacker can use it to launch privileged containers or access container images. Similarly, runc vulnerabilities or containerd vulnerabilities could allow escape. Penetration testers should verify that container runtime interfaces are not unnecessarily exposed to containers and that the container runtime is running the latest patched version.
Misconfigured security options provide another path to container escape. Containers can be launched with various security flags that weaken isolation. A container launched with the privileged flag has direct access to all host devices. A container with CAP_SYS_ADMIN capability can mount filesystems and access kernel interfaces. A container running as root with the ability to write to the host filesystem can modify host binaries or configurations. Penetration testers should test which containers are running with dangerous capabilities or privileged mode, and whether those elevated privileges are necessary for the application to function.
Kubernetes RBAC Testing and Authorization Bypass
Kubernetes Role-Based Access Control (RBAC) provides the primary mechanism for controlling which users and service accounts can perform actions within the cluster. However, RBAC misconfigurations are common, and many organizations implement overly permissive RBAC policies that grant unnecessary access to service accounts and users. Penetration testers should thoroughly test RBAC configuration to identify overly permissive roles and service accounts.
RBAC testing begins with enumerating the permissions granted to different service accounts and users within the cluster. Kubernetes provides built-in service accounts for applications running in pods to authenticate to the Kubernetes API. Many applications unnecessarily have permission to list secrets, create pods, or modify cluster resources. A compromise of an application pod might grant attackers permission to perform privileged operations within the cluster. Penetration testers should identify which service accounts have excessive permissions and determine whether those permissions are actually required for the application to function.
The default RBAC configuration in many Kubernetes clusters is overly permissive. The default service account in the default namespace might have permissions to list or access secrets. Users or service accounts might be granted cluster-admin role unnecessarily. RBAC policies should follow the principle of least privilege, granting only the minimum permissions necessary for legitimate operations. Penetration testers should test whether they can escalate privileges by leveraging overly permissive RBAC configurations, such as using a compromised service account to create new admin accounts or modify cluster policies.
Kubernetes API access controls also require careful testing. The Kubernetes API server is the central control point for the cluster. If the API server is accessible without proper authentication, or if authentication mechanisms are weak, attackers can gain complete control of the cluster. Penetration testers should verify that the API server requires strong authentication, that API endpoint access is properly restricted, and that audit logging is enabled to track suspicious API access patterns.
Secrets Management and Data Exposure in Kubernetes
Kubernetes secrets provide a mechanism for storing sensitive data like database passwords, API keys, and TLS certificates. However, the default Kubernetes secret storage mechanism stores secrets as base64-encoded text in etcd, the cluster's distributed database. This encoding provides encoding rather than encryption, meaning anyone with access to etcd or to secret objects can read the sensitive data. Many organizations store secrets in Kubernetes secrets without understanding that these secrets are not encrypted by default.
Penetration testers should assess how secrets are stored and accessed within Kubernetes clusters. First, determine whether encryption at rest is enabled for etcd, ensuring that even if the database is compromised, secrets remain protected. Second, test access to secrets by attempting to read secrets from compromised pods or through the Kubernetes API. Many service accounts have permission to list or read secrets, creating excessive exposure of sensitive data. Third, verify that external secret management systems are used for truly sensitive data rather than relying on Kubernetes native secrets for API keys, database credentials, or cryptographic material that absolutely must remain secret.
Container images often contain embedded secrets in environment variables or configuration files. During penetration testing, testers should examine running containers to determine what secrets or sensitive information is accessible. Environment variables set at container creation time can be read by applications running in the container and could be exposed through application logs or debugging endpoints. Secrets should never be embedded in container images or passed as environment variables; they should be injected at runtime from secure external systems.
Kubernetes ServiceAccount tokens provide another source of sensitive data. Every pod automatically receives a ServiceAccount token that allows it to authenticate to the Kubernetes API. If an attacker compromises a pod, they obtain the ServiceAccount token and can use it to interact with the Kubernetes API with the permissions granted to that service account. Penetration testers should assess whether they can use compromised ServiceAccount tokens to perform unauthorized operations within the cluster.
Network Policy Testing and Lateral Movement
Kubernetes network policies provide the mechanism for controlling network traffic between pods. A well-designed network policy restricts communication between pods based on labels and namespaces, implementing network segmentation to limit lateral movement if a pod is compromised. However, many Kubernetes clusters don't implement network policies at all, allowing any pod to communicate with any other pod. Penetration testers should verify that network policies are properly implemented to restrict unnecessary communication.
Testing network policies involves attempting to establish network connections between pods that should be restricted by policies. If a network policy is supposed to prevent communication from a web application pod to a database pod, a penetration tester should attempt to establish that connection and verify that it's blocked. If the connection succeeds despite the policy, the policy is misconfigured and allows unnecessary lateral movement. Penetration testers should map out the actual network connectivity between pods and compare it to the intended network policies defined in the cluster.
Default-deny network policies represent a security best practice where all traffic is denied by default and specific communication paths are explicitly allowed. However, many organizations don't implement default-deny policies, instead using allow policies that permit specific traffic while leaving most communication permissible. Penetration testers should assess whether the cluster implements default-deny policies and recommend this as a hardening measure if not currently in place.
Network policies can also be bypassed in certain configurations. If a pod has host network access, it might bypass network policies designed to restrict its communication. If multiple namespaces exist in the cluster and aren't properly isolated, network policies in one namespace might not prevent pods in different namespaces from communicating. Penetration testers should thoroughly test whether configured network policies actually prevent the intended lateral movement and whether any bypass techniques allow unauthorized communication.
Container Image Supply Chain Security
Container images form the foundation of containerized applications, yet many organizations don't adequately assess the security of images they deploy. Images are typically pulled from container registries, modified slightly, and deployed to production. If the image registry is compromised, or if images are pulled from untrusted sources, the entire deployed application could be compromised. Penetration testers should assess the security of the container image supply chain.
Image scanning should be performed throughout the supply chain. When images are built, they should be scanned for known vulnerabilities before being pushed to the registry. When images are pulled from upstream registries, they should be verified as legitimate and scanned again. Before images are deployed to production, they should be scanned to ensure they haven't been modified since the last scan. Many organizations scan images once during build but don't re-scan before deployment, missing vulnerabilities discovered between the build and deployment phases.
Image signing and verification represent critical controls for ensuring image integrity. Container images can be digitally signed by the image publisher, and Kubernetes can be configured to only run images with valid signatures from trusted publishers. Penetration testers should assess whether image signing is implemented and whether Kubernetes is configured to enforce image signature verification. If signatures are not enforced, attackers could potentially deploy modified images that contain malicious code alongside legitimate applications.
Base images used as foundations for application images should be carefully selected and regularly updated. Many organizations use public base images from Docker Hub or other public registries without verifying the image's legitimacy or checking for vulnerabilities. Penetration testers should verify that base images come from trusted sources, that the images are scanned for vulnerabilities, and that base images are regularly updated to patch vulnerabilities. Additionally, the contents of container images should be examined for embedded secrets, credentials, or unnecessary tools that could facilitate post-compromise attacks.
Kubernetes API and Kubelet Security Assessment
The Kubernetes API server forms the central control plane for the entire cluster. All cluster management operations flow through the API server, making its security paramount. Penetration testers should thoroughly assess the security of the Kubernetes API by attempting unauthorized access, testing authentication mechanisms, and verifying that authorization controls are properly enforced. The API server should require strong authentication, restrict access based on user identity and roles, and log all API access for audit purposes.
The Kubelet runs on every node and manages containers on that node. The Kubelet provides an HTTP API that allows querying node and pod information and, if misconfigured, can allow executing commands within containers. Penetration testers should scan for exposed Kubelet APIs and attempt to access them without proper authentication. If Kubelet APIs are accessible, a penetration tester might be able to execute commands in containers, read pod logs, or access kubelet's mounted secrets.
Default Kubernetes service discovery exposes the Kubernetes API endpoint within the cluster. Any pod can query the Kubernetes API to discover information about other pods, services, and cluster configuration. If RBAC is not properly configured, a pod might be able to query the API and discover sensitive information about other workloads in the cluster. Penetration testers should assess what information is accessible through the Kubernetes API when authenticating as a compromised service account.
The etcd database stores all cluster state and configuration. If etcd is compromised, the entire cluster is compromised, as the attacker can read all secrets, modify cluster configuration, and gain complete control. Penetration testers should verify that etcd is not directly accessible from pods or external networks, that etcd requires strong authentication, and that etcd traffic is encrypted. Additionally, etcd backups should be stored securely and protected from unauthorized access.
Implementing Container and Kubernetes Penetration Testing
Organizations planning to conduct penetration testing of container and Kubernetes deployments should establish clear scope and objectives. Testing should cover container images, container runtime security, Kubernetes API and RBAC, network policies, secrets management, and cluster infrastructure. Testing should include both automated scanning for known vulnerabilities and manual testing to identify configuration issues and design flaws that automated tools might miss.
Penetration testing of Kubernetes requires specialized tools and expertise. Tools like kubesec scan Kubernetes manifests for security issues, kube-bench assesses Kubernetes cluster configuration against CIS benchmarks, and kube-hunter attempts to find vulnerabilities from within the cluster. However, tools are only part of the assessment. Manual testing by experienced security professionals is essential for understanding the implications of discovered issues and identifying complex attack chains that might not be apparent from automated output alone.
Testing should involve close coordination with the development and operations teams responsible for the Kubernetes cluster. Testers should understand how applications are deployed, what data is processed, and what the acceptable risk tolerances are for different components. This understanding allows testers to focus on the highest-risk areas and provide recommendations that balance security improvements with operational requirements.
After testing completes, organizations should prioritize remediation of identified findings. Critical issues affecting isolation, authentication, or data protection should be remediated immediately. High-priority issues should be addressed within days or weeks. Medium and low-priority issues should be tracked and remediated as part of ongoing hardening efforts. Organizations should also consider implementing continuous scanning of container images and Kubernetes configurations to identify new vulnerabilities as they emerge.
Integrating Container Security into DevSecOps
Container and Kubernetes security shouldn't be addressed only through penetration testing. Organizations should integrate security practices throughout the development and deployment pipeline. Container images should be built with security scanning as part of the CI/CD pipeline. Kubernetes manifests should be validated against security policies before deployment. Runtime scanning should detect suspicious behavior in running containers. This layered approach catches security issues before they reach production and reduces the likelihood that penetration testing will discover vulnerabilities that should have been caught during development.
Development teams should follow secure coding practices when building container images. Images should be built from minimal base images that contain only necessary software. Unnecessary tools and libraries should be removed to reduce the attack surface. Default credentials should never be included in images. Secrets should never be embedded in images. User accounts in containers should have minimal privileges. Health checks and resource limits should be configured to prevent containers from being used as launch points for attacks against the broader infrastructure.
The shift-left security model applies to container environments. Security assessments should begin during development when changes are cheap and fast. Developers should have access to security scanning tools that run on their workstations and CI systems. Container images should be scanned automatically every time they're built. Kubernetes manifests should be validated before being committed to version control. This approach catches security issues during development rather than discovering them during production penetration testing.
Conclusion: Securing Cloud-Native Infrastructure
Container and Kubernetes penetration testing addresses security challenges specific to cloud-native infrastructure. The complexity of container and Kubernetes environments, combined with the rapid pace of change in organizations adopting cloud-native practices, means that security vulnerabilities in container deployments are common. Organizations running containerized applications at scale should conduct regular penetration testing specifically focused on container security, Kubernetes configuration, and cloud-native infrastructure.
Effective container and Kubernetes security requires a comprehensive approach combining secure development practices, automated security scanning, penetration testing, and operational security controls. Organizations that combine these approaches build containerized systems that resist attack and reduce the likelihood of compromise. Regular penetration testing helps identify gaps in this layered defense and guides continuous improvement of container security practices.