Spiffe.io is a universal identity control plane for distributed systems. SPIFFE is a set of standards to help us achieve zero trust identity framework and SPIRE is a reference implementation of SPIFFE. SPIRE can authenticate and authorize workloads in a distributed system.
Essentially SPIRE maps workloads to SPIFFE identities and distributes credentials for secure micro-services communication between Kubernetes workloads. SPIRE servers can be deployed in different architectures. SPIRE Federation (Federating SPIRE Servers with Different Trust Domains) and Nested SPIRE (Chaining SPIRE Servers to Use the Same Trust Domain). SPIFFE/SPIRE Architectures
In this blog, we will demonstrate how to authenticate two SPIFFE-identified workloads that are identified by two different SPIRE Servers.
The first part of this tutorial demonstrates how to set up SPIRE federation by showing the SPIRE configuration changes and
spire-server commands used to set up a stock quote webapp frontend and service backend. The second part of this document lists the steps to show the scenario in action using Helm charts and
In this demonstration you will learn to:
To demonstrate SPIRE Federation, we will set up two Kubernetes clusters using Kind. Then deploy SPIRE server-agent workloads using SPIRE Helm chart provided in the example for distributing SPIFFE ID and trust credentials. Next, we will deploy MetalLB using the MetalLB Helm cart provided in the example to auto-assign external load balancer IP addresses to our clusters. Lastly, we will configure SPIRE servers to federate with its neighboring cluster using the cluster’s external IP.
The stock quote webapp frontend and service backend we deploy in our Kubernetes cluster leverages the SPIFFE workload API to get mTLS credentials. In the first cluster server app servers client endpoint with stocks values to the client, server fetches x509 certificate from SPIRE server and sets up a listening port for the client. Client app in the second cluster with SPIRE workloads fetches TLS bundles from all the trusted domains and selects the right certificate to connect to the server.
The server-client application is built to use
grpc.dial from gRPC library with spiffe-api the API provides functionality to use x509 trust bundle for setting up gRPC channel with its targets.
A port forwarded to your localhost on port 8080 can be used to access the client-server stockbroker webpage and initiate a mTLS request from client to server, which can be verified from the client-server pods logs.
To demonstrate SPIRE Federation, we’ll use a simple example from the official SPIRE user guide for the Federation model. Source available here spire-tutorials/docker-compose/federation
I have modified the server-client example that can be deployed in the Kubernetes cluster with SPIRE server-agent model. spire-federation-kind This example demonstrates a stock quote web app frontend and service backend by setting up two SPIFFE-identified workloads that are identified by two different SPIRE Servers.
Let’s say we have a stockbroker’s web app that displays stock quotes periodically from a stock market web service provider. We can see the scenario as below:
The server holds a set of secure information (stock quotes) and the client exposes HTTP endpoint to display quotes via a web browser. The communication between server-client is via gRPC protocol endpoints, and security for this connection is mTLS using SPIFFE identity framework.
Each workload has been implemented to use the SPIFFE API to authenticate with SPIRE server and obtain trust bundle credentials. The following describes the details of the server-client application authenticated by SPIFFE and secured by mTLS connection.
First, let’s clone spire-federation-kind repository
git clone https://github.com/nishantapatil3/spire-federation-kind
Compile and build modified stock quotes server and broker webapp client Docker images
# Replace docker push with your public docker repository ./1-build.sh
Create two clusters using Kind, by following the instructions given in below URL
Let’s name the clusters creates as
kind-2 for our deployment
# Create Kind clusters kind create cluster --name kind-1 kind create cluster --name kind-2 # Export kubeconfig to desired location mkdir -p ~/kubeconfigs kind get kubeconfig --name=kind-1 > ~/kubeconfigs/kind-1.kubeconfig kind get kubeconfig --name=kind-2 > ~/kubeconfigs/kind-2.kubeconfig
Once we have two clusters running, deploy MetalLB load-balancer and SPIRE components(server-agent). MetalLB load balancer is required to allow ingress traffic to the Kubernetes cluster and redirect traffic to target pods in our cluster.
globalPrefix refers to
xx.xx.<globalPrefix>.xx IP address that MetallLB can be configured to allow subnet range for the external IP address in Kubernetes. Example: If globalPrefix=255, external IP address configuration range is
172.17.255.1 to 172.17.255.255
# Apply the Helm chart helm template helm/metallb-system --set globalPrefix="255" | kubectl apply --kubeconfig $cluster1 -f - helm template helm/metallb-system --set globalPrefix="254" | kubectl apply --kubeconfig $cluster2 -f -
After we deploy load balancers in our clusters, let’s deploy spire-server and spire-agent
To do that we will apply below Helm charts with SPIRE values as shown below. We can set trustDomain with any alphanumeric combination and provide this trust name to federatesWith argument that the cluster federates with.
federatesWith.address should match the external IP of SPIRE service that it federates with, you can check this IP address using
kubectl get svc -n spire. Also,
port=8443 refers to the federation-endpoint of spire-server service configuration.
helm template helm/spire --set trustDomain=cluster1.com --set federatesWith.trustDomain=cluster2.com --set federatesWith.address=172.17.254.1 --set federatesWith.port=8443 | kubectl apply --kubeconfig $cluster1 -f - helm template helm/spire --set trustDomain=cluster2.com --set federatesWith.trustDomain=cluster1.com --set federatesWith.address=172.17.255.1 --set federatesWith.port=8443 | kubectl apply --kubeconfig $cluster2 -f -
Next, bootstrap SPIRE certificates with each other such that the workloads (server and client) can receive the certificate from either trust domains to establish the connection
2-bootstrap.sh uses spire-server’s tool
bin/spire-server bundle show to get trust bundle from one trust domain and using that it sets the trust bundle in the other clusters spire-server using
bin/spire-server bundle set
$ ./2-bootstrap.sh Setting clusters kubeconfig /home/ubuntu/spire-federation-kind/lab_clusters.sh Bootstrap certificate from cluster1 to cluster2 bundle set. Bootstrap certificate from cluster2 to cluster1 bundle set.
Run the following command to create SPIRE entries such that spire-server can fetch the right certificate from a list of registered trust domains to connect to its target.
3-register.sh uses spire-server’s container tool to register workloads stockbroker server and client in their respective clusters with SPIRE. We set the FederatesWith field to authenticate itself with other cluster’s spire-server in order to connect to workloads deployed in that cluster.
$ ./3-register.sh Setting clusters kubeconfig /home/ubuntu/spire-federation-kind/lab_clusters.sh ------------------------- Registering workload: server Entry ID : 12c07f5f-8629-4654-9961-dd6dbeba577e SPIFFE ID : spiffe://cluster1.com/server Parent ID : spiffe://cluster1.com/spire-agent Revision : 0 TTL : default Selector : k8s:sa:server-service-account FederatesWith : spiffe://cluster2.com ------------------------- ------------------------- Registering workload: client Entry ID : 8082ce59-4bca-43a5-83d5-006890f822b5 SPIFFE ID : spiffe://cluster2.com/client Parent ID : spiffe://cluster2.com/spire-agent Revision : 0 TTL : default Selector : k8s:sa:client-service-account FederatesWith : spiffe://cluster1.com -------------------------
Deploy stockbroker server in cluster1 and web app client in cluster2
$ kubectl apply -f helm/server.yaml --kubeconfig $cluster1 serviceaccount/server-service-account created service/stock-quotes-service created deployment.apps/stock-quotes-service created $ kubectl apply -f helm/client.yaml --kubeconfig $cluster2 serviceaccount/client-service-account created service/broker-webapp created deployment.apps/broker-webapp created
Stockbroker server utilizes SPIRE API to create a trust bundle and initiates a listening port with x509src security. SPIRE servers exchange this trust bundle and distribute it to authenticated workloads that ask for bundles to connect to its target. The client gets the trust bundle for a given domain (Trust domains that the client is registered to federate with) using bundleSrc.
Client gets two certificates from two trust domains and iterates over them to find the right certificate to connect to the stockbroker server workload and establishes mTLS secure connection to it. Client calls SPIRE API for all the trust domain that spire-server is configured to federate with and gets the trust bundle in an iterable-list Each bundle is picked up to connect to the target and if the connection succeeds, that bundle is channeled to caller function via a goroutine.
After a secure channel is established between server-client, the server provides stocks information to the client which can be processed to display information to the user.
Now we can Port forward the client pod to
localhost:8080 using kubectl. Replace the below command with client-pod’s name and run the command
Example: kubectl port-forward broker-webapp-85574f4585-cxvxg 8080:8080 - kubeconfig $cluster2 $ kubectl port-forward broker-webapp-85574f4585-q92x4 8080:8080 --kubeconfig $cluster2 Forwarding from 127.0.0.1:8080 -> 8080 Forwarding from [::1]:8080 -> 8080 Handling connection for 8080 Handling connection for 8080 Handling connection for 8080 Handling connection for 8080 Handling connection for 8080
Open any browser and navigate to
http://localhost:8080/quotes, you should see a grid of randomly generated phony stock quotes that are updated every 1 second.
Server (left k9s) and client (right k9s), you can see that client fetched two certificated from cluster1.com and cluster2.com, iterated over these certificates, and picked the right certificate to connect to stockbroker server. Source
By demonstrating the above steps we have successfully created SPIRE Federated clusters and secured server-client communication.