Cisco Tech Blog CiscoTech Blog Close
Home Contact

Inside Kubernetes Network Policy

Author

The Kubernetes Networking Model guarantees that all Kubernetes Pods on a Kubernetes Cluster are able to communicate, that is, able to exchange messages. However, the Networking Semantics change in the presence of Networking Policies from Default Allow for all Pods to Default Deny for selected Pods with Explicit Allow:

Figure 1. Default Allow, Default Deny, and Explicit Allow Figure 1. Default Allow, Default Deny, and Explicit Allow

Default Allow ๐Ÿ”—︎

The Kubernetes Network permits all incoming and outgoing messages to and from any Pod per default.

Default Deny ๐Ÿ”—︎

However, if there exists some Network Policy selecting a Pod, the Kubernetes Network prohibits all incoming and outgoing messages to and from the Pod per default.

Explicit Allow ๐Ÿ”—︎

Then, if there exists some Network Policy selecting a Pod and some rule of the Network Policy allowing Ingress or Egress, the Kubernetes Network permits matching incoming and outgoing messages to and from the Pod.

K8s Networking Policies ๐Ÿ”—︎

Kubernetes Networking Policies are a core abstraction of Kubernetes: Per Pod, Network Policies determine if an incoming (Ingress) or outgoing (Egress) message is permitted or prohibited by the Kubernetes Network.

Figure 2 Kubernetes Network Policies Figure 2 Kubernetes Network Policies

Not this ๐Ÿ”—︎

Since Kubernetes Pods are arguably the core abstraction of Kubernetes, one may reasonably conclude that Kubernetes Network Policies express allowed traffic flow holistically in terms of Pods e.g. in terms of a set of Pods, the possible source, and a set of Pods, the possible target.

Figure 3. Not like this Figure 3. Not like this

However, this is not how Network Policies work.

But that ๐Ÿ”—︎

A Kubernetes Network Policy determines whether an incoming or outgoing message is either permitted or prohibited.

For two Pods to communicate

  • both Pods must be unselected or
  • there must exist
    • a Network Policy that permits Egress from the source to the target and
    • a Network Policy that permits Ingress to the target from the source.

Figure 4. But like that Figure 4. But like that

K8s Networking Policies โ— Formal Description ๐Ÿ”—︎

Figure 5. Kubernetes Network Policy Object Data Structure Figure 5. Kubernetes Network Policy Object Data Structure

A Kubernetes Network Policy Object may be classified as an Ingress or Egress Policy. Note that a Kubernetes Network Policy Object may be an Ingress and Egress Policy at the same time:

Ingress-Policies โ‰
  {P โˆˆ Policies : "Ingress" โˆˆ as-set(Pol.Spec.PolicyType)}
Egress-Policies โ‰
  {P โˆˆ Policies : "Egress" โˆˆ as-set(Pol.Spec.PolicyType)}

As the names imply

  • Ingress Policies default to deny and explicitly allow incoming messages

  • Egress Policies default to deny and explicitly allow outgoing messages of selected Pods.

  • Ingress of a Pod is restricted if there is at least one Ingress Network Policy that selects that Pod.

  • Egress of a Pod is restricted if there is at least one Ingress Network Policy that selects that Pod.

IngressRestricted(Pod) โ‰
  โˆƒ Pol โˆˆ Ingress-Policies:
    Pod.Labels match Pol.Spec.PodSelector
EgressRestricted(Pod) โ‰
  โˆƒ Pol โˆˆ Egress-Policies:
    Pod.Labels match Pol.Spec.PodSelector
  • Ingress to a Pod is allowed if the Pod is not restricted or if the Pod is restricted but there exists an Ingress Rule that explicitly allows Ingress to this Pod.
  • Egress from a Pod is allowed if the Pod is not restricted or if the Pod is restricted but there exists an Egress Rule that explicitly allows Egress from this Pod.
IngressAllowed(Pod, (sAddr, sPort, tAddr, tPort, Proto)) โ‰
 IngressRestricted(Pod)
  โŸน
   โˆƒ Pol โˆˆ Ingress-Policies:
     โˆง Pod.Labels match Pol.Spec.PodSelector
     โˆง โˆƒ Rule โˆˆ Pol.Spec.Ingress:
       โˆง โˆƒ Peer โˆˆ Rule.From:
           PeerAllowed
             (Peer, sAddr)
       # if no port is specified all ports and protocols are allowed 
       โˆง Rule.Ports โ‰  โˆ…
          โŸน
           โˆƒ Port โˆˆ Rule.Ports:
             โˆง Port.Protocol = Proto
             โˆง Port.Port match tPort
EgressAllowed(Pod, (sAddr, sPort, tAddr, tPort, Proto)) โ‰
 EgressRestricted(Pod)
  โŸน
   โˆƒ Pol โˆˆ Egress-Policies:
     โˆง Pod.Labels match Pol.Spec.PodSelector
     โˆง โˆƒ Rule โˆˆ Pol.Spec.Egress:
       โˆง โˆƒ Peer โˆˆ Rule.To:
           PeerAllowed
             (Peer, tAddr)
       # if no port is specified all ports and protocols are allowed
       โˆง Rule.Ports โ‰  โˆ…
          โŸน
           โˆƒ Port โˆˆ Rule.Ports:
             โˆง Port.Protocol = Proto
             โˆง Port.Port match tPort
  • An Ingress Policy selects its peers based on the Source Address of the incoming message,
  • an Egress Policy selects its peers based on the Target Address of the outgoing message, however both follow the same logic.
PeerAllowed(Peer, Addr) โ‰
 CASE Peer.PodSelector โ‰  NIL Peer.NamespaceSelector โ‰  Nil
    โ†’ โˆƒ S โˆˆ Pods:
        โˆง S.Labels match From.PodSelector
        โˆง S.NameSpace match From.NamespaceSelector
        โˆง S.Status.PodIP = Addr
    โ—ป๏ธŽ Peer.PodSelector โ‰  NIL Peer.NamespaceSelector = Nil
    โ†’ โˆƒ S โˆˆ Pods:
        โˆง S.Labels match From.PodSelector
        โˆง S.NameSpace = Pol.Namespace
        โˆง S.Status.PodIP = Addr
    โ—ป๏ธŽ Peer.PodSelector = NIL Peer.NamespaceSelector โ‰  Nil
    โ†’ โˆƒ S โˆˆ Pods:
        โˆง S.NameSpace = From.NamespaceSelector
        โˆง S.Status.PodIP = Addr
    โ—ป๏ธŽ Peer.PodSelector = NIL Peer.NamespaceSelector = Nil
    โ†’ โˆง sAddr โˆˆ as-set(From.IPBlock.CIDR)
      โˆง โˆ„ Except in From.IPBlock.Except
          Addr โˆˆ as-set(Except)

Implementation ๐Ÿ”—︎

Please visit Kubernetes Networking for a discussion of the Kubernetes Networking Model and the Conceptual Networking Model used throughout this blog post:

In this model, a network is a Network Graph that consists of a set of Nodes and a set of Links between the Nodes: One Node may exchange a message with another Node if and only if there exists a link between the Nodes.

A Node in the network is either a Process or a Switch. Processes produce and consume messages, Switches process messages according to their Forward Information Base (FIB).

Kubernetes provides a Network Specification but Kubernetes does not provide a Network Implementation. In fact, many alternative implementations, called Network Plugins, exist today.

In order to use Kubernetes Network Policies, that is, in order for the presence of a Network Policy Object to have the desired effects, the installed Network Plugin must support Network Policies.

Example ๐Ÿ”—︎

Figure 6. Kubernetes Cluster Kโ‚ Figure 6. Kubernetes Cluster Kโ‚

Figure 6. illustrates the example used in this section: A Kubernetes Cluster Kโ‚ that consists of two Nodes. Each Node hosts two Pods. Each Pod resides in the default namespace and executes two Containers, one Container listening on port 8080, one Container listening on port 9090.

In addition

  • Pod Pโ‚ has a labelset of {role: frontend}
  • Pod Pโ‚‚ has a labelset of {role: backend}

For this example, we will assume that following requirements:

  • Pโ‚ may receive traffic from any source
  • Pโ‚ may send traffic to Pโ‚‚ only
  • Pโ‚‚ may receive traffic from Pโ‚ only
  • Pโ‚‚ may send traffic to any target

Default Deny ๐Ÿ”—︎

In order to restrict communication we must flip

  • Pโ‚โ€™s Egress and
  • Pโ‚‚โ€™s Ingress from Default Allow to Default Deny, that is, we must create
  • an Egress Policy that selects Pโ‚ and
  • an Ingress Policy that selects Pโ‚‚. Pโ‚ƒ and Pโ‚„ are not selected, therefore Pโ‚ƒโ€™s and Pโ‚„โ€™s Ingress and Egress are not affected.
# Default Deny Pโ‚โ€™s Egress
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: frontend-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: frontend
  policyTypes:
  - Egress
# Default Deny Pโ‚‚โ€™s Ingress
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: backend
  policyTypes:
  - Ingress

The frontend-policy and backend-policy result in a network configuration that is equivalent to

  • Pโ‚โ€™s FIB discarding all messages with a source address of Pโ‚โ€™s IP Address 10.0.0.1 and any target address other than its own.
  • Pโ‚‚โ€™s FIB discarding all messages with a target address of Pโ‚‚โ€™s IP Address 10.0.0.2 and any source address other than its own.

Figure 7. FIB for Pโ‚ and Pโ‚‚ Figure 7. FIB for Pโ‚ and Pโ‚‚

Explicit Allow ๐Ÿ”—︎

In order to allow communication from Pโ‚ to Pโ‚‚ we must flip

  • Pโ‚โ€™s Egress to Pโ‚‚ and
  • Pโ‚‚โ€™s Ingress from Pโ‚ from Default Deny to Explicilty Allow, that is, we must update
  • frontend-policy and add an Egress Rule and
  • backend-policy to add an Ingress Rule
# Explicitly Allow Pโ‚โ€™s Egress to Pโ‚‚
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: frontend-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: frontend
  policyTypes:
  - Egress
  egress:
  - to:
    - podSelector:
        matchLabels:
          role: backend    
# Explicitly Allow Pโ‚‚โ€™s Ingress from Pโ‚
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: backend
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          role: frontend

The frontend-policy and backend-policy result in a network configuration that is equivalent to

  • Pโ‚โ€™s FIB discarding all messages with a source address of Pโ‚โ€™s IP Address 10.0.0.1 and any target address other than its own except if the target address is Pโ‚‚โ€™s IP Address and target port is 8080 or 9090.
  • Pโ‚‚โ€™s FIB discarding all messages with a target address of Pโ‚‚โ€™s IP Address 10.0.0.2 and any source address other than its own except if the source address is Pโ‚โ€™s IP Address and target port is 8080 or 9090.

Figure 8. FIB for Pโ‚ and Pโ‚‚ Figure 8. FIB for Pโ‚ and Pโ‚‚

In conclusion, Pโ‚ may receive messages from any source but may only send messages to Pโ‚‚, while Pโ‚‚ may only receive messages from Pโ‚ but may send messages to any target.

In combination, Pโ‚ may exchange messages with Pโ‚‚.

Conclusion ๐Ÿ”—︎

The Kubernetes Networking Model guarantees that all Kubernetes Pods on a Kubernetes Cluster are able to communicate, that is, able to exchange messages. However, the Networking Semantics change in the presence of Networking Policies from Default Allow for all Pods to Default Deny for selected Pods with Explicit Allow.

In a nutshell, for two Pods to communicate

  • both Pods must be unselected or
  • there must exist
    • a Network Policy that permits Egress from the source to the target and
    • a Network Policy that permits Ingress to the target from the source.