Securing API Calls in Kubernetes, a simple and effective Approach

Alessandro Duminuco
Alessandro Duminuco

Wednesday, April 13th, 2022

Historically, API authentication methods have involved a tradeoff between security and convenience. Today, with cloud native applications and Kubernetes, there are several options, but security tradeoffs remain:

  • You could hardcode credentials, which is convenient for developers, but it also leaves your secrets at risk of exposure to anyone who manages to access your source code.
  • A second option would be to hard-code credentials in deployment manifest and make them available through mounted files or environment variables. The problem here is devops generally has access to the manifest, so now everyone has access to such credentials. Manifests are also kept in etcd which inherently is not secure.
  • A third option is setting the credentials as K8s secrets and mounting them as files or an environment variable. In this scenario, however, providing fine grained access control and enabling automatic key rotation is challenging, as it's not directly available within K8s. Moreover secrets are still kept in etcd.

The solution - and of course you were already there already, put the credentials in a vault and inject them in the pod at startup

This scenario is the most secure option as vaults use a very secure encrypted storage mechanism, enable fine grained access control, and provide the ability to automatically rotate secrets on a routine basis.

Now the challenge is getting the secrets from the vault to the application - when accessed.

Increasing API security attacks

Before diving into how to use a secret vault, let’s explain why such a tool is so important.

Security attacks against internal APIs -- meaning the APIs that businesses use to manage access to internal resources, as opposed to external APIs (where one accesses a third party service) -- have been on the rise in recent years.

One recent attack was due to an authentication issue with Peleton software. It allowed attackers to make unauthenticated requests to Peleton’s API for users account data -- including sensitive personal information like age and height.

Along similar lines, a LinkedIn breach exposed the records of 700 million users -- about 92 percent of the LinkedIn community. The data was scraped via an insecure API. Although it was technically publicly available data, the ability to collect it systematically via the API was a boon to threat actors looking to target high-profile executives or particular business functions, such as finance.

You can find plenty of similar examples of API security vulnerabilities if you search through the CVE Program database, which records API security flaws and other security risks.

Using a secret vault to secure API calls

How can businesses prevent risks like these?

Obviously, designing secure APIs is part of the answer. But even the best-designed API can be abused when the access credentials it uses to authenticate requests are not managed securely.

That’s why a secret vault is a key ingredient in securing API calls. A secret vault is a resource that allows you to:

  • Store passwords, API tokens and other access credentials securely, with full data encryption.
  • Define fine-grained access controls over secrets.
  • Define token values, such as where and for how long each token can be used, in order to add an extra layer of protection against abuse.

Token Injection and Secret Vaults

With a secret vault, and token injection, you can ensure that every token reaches an application, Kubernetes Pod or other resource when necessary. You don’t need to worry about sharing tokens through files, storing them directly in your cluster or (worse) hard-coding them. Using a tool like Panoptica, which integrates with secret vaults, you can ensure that the relevant token reaches the relevant application, Kubernetes Pod or other resources as required via token injection. This means you don’t need to worry about sharing tokens through files, storing them directly in your cluster or (worse) hard-coding them

And you can do all of this simply, without a complex coding or secrets management process.

Secret vault example

To prove the point, here’s a look at how a secret vault can be used in the real world.

Imagine that you have an application that needs to interact with PayPal to make a payment. The application is running inside a Kubernetes pod. In the example below we are showing a sample e-commerce selling socks.

However, the payment service that makes the call to PayPal is not configured with the proper token to authenticate with PayPal. As a result, authentication will fail; Paypal will reject the payment because the pod hosting the application is not authenticated.

It is not recommended to keep the secrets hard-coded within the existing K8s pod, because if the credentials are compromised, then the information will be exposed to all. To overcome this issue, we can use an application, like Secure Application Cloud, which uses secret vaults to provide stronger security in a cloud native environment.

Panoptica provides a navigator view where you can visualize the payment pod and how it connects to the application. (see the arrow from the payment pod to paypal server via the api). In this scenario, with a secret vault already configured, we can just create a policy to inject the secret securely into the pod.

To do so, we first define the token:

In this case, we create a token pointing to a specific location within the vault: secret/data/paypalpaypaltoken

We also designate an attribute type for visibility into API activity. We also specify where the token will be used in HTTP requests, for example the header name where the token will be set. This allows us to track where and when the token is used.

Then we specify where to inject the token using the injection policy:

As you can see, we specify that the token can be injected by all pods that are named “payment.” That way, only those specific pods can use the token, which significantly reduces risks and gives us granular access control over the token. We can also specify clusters and environments that can use the token.

We then specify which token to use -- the one we created before -- and we specify where to find the token in the system Finally, we also specify which environment variable will hold the token:

If we need to change rules at any point, doing so is very easy. Simply log into the Panoptica UI.

Once the policy is in place the application is able to process the payment:

This time the payment was successful as the token was able to interact with PayPal.

Note that when you look in the pod for the environment variable, the secret remains hidden. Only the location is revealed. So, we’ve enabled simple authentication, while keeping our secret secure.

API security visibility

In addition to configuring an authentication process that is both simple and secure, the secret vault approach combined with Panoptica offers the important advantage of deep visibility into API requests.

You can view all events in the Panoptica portal. You can also drill down into requests related to the payment authentications and see which resources were being used and when.

The image below shows the runtime workload where the token is injected:

The second image shows the connection info between the payment workload and the Paypal API service, and shows that the token has been injected.


API authentications used to require compromise between security and convenience.

However, when using a secret vault, combined with Panoptica, this is no longer the case. With Panoptica you can store tokens and other secrets securely in a vault, while easily defining granular access-control policies that specify exactly how and where tokens can be used.

Try Cisco Panoptica for free.