The Vault swiss-army knife

Bank-Vaults is a thick, tricky, shifty right with a fast and intense tube for experienced surfers only, located on Mentawai. Think heavy steel doors, secret unlocking combinations and burly guards with smack-down attitudes. Watch out for clean-up sets.

Bank-Vaults is a wrapper for the official Vault client with automatic token renewal, built in Kubernetes support, dynamic database credential management, multiple unseal options, automatic re/configuration and more.

At Banzai Cloud, we’ve been building an open source next generation platform as a service, called Pipeline, which is constructed on top of Kubernetes. With the Banzai Cloud Pipeline platform, we provision large, multi-tenant Kubernetes clusters on all major cloud providers, and deploy different workloads to these clusters. We needed to find an industry standards-based way for our users to publish and interact with protected endpoints, while simultaneously providing dynamic secrets management for all the applications we support, each of which is native to Kubernetes. After several proof-of-concepts, we chose HashiCorp’s Vault, and we began to integrate our many Pipeline PaaS components. After awhile, work on this project (all of which is open sourced and available on GitHub) began to feel like reinventing the wheel, so we decided to externalize all the enterprise grade security features into a new project. Welcome to Bank-Vaults.

Security series:
Authentication and authorization of Pipeline users with OAuth2 and Vault
Dynamic credentials with Vault using Kubernetes Service Accounts
Dynamic SSH with Vault and Pipeline
Secure Kubernetes Deployments with Vault and Pipeline
Policy enforcement on K8s with Pipeline
The Vault swiss-army knife
The Banzai Cloud Vault Operator
Vault unseal flow with KMS
Kubernetes secret management with Pipeline
Container vulnerability scans with Pipeline
Kubernetes API proxy with Pipeline

Credit to HashiCorp for open sourcing Vault and making secret management easier and more secure. This project would not have been possible without the open source community.

Bank-Vaults, the swiss-army knife

We repeatedly push large deployments to different cloud providers, all of which are automated and managed by Pipeline and use Kubernetes. However, we purposefully designed Bank-Vaults in such a way that it is able to operate independently of Kubernetes. Pushing out a secure deployment to the cloud or Kubernetes has a certain order or flow, and while we were repeatedly doing this we incorporated that feature in Bank-Vaults. At high level, the project is a CLI tool to manage and configure Vault, a Golang library on which to build functionality, a Kubernetes deployment using a Helm chart to deploy production ready HA Vault clusters, a Vault operator to manage the lifecycle of the cluster, and a collection of scripts to configure cloud instances.

Lets briefly review these features; for additional information and to get the code in question, please visit the project’s microsite or GitHub repository.

The CLI tool

The bank-vaults CLI tool helps automate the setup and management of Vault.

Features:

• Initializes Vault and stores the root token and unseal keys in one of the following:
  • AWS KMS keyring (backed by S3)
  • Azure Key Vault
  • Google Cloud KMS keyring (backed by GCS)
  • Kubernetes Secrets (these should only be used for development purposes)
  • Dev Mode (useful for vault server -dev dev mode Vault servers)
• Automatically unseals Vault with these keys
• Continuously configures Vault with a YAML/JSON based external configuration (besides the standard Vault configuration)
  • If the configuration has been updated, Vault will be reconfigured
  • It supports configuring Vault secret engines, authorization methods, and policies

For a list of external configurations, please visit the CLI tool’s site. Once a configuration YAML is pushed with the CLI tool, we re/configure the Vault cluster in the backend, and apply those changes. Here are a few of our options:

• create policies in Vault which can be used later for Kubernetes based authentication
• configure Auth Methods in Vault – Kubernetes and GitHub are currently supported, and we’ve already begun work on a few alternatives
• create team maps in Vault, which can be used later for GitHub based authentication
• configure Secrets Engines in Vault – KV, Database and SSH have been tested, but the config is freeform so more are probably supported
• store arbitrary secrets within configured physical storage for Vault
• create a named Vault role for signing SSH client keys

The Go library

This repository contains several Go packages for interacting with Vault:

• auth
  Note: This is currently WIP, we are extracting the code from https://github.com/banzaicloud/pipeline/tree/master/auth
  A GitHub OAuth2 based authentication system as Gin Middleware, stores JWT bearer tokens in Vault.
• vault
  A wrapper for the official Vault client with automatic token renewal, and Kubernetes support.
• databaseA helper for creating database source strings (MySQL/PostgreSQL) with database credentials dynamically based on configured Vault roles (instead of username:password).

Helm Chart

We have a fully fledged, production ready Helm chart for Vault that uses bank-vaults. With the help of this chart, you can run a HA Vault instance with automatic initialization and unseal an external configuration, which used to be a tedious manual operation. In addition, this chart can easily be used for development purposes.

Operator

We have a Vault operator built on bank-vaults with features such as:

• TLS support
• external, API based configuration (secrets engines, auth methods, policies) to automatically re/configure a Vault cluster
• automatic unsealing with AWS, GCE, Azure, Alibaba, Kubernetes Secrets (for dev purposes), file, and Vault -dev mode

What’s next

We have some interesting features in beta testing/the development pipeline, most of them tracked as GitHub issues, however, we’d like to mention one that’s particularly popular among our users:

• the availability to remotely and securely unseal Vault, using an API and a mobile application supporting multiple secret shares
• and monitoring and alerts with Prometheus

We hope you’re as enthusiastic about this project as we are, and appreciate the effort we’ve put into securing our Pipeline PaaS. We take bank-vaults security and our users’ trust very seriously, so if you believe you have found a security issue in bank-vaults, please contact us at security@banzaicloud.com.