At Banzai Cloud we work with Istio quite a bit and run a lot of Istio-based service meshes for our customers. Earlier this year we opensourced the Banzai Cloud Istio operator in order to simplify provisioning, management, upgrades, and multi-cluster scenarios. However, we still felt that this was not simple enough. A few months ago we announced Backyards, a method of simplifying, creating and managing Istio, with planned support for all Istio features – the more complicated, the better!
As Backyards will be GA in September, we’d like to start a series of posts that take a deep dive into some intertesting Istio features, exploring their benefits, configuring/using them on Istio, and also showcasing how extremely simple it is to do that with Backyards (now Cisco Service Mesh Manager), even in multi- and hybridcloud environments.
In this particular post, we’ll be focusing on Istio’s traffic shifting feature.
Backyards is a core component of the Pipeline platform as well.
Traffic shifting makes it possible to gradually migrate traffic from one version of a microservice to another. This usually happens when migrating from an older version of an app to a newer one.
When the migration process itself is automated and rolled back automatically on failure, it’s called a canary release. To learn more about how to execute canary releases with Backyards, check out our introductory blog post on automatic canary deployments.
Istio’s traffic shifting can be configured by two Istio Custom Resources, namely
Destination Rule and
Virtual Service. In short, in a
Destination Rule so-called subsets need to be defined to identify specific versions of a service, and in a
Virtual Service different weights can be assigned to these subsets in order to control how much traffic should be directed to each service version.
For the subsets, you’ll need to define a
Destination Rule like this:
apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: movies spec: host: movies subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 - name: v3 labels: version: v3
This defines the following subset names:
v3. These subset names can be used later in
Virtual Services to route traffic to the appropriate version of our
movies service, based on predefined version labels.
There’s one thing which you need to pay special attention to when you manually create the
Destination Ruleresource, which is whether or not you have mutual TLS enabled for this service. If you do, you’ll also need to set the field below within your
Destination Rule, otherwise your caller services will probably receive 503 responses when calling the
trafficPolicy: tls: mode: ISTIO_MUTUAL
Mutual TLS can be enabled globally for a specific namespace or for a specific service, as well. You should be aware of these settings in order to determine whether you should set
ISTIO_MUTUALor not. More importantly, it is very easy to forget to set this field when you are trying to configure a completely different feature (e.g. traffic shifting).
Tip: Always think about mutual TLS before creating a
For the weights, you’ll need to define a
Virtual Service like this:
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: movies spec: hosts: - movies http: - route: - destination: host: movies subset: v1 weight: 33 - destination: host: movies subset: v2 weight: 33 - destination: host: movies subset: v3 weight: 34
Virtual Service is set to route 33% of all traffic to subset
v1, 33% to
v2 and 34% to
To make sure that these settings are actually functional, you can check the application logs or use a monitoring tool to examine the flow of traffic.
When using Backyards, you don’t need to manually edit
Destination Rules and
Virtual Services to set traffic shifting rules. Instead, you can achieve the same result via a convenient UI, or, if you prefer, through the Backyards CLI command line tool.
We’ll introduce the Backyards CLI tool in detail in an upcoming blog post!
You don’t need to worry about misconfiguring your
Destination Rule by forgetting to set
ISTIO_MUTUAL. Backyards takes care of this for you; it finds out whether your service has mutual TLS enabled or not and sets the aforementioned field accordingly.
This is just one example of the validation features of Backyards which can protect you from a potential misconfiguration. There are many more to come which will be introduced soon!
On top of this, you can see visualizations of your services and requests, so you can easily determine how much traffic is being directed to each of your workloads.
First of all, we’ll need a Kubernetes cluster.
I created a Kubernetes cluster on GKE via the Pipeline platform. If you’d like to do likewise, go ahead and create your cluster on any of the several cloud providers we support or on-premise using Pipeline for free.
The easiest way by far of installing Istio, Backyards, and a demo application on a brand new cluster is to use the Backyards CLI.
You just need to issue one command (
KUBECONFIG must be set for your cluster):
$ backyards install -a --run-demo
This command first installs Istio with our open-source Istio operator, then installs Backyards itself as well as a demo application for demonstration purposes. After the installation of each component has finished, the Backyards UI will automatically open and send some traffic to the demo application. By issuing this one simple command you can watch as Backyards starts a brand new Istio cluster in just a few minutes! Give it a try!
You can do all these steps in a sequential order as well. Backyards requires an Istio cluster – if you don’t have one, you can install Istio with
$ backyards istio install. Once you have Istio installed, you can install Backyards with
$ backyards install. Finally, you can deploy the demo application with
backyards demoapp install.
If you’ve installed Backyards with the Backyards CLI, by default, traffic shifting will be configured for the
movies service so that it routes 33% of all traffic to subset
v1, 33% to
v2 and 34% to
v3. You don’t have to fetch the
Virtual Service (e.g. with
kubectl) to get this information. It is immediately available on the right side of the Backyards UI when you click on the
movies service icon.
When traffic begins flowing, the request rates from the
movies service to its workloads should be very close to each other. You don’t need to check application logs to find out how much traffic is flowing to each subset, since you can see that immediately on the UI.
See it in action: https://youtu.be/n1jEjuSAqLA
You don’t need to edit the corresponding
Virtual Service resource manually, you can easily change the weights for each subset from the UI. Let’s set it so that all traffic goes to
As you will see, Backyards (compared e.g. with Kiali) is not just a web based UI mainly for observability, but also a featureful management tool for your service mesh, which is able to work with single- and multi-cluster service meshes, have a powerful CLI and GraphQL API as well.
After setting this rule and sending a workload to the service, we can make sure that all traffic is actually being routed to
See it in action: https://youtu.be/fIb_1oyLoZw
You can easily remove traffic shifting rules with the
Remove rules button. If you do so, traffic will return to being distributed equally across the workloads, so request rates should also return to about equal:
See it in action: https://youtu.be/CWMgvb8W_l8
You can list the traffic shifting rules of a service in a given namespace with the following command:
$ backyards routing ts get backyards-demo/movies INFO traffic shifting for backyards-demo/movies is currently set to v1=33, v2=33, v3=34
Now let’s make the same changes we did earlier, but this time through the CLI.
$ backyards routing ts set backyards-demo/movies v2=100 INFO traffic shifting for backyards-demo/movies set to v2=100 successfully
To verify that the command was successful:
$ backyards routing ts get backyards-demo/movies INFO traffic shifting for backyards-demo/movies is currently set to v2=100
To remove the traffic shifting rules:
$ backyards routing ts delete backyards-demo/movies INFO traffic shifting rules set to backyards-demo/movies successfully deleted
To verify that the command was successful:
$ backyards routing ts get backyards-demo/movies INFO no traffic shifting rules set for backyards-demo/movies
Backyards is composed of several components, like Istio, Banzai Cloud’s Istio operator, our multi-cluster Canary release operator, as well as several backends. However, all of these are behind Backyards’ GraphQL API.
The Backyards UI and CLI both use Backyards’ GraphQL API, which will be released with the GA version (we are in the middle of a refactor). Thus, users will soon be able to either use our tools to manage Istio or build their own clients!
To remove the demo application, Backyards, and Istio from your cluster, you just need to apply one command, which takes care of removing these components in the correct order:
$ backyards uninstall -a
With Backyards, you don’t necessarily need to be familiar with Istio’s Custom Resources, and you don’t have to edit them manually to set traffic shifting rules. Instead, you can easily configure these rules from a convenient UI or with the Backyards CLI command line tool. You can then check the visualized traffic flow to make sure that the rules and your services are working correctly.
As you see, even a relatively simple feature like traffic shifting can lead to potential misconfigurations (e.g. omitting to set
Destination Rule). Backyards automatically protects you from such > mistakes.
Next up: Circuit Breaker.
Banzai Cloud’s Backyards (now Cisco Service Mesh Manager) is a multi and hybrid-cloud enabled service mesh platform for constructing modern applications. Built on Kubernetes and our Istio operator, it gives you flexibility, portability, and consistency across on-premise datacenters and cloud environments. Use our simple, yet extremely powerful UI and CLI, and experience automated canary releases, traffic shifting, routing, secure service communication, in-depth observability and more, for yourself.
Banzai Cloud is changing how private clouds are built: simplifying the development, deployment, and scaling of complex applications, and putting the power of Kubernetes and Cloud Native technologies in the hands of developers and enterprises, everywhere.
#multicloud #hybridcloud #BanzaiCloud