Cisco Research has started an ambitious Quantum research vision with an aim to develop hardware and software technologies for quantum computing, networking, and cryptography. In a previous blog post, we have already articulated how quantum photonics can be a common workhorse technology for powering them. In addition to in-house research, we are already conducting within Cisco’s Quantum Research team, we have initiated collaborations with several different universities on projects that are well-aligned with our research vision.

There are many ancient software development principles that still hold true today, and one that is undoubtedly true is that no software is perfect. There will always be bugs, new feature requests, and perhaps most importantly, every company will always want to be innovative and get more done in less time. Every company can choose to either do it alone or band together with other companies to develop open source projects.

GitOps is increasingly popular among developers as it accelerates development, but as security requirements grow, a new approach is needed. GitOps security needs to shift left. Here’s how to secure your GitOps repository. GitOps is gaining traction among developers as it accelerates development work significantly in the CI/CD process. GitOps provides developers with a single, unified source of truth, which eliminates many of the teams’ workflow administrative hurdles. It also simplifies sharing code and working collaboratively on shared repositories.

This is a guest post by Daniel Tkacik. This post was originally published in the Carnegie Mellon University CyLab website. The company will serve as a founding sponsor of the Future Enterprise Security Initiative As cyberattacks continue to plague our world’s critical infrastructure, researchers at Cisco are constantly looking for ways to up their game. “Cisco’s products are foundational technologies that power today’s digital infrastructure,” says Dr. Ramana Kompella, Distinguished Engineer and the Head of Research in the Emerging Tech and Incubation group at Cisco.

You can’t secure what you can’t see. APIs are no exception. Whether your application uses internal APIs to manage interactions between microservices, external APIs to integrate with third-party services, or both, you need to be able to visualize your API architecture, identify risks within it and enforce security policies. This blog explains why API visualization and policy enforcement are critical elements of API security, and the roles they play in securing API calls.

Running multiple Kubernetes clusters in a company or even within a team is a common practice. You need test environments or just want to separate workloads between customers. Prometheus is an awesome tool to monitor a single cluster. But if you want to query multiple clusters, you need the help of other tools. If you are a regular reader, you know that our choice for this task is Thanos. If you are not familiar with Thanos, read our Multi cluster monitoring with Thanos blog post first.

Containers Over the years, the way we build our applications has changed from a monolithic paradigm to a Microservices paradigm. Containerization gained strength, and with it the success of Docker and Kubernetes. Briefly, a container is a set of processes. If you have multiple containers, you will have multiple sets of processes which are isolated from each other using namespaces and cgroups. Namespaces are responsible for isolating processes (which processes can my process see?

Although Deep Learning (DL) has been the dominant field in Artificial Intelligence for more than a decade, its history can be traced back to the 1940s. The term ‘Deep Learning’ was introduced to the machine learning community by Rina Dechter in 1986. Later, Yan LeCun and his colleagues demonstrated the famous and laudable implementation of DL to recognize handwritten ZIP codes, a feat which took 3 days of training. The “big bang” of DL occurred in 2009 when Nvidia’s enlisted GPUs to train DL models, an approach which reduced training time by a factor of many hundreds.

The apps we are building are becoming atomized: they are fracturing into microservices and ultra-specific APIs. I am excited that one of the most interesting and influential groups of developers working on these apps will be coming together to explore the paths forward in software next week at KubeCon + CloundNativeCon 2021. I’ll be there with the Cisco team and I want to give you a preview of what we’re looking forward to discussing…and announcing.

Every API falls into one of two categories: Internal or external. External APIs are APIs that developers use to integrate their applications with a third-party resource, such as a public cloud service or a SaaS application. This type of API is probably what you think of first and foremost when you think about APIs. However, there are also internal APIs. These are the APIs that handle communication between the constituent parts of an application.

If you asked me two years ago if I was going to write this blog – I would have told you no way. I’m a technology architect – I design things. As many that know me well will tell you, I say it often, I am “not a coder”. “But wait, didn’t you go to college for programming?” – not really, I took some college courses (Borland Turbo C), and in high school I did some Visual Basic, and I have done some scripting.

It’s fair to say that we’ve been champions of technology’s role in driving benefit for the UK for some time now, but last week the air was rife with talk of the future, of what needs to happen now and next. From London Tech Week to the publishing of the National AI Strategy, the focus is now on the next decade to position the UK as leaders, innovators and most importantly beneficiaries of all things AI.

Developers are increasingly making use of APIs to accelerate their software’s time to market. While extremely beneficial to the business, the growing trend of leveraging APIs coupled with building applications in a cloud native environment has brought into focus the need for increased visibility and security for APIs. APIs, as designed, often differ from their runtime state due to differences over time, deprecated APIs, zombie APIs, and breaking backward compatibility. Many applications do not give clients access to their OpenAPI specification, making new integrations and API risk assessment difficult at best.

Cloud native is here for a while now and runs many, if not most digital leader’s products with high resiliency across the world. Kubernetes became the de-facto platform managing these containerized applications but what about their communication? Microservices and event driven architectures thrive in these environments and Apache Kafka remained the leader in data streaming and message brokering. So the question in the title presents itself. Hint: I could just say that of course with everything the easy answer is it depends, but in my opinion, it’s more about the how instead of the should.

This is a guest post by Andrew Masuda. This post was originally published in the University of California Santa Barbara Engineering news events page. A new collaboration between UC Santa Barbara researchers and Cisco Systems aims to push the boundaries of quantum technologies. Assistant professors Yufei Ding and Galan Moody have received research awards from the technology giant to work with its new Quantum Research Team, which was formed to pursue the research and development required to turn quantum hardware, software, and applications into broadly used technologies.

Spiffe.io is a universal identity control plane for distributed systems. SPIFFE is a set of standards to help us achieve zero trust identity framework and SPIRE is a reference implementation of SPIFFE. SPIRE can authenticate and authorize workloads in a distributed system. Essentially SPIRE maps workloads to SPIFFE identities and distributes credentials for secure micro-services communication between Kubernetes workloads. SPIRE servers can be deployed in different architectures. SPIRE Federation (Federating SPIRE Servers with Different Trust Domains) and Nested SPIRE (Chaining SPIRE Servers to Use the Same Trust Domain).

What is Federated Learning? Federated learning (FL) is a machine learning (ML) mechanism where different parties participate in a machine learning task and build a global model without sharing training data with any other parties. While there are several different training modes, a typical setting consists of two types of computing nodes: (1) trainer and (2) aggregator. The trainer node processes a dataset locally to build a model; and a set of trainer nodes share their model parameters with the aggregator node.

Since the early days of the Internet (Arpanet in 1970), the topic of Routing Protocol Convergence Time (time required to detect and reroute traffic in order to handle a link/node failure) has been a top- of-mind issue. A number of protocols and technologies have been developed and deployed at a large scale with the objective of improving overall network reliability. Although such approaches have dramatically evolved, they all rely on a reactive approach: upon detecting a network failure, the traffic is rerouted onto an alternative path.

Automated Policy - Infrastructure as code is a core element of today’s CI/CD pipeline and led to the emergence of Continuous Configuration Automation (CCA) tools, such as Terraform, a leading declarative push CCA solution launched by HashiCorp in 2014. Security is often an afterthought for IaC solutions, and unfortunately, not prioritized nearly as much as it should be. Cisco Secure Cloud-Native provides comprehensive security from development through runtime, and now adds an important tool to eradicate the information gap between developers at the build stage and DevOps at the deployment stage.

Seccomps - An often overlooked way to harden Kubernetes containers’ security is by applying seccomp profiles. Customizing seccomp profiles, in effect, provides a deeply embedded line of defense that adds a layer of protection to your application in case of breach. Ever-increasing interconnections between applications, increased reliance on external service providers, and security risks in open-source images makes restricting seccomp profiles crucial to improving cloud-native security. In our eBook we detail:

Forrester dubbed API Insecurity “the lurking threat in your software.” Understanding API security-specific risks is key to protecting your API. New ways of thinking about API security are emerging. API Security - The Challenges & Latest Developments Using external services through APIs is routinely done to speed up development cycles in today’s microservices architecture. Unfortunately, this potentially creates additional exposure for the application. When calling external services or data from an enterprise application, penetration vectors cloaked in these external sources might expose databases or the application back-end to attacks.

Kubernetes and service mesh are increasingly interrelated, yet their new versions are released separately on different dates. Kubernetes and Istio’s last releases help to get a clearer understanding of the interplay between them and how one affects the other. What’s New in Kubernetes v1.21 and Istio 1.9 Releases and their implications on DevOps? Kubernetes was updated in April 2021 to version 1.21 and Istio Service Mesh has also been updated to version 1.

Admission webhooks are widely used in the Kubernetes world, but people often don’t know how easily a faulty webhook can cause unwanted outages or even worse: bring down entire clusters. In this post, we’ll explore the potential issues caused by webhooks and how you can avoid them. Admission webhooks Before talking about webhooks, we need to mention admission controllers. Admission controllers are “pieces of code that intercept requests to the Kubernetes API server prior to persistence of the object”, acting as middleware in processing objects.

It’s been a while since we posted about our observability solution. We did not abandon observability, we were just busy with the development. First things first: One Eye is now called Cisco MCOM (Multi-Cloud Observability Manager). As you may know, Cisco has acquired Banzai Cloud. With the new environment come some changes as well. For those who are not familiar with One Eye, let’s quickly review the basics. We will cover the current state of our Kubernetes observability solution.

There is a growing realization in academia and industry that that the state-of-the-art in artificial intelligence, namely deep learning, is flawed but remains quite useful in many domains ranging from conversational agents, demand forecasting to recommendations and more. Hinton, who is thought of by many as "the Father of Deep Learning", has stated that in order to achieve a higher level of capability, we “need to start over”. It seems every day a new business leader or luminary, even the normally very positive Woz has viciously attacked deep learning based AI saying there is nothing intelligent about it.

Data networks are complex systems with plenty of counters and other data collection mechanisms built into them. Topological Data Analysis (TDA) techniques can be used to provide system breakdowns information or early warning signals in real-time. TDA can also be used for exploratory analysis and can help data center administrators and customers solve the big data dilemma, the tradeoff between data abundance and data scarcity. Administrators do not need all information from all components all the time.

The AWS Database Migration Service (DMS) helps you migrate databases to AWS quickly and securely. The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database. With the AWS Database Migration Service, you can migrate your data to and from most widely used commercial and open-source databases. The following steps explain how to migrate a database from the AWS MariaDB to AWS Redshift using DMS.

In recent years, the usage of machine learning techniques has grown substantially, both in industry and research. New types of models based on deep neural networks (DNNs) are now becoming ubiquitous in the industry. These models are extremely powerful and have substantially improved the state of the art in many domains. This improvement, however, is achieved at a cost. Compared to classical machine learning algorithms (e.g., Logistic Regression, SVM), these models are more complex, and typically use many orders of magnitudes more parameters.

As the cloud data warehouse in AWS, Amazon Redshift provides seamless integration with other storages, such as Amazon S3. It enables a very cost-effective data warehouse solution, where the user can keep warm data in Amazon Redshift storage and cold data in S3 storage. The user can access the S3 data from Redshift in the same way, retrieving data from the Redshift storage itself. This blog post explains the process for doing just that.

Over the next couple of months, I will introduce you to a three-part series called What in the World of Applications. As applications, API and security take the world by storm, I’ll take you on a journey to better understand the evolution of applications in our everyday lives. My goal is to shed light on how businesses can remain competitive by understanding that better application connectivity and security in the microservices era can directly impact their bottom line, learning how applications can deliver more business value to both your business and customers.

This is a guest post by Jake Walden. Under Armour at ScaleX Spring 2021 On May 4, Day 0 of KubeCon + CloudNativeCon EU/Virtual 2021, Cisco hosted the 2nd edition of ScaleX – a full day of sessions that explore what building for scalability and reliability means for the modern cloud native developer. This second edition of ScaleX featured a full day of tech talks, deep dives, and use cases from the people building, operating, and maintaining reliable cloud native systems at scale.

This is a guest post by Kevin Coss. This post was originally published in the University of Minnesota’s Inquiry blog on research and innovation A new collaboration between University of Minnesota researchers and Cisco Systems seeks to advance cutting-edge technologies that transform the way people access, manage, and protect data. Cisco—which develops, manufactures, and sells networking hardware, software, telecommunications equipment, and other high-technology services and products (like WebEx)—has funded six projects at the University and plans to fund more in the near future.

Topology – The Stratosphere of Human Thought In the previous episode, we raised the need in networking to use methodologies that enable data mining services in complex environments and allow getting insights directly from data. To this aim, we will introduce in this post the concepts behind Topological Data Analysis. First, let’s introduce topology. For a network engineer, a topology is a physical configuration of a network that defines how devices are connected.

It is no wonder companies are taking stringent measures to make sure they are fully compliant towards EU’s General Data Protection Regulations (GDPR) which protects privacy of Personally Identifiable Information (PII) of EU residents, a lot more stricter regulations are coming sooner than later such as California Privacy Right Act in 2023, SAFE DATA Act by the end of 2021. Hence protecting PII is becoming a matter of paramount importance to businesses.

Billing is core to the smooth operation of any business, and an organized AWS account structure using the AWS Organizations Service ensures that your billing processes run smoothly. This post walks through how the AWS Organizations Service can be used to simplify account-based billing for organizations. The image below shows a sample AWS account structure made using the AWS Organizations Service. Once configured, after going to the AWS Billing console, the user will see the billing information for each account added.

Swimming in Sensors and Drowning in Data Modern sensor networks and communication networks provide large sets of operational data, including information per system, subsystem, feature, port, or even per packet. A network device, like a router or a switch, offers millions of multivariate time series that portray the state of the device. But the fact that data is available does not mean that it is easily consumable. Indeed, the reality is that network operations produce a lot of data but at the same time they are starving for insights.

Quantum computers (QC) are machines that we can program their quantum nature to do computation. Algorithms for QCs have been invented showing these machines can solve some problems more efficiently than classical digital computers. Heralding a new computing paradigm for the 21st century, the development of QCs has moved from academic labs into industrial settings over the last few years. Aligned with that, a business ecosystem has been formed to connect the technology developers to the potential end-users, now almost in all sectors of industry.

With the global pandemic increasing remote work and forcing the acceleration of enterprise digital transformation, IT leaders are increasingly turning to low-code platforms. According to Gartner, Inc., the worldwide low-code development technologies market will grow 23% in 2021, and it’s not hard to understand why. By simplifying the application development lifecycle, low-code platforms can increase speed-to-market. They also address the shortfall of niche and in-demand developer skillsets by opening development up to a broader pool of technical talent.

By External Authors: Kenneth R. Fleischmann, Sherri R. Greenberg, and Stephen C. Slota Given the life-and-death stakes of AI innovation, it is critical for designers to reflect and to seek input from experts in other domains, such as the legal and policy domains. In our research, funded by Cisco Research’s Legal Implications for IoT, Machine Learning and Artificial Intelligence Systems program, we interviewed 26 experts across these three domains, including eight AI researchers, eight legal scholars and practitioners, and ten policy scholars and practitioners.

AWS DataSync is an online data transfer service that simplifies, automates, and accelerates the process of copying large amounts of data to and from AWS storage services over the Internet or AWS Direct Connect. DataSync can be particularly useful when migrating large amounts of data between clouds or from on-premises systems to the cloud. With DataSync, you can copy data between Network File System (NFS) or Server Message block (SMB) file servers, Amazon Simple Storage Service (S3), Amazon Elastic File System (EFS) and Amazon FSx for Windows File Servers.

Last week (April 29th) the MITRE org released the ATT&CK matrix for Containers. The release marks the culmination of a research project investigating the viability of container-related techniques into an ATT&CK matrix. Based on extensive community feedback and contributions (which I was happy to take part in) a new ATT&CK for Containers matrix was created and published. In Cisco, we’re working hard to support and contribute to this effort.

What Is Media Streaming Mesh? Media Streaming Mesh is a new concept for supporting real-time applications (such as media production and multiplayer online gaming) in Kubernetes. The goal of Kubernetes is to provide a “platform for automating deployment, scaling, and operations of application containers across clusters of hosts”. Most applications deployed in Kubernetes today are web-based, and so much of the effort around networking in kubernetes is optimised for web applications.

Istio 1.9 contains some exciting new features and numerous enhancements. The core focus of the release, however, is to increase operational stability for production users. In this post, we’ll review what’s new in Istio 1.9, and highlight a few potential snags to look out for when upgrading to the latest version. Istio 1.9 changes High impact changes (can cause issues when upgrading the mesh): Supported Kubernetes versions changed EnvoyFilter XDS v2 removal Service tags added to trace spans Under the radar changes (smaller changes, but which we believe are interesting/useful):

Photo by Laura Olsen on Unsplash When I first heard about this position and learned that it entailed writing blog posts, little did I know that I would be introduced to an expansive breadth of topics and given the autonomy to explore often overlooked questions of the latest trends in the cloud-native ecosystem. I’ve had the amazing opportunity to work directly with and learn from Dominik Tornow, Principal Engineer at Cisco, as an intern in the Cisco Emerging Technologies and Incubation Group (ET&I).

In 2016, I worked on a research project called “Exploring Reality” or ExR for short. The idea was to provide the ability to do 360-degree security monitoring of public areas without compromising people’s privacy. For example, imagine a playground where children are playing and people are walking around. By placing cameras that continuously monitor the area, we basically violate those people’s privacy. Anyone with access to the security camera recordings can see everything each one of those children and people are doing.

This post will be short, because it really is that simple. My team builds platform services that underlie the multiple projects within the Emerging Technology and Incubation team at Cisco. Our mission is to build ‘paved roads’ for developers; we create common reusable patterns and services that any engineer or venture needs to deliver an application. Recently, one of my peers pinged me via WebEx Teams and pointed out that someone on my team had just announced the expansion of our CI build environment with the addition of “slaves 10-15”.

Banzai Cloud was recently acquired by Cisco and some of the product names will be changing. The Backyards product is being renamed to Cisco Service Mesh Manager. We have continued to use the Backyards name in this blog post to maintain continuity with prior blog posts. Future blogs will fully adopt to the new naming. In production systems, you must react quickly on user-facing issues if you want to maintain high customer satisfaction.

As a member of Cisco’s Future Product Innovation team, I recently led a technical investigation focused on using natural language to configure a network management system (NMS). We wanted to help non-technical personnel carry out various types of network configurations in an intuitive and human-friendly way, using natural language (NL) in combination with speech-to-text. Our team consisted of a Cisco Fellow engineer, a senior software engineer and me, serving as the technical lead of the project.

Introduction During a recent Hackathon I had a task to use my PC webcam to scan and detect networking ports. At the beginning, I searched open-source projects but I couldn’t find a similar one. Fortunately, TensorFlow is one of the most popular open-source SDKs for developers to create custom models and the latest version of TensorFlow2 also supports Nvidia GPU to speed up the large dataset model training time. With that, I am able to train my custom model and detect electronic device ports from PC webcams as shown in Figure 1 below.

In the age-old waterfall model, discrete amounts of time and resources were allocated for quality testing after a product was stabilized and the majority of the development process was completed. As we adopt agile methodologies, the software development life cycle (SDLC) can make it challenging to fit quality testing and its expectations into the overall process. Additionally, today’s continuous improvement methodology — where development, DevOps and QA collaborate closely — calls for better “lines of demarcation” between the deliverables, expectations and outcomes expected of each role.

Kubernetes API Security - The Challenges & Best Practices In a Kubernetes environment, the control pane core element is the API server. It’s an HTTP API that manages all communication between your application’s different parts, from cluster segments to external components and end-users. Kubernetes API also allows developers to query and modify Kubernetes API objects (i.e., Pods, Namespaces, ConfigMaps, Events, and more), so it’s critical to prevent any gaps in the API security.

Kubernetes Security - What is the TARA report? And why should DevOps pay attention to it? Kubernetes security is key. A classic attack starts when a container vulnerability is exploited to perform actions on the container that has been breached. The cause for concern is greatly increased when the attacker obtains additional permissions to access the host and gain access to further containers connected to the same host. As the attack progresses the attacker reaches more nodes and can abuse cluster resources or even access the organization’s crown jewels.

The Kubernetes Networking Model guarantees that all Kubernetes Pods on a Kubernetes Cluster are able to communicate, that is, able to exchange messages. However, the Networking Semantics change in the presence of Networking Policies from Default Allow for all Pods to Default Deny for selected Pods with Explicit Allow: Default Allow The Kubernetes Network permits all incoming and outgoing messages to and from any Pod per default.

It is safe to say that none of us could have predicted the pandemic in 2020. The application-first digital economy and future of work that was slowly taking shape over the past few years, suddenly got a jolt of adrenaline. Before the pandemic hit us, it was expected that 50% of companies polled by the World Economic Forum expected that automation, AI, robotics and remote work capabilities would lead to some reduction in their current workforce and require significant re-skilling of their future workforce.

Understanding and Applying Kubernetes Pod Security Policy Kubernetes Pod Security Policy (PSP) is a central part of the Kubernetes Security Context. Implementing robust container security is only one of the required steps. Pod Security Policy allows granular security configuration at cluster level. Pod Security Policy default values are typically open to maximize flexibility, so the responsibility to calibrate policies to tighten security falls on DevOps. Though Kubernetes provides an excellent security infrastructure it is humanly almost impossible to make sure all PSP profile entries will ensure secure configuration and hardening of the container deployment.

Kubernetes Networking is a core abstraction of Kubernetes. In a nutshell, the Kubernetes Networking Model guarantees that all Kubernetes Pods on a cluster are able to communicate. In addition, on top of the Kubernetes Networking Model, Kubernetes provides additional core abstractions, Kubernetes Services and Kubernetes Ingress. Using a systems modeling approach, this blog post will explore Kubernetes Networking. Here, we will develop a concise mental model to reason about Container-to-Container and Pod-to-Pod communication.

Integrated Services have long been a key feature of the Banzai Cloud Pipeline platform, making complex infrastructure components ready for use with just a few clicks in the UI or a single command from the Banzai CLI. Some of the most important Integrated Services: Monitoring and Logging DNS Secrets Management Vulnerability scanning There’s a lot of value already added by the current generation of Integrated Services: We can enable and disable complex features on demand through the Integrated Service API easily, without any extra tooling from the user We can upgrade these Services to their latest version with a single click, as long as the upgrade doesn’t require special steps when migrating We can leverage existing Helm charts and use Helm libraries for deployment in the background The current architecture and features have served us well, but there are a few shortcomings we’ve been wanting to address.

Infrastructure as a Service, Infrastructure as Code, or recently Infrastructure as Data — certainly infrastructure comes in many forms, shapes, and sizes. Yet, even though these terms sound deceptively similar and deceptively simple, they denote fundamentally different and intricate concepts. After Infrastructure as a Service — or Cloud Computing — entered the stage, the software engineering community had to determine just out how to work with Infrastructure as a Service.

Realtime Vulnerability Management for Containers: Time to Step Things up! Securing Kubernetes environments remains a major challenge. In recent years we’ve seen a number of container vulnerabilities, data breaches, and plain misconfigurations. In the SANS “Endpoint Protection and VResponse” survey from 2018, ‘‘‘42% of respondents indicated that at least one of their endpoints had been compromised’'’. A lot of development and devops teams create their container-based infrastructure locally before pushing into a repository.

When implementing a microservice-based architecture on top of Kubernetes it is always hard to find an ideal alerting strategy, specifically one that ensures reliability during day 2 operations. Modern Kubernetes-based deployments - when built from purely open source components - use Prometheus and the ecosystem built around it for monitoring. The unparalleled scalability of Prometheus allows you to measure (and store) most of the data reported by both workloads and cluster components.

The Cisco Emerging Technologies & Incubation team is initiating a comprehensive program to support and investigate Quantum Compute and Communication Science & Technology. The team has identified a set of initial topics to be studied, in close collaboration with selected academic partners that have a similar interest and complementary expertise. A focus on Quantum Communication Quantum entanglement implies a strong correlation between the states of the individual components of a system regardless of their physical separation.

Excellent question! After all, does simply adding a G to AI make that much of a difference? The fact is that the G in Artificial General Intelligence is as unnecessary as the A in Artificial Intelligence. Intelligence is intelligence, its origins are of secondary importance. However, we can’t ignore the fact that Artificial Intelligence has become, at least in many circles, practically synonymous with ML/DL and statistical learning. Hence, in order to clearly distinguish today’s pattern recognition technology from “thinking machines” that are comparable to the human mind in many non-trivial aspects, the term AGI is helpful.

Understanding and Applying Container Security In a microservices architecture, maintaining container security at all stages is a primordial requirement and needs to cover many facets. ‘‘‘The container’s inner components’’’ – a container contains images, imported or custom-created, common tools, compilers and RIBs, framework, and app code. Each of these components’ vulnerabilities needs to be managed at every step of the container’s lifecycle to ensure container security. ‘‘‘The container’s communication system’’’ – a container interacts with the host’s operating system and with other containers on the same host.

Kubernetes Security for Exposed Workloads A successful breach of an exposed workload carries the risk of spreading laterally, identifying targets, and ultimately gaining access to the organization’s crown jewels Kubernetes security framework provides default security policies and guidelines from pod level, with Pod Security Policies (PSP), to cloud level. These policies need to be configured, but even perfectly calibrated security policies do not provide optimal security at runtime. The number of system workloads, both proprietary and imported, can quickly run into thousands.

We are experiencing a once-in-a-century global pandemic, which has had a profound impact on all of humankind. Even people who have not been infected have seen unprecedented disruptions to their everyday lives, with overburdened health systems, business shutdowns leading to job losses, and food shortages, among others. This pandemic is a clear wake-up call, reminding us to be better prepared as a society to deal with these challenges. Connecting researchers and developers Earlier this year, at Cisco Research, we wanted to step up to the challenge of pandemic preparedness by soliciting research proposals that address both the science and technology aspects of problems around detection, mitigation, and analysis of COVID-19.

In this post, we’ll describe how to provision your Apache Kafka cluster using either Supertubes or the Koperator so that it is accessible by external client applications that run outside the Kubernetes cluster. There are two methods that allow you to externally publish your Kafka cluster: using LoadBalancer type services using NodePort type services The LoadBalancer method is a very convenient way to publish your Kafka cluster, as it allows you to forgo setting up a Load Balancer, provisioning public IPs, configuring routing rules, etc.

Mitre ATT&CK framework: What is it and does it work for K8s environment? Faced with the proliferation of complex attacks, cyberdefense needs to switch to Tactics, Techniques, and Procedures (TTPs) focused strategies. Defined as the toughest obstacle to overcome in Bianco’s pyramid of pain, TTPs based defenses force attackers to desist from simply using available attack vectors and tools, compelling them to continuously invent new behaviors. MITRE ATT&CK is the most advanced taxonomy of TTPs available today and provides cyber defenders with interactive matrices invaluable in to defining effective defensive strategies.

It’s been a while since we published a new release of One Eye. That doesn’t mean we haven’t been working on it, quite the opposite! We’ve been busy making lots of improvements and adding features. This blog will highlight the many changes made since version 0.3. Let’s start with some great news! Logging Operator (responsible for the One Eye logging sub-system) is now the official logging backend of Rancher.

Istio 1.8 has just been released and is one of the best Istio releases so far. The new version contains exciting experimental features, numerous enhancements, as well as deprecations and removals. The core focus of the release, however, is to increase operational stability. In this post, we’ll review what’s new in Istio 1.8, and highlight a few potential snags to look out for when upgrading to the latest version. If you’re looking for a production ready Istio distribution based on the new Istio 1.

Today we’ve launched the 1.5 release of Backyards, Banzai Cloud’s production ready Istio distribution. The vision of Backyards is to become a go-to solution for SREs and operation engineers to observe and control the health of their services and applications. The service mesh acts as the foundation layer that produces the most important network traffic metrics and traces in a unified way. The previous release was the first step towards this goal, and the new, 1.

Ever since its inception, Supertubes has been about asking the question, “What if?” What if we built Kafka on the solid foundation of Istio and leveraged it and the Envoy proxy’s powerful networking capabilities to combine the two into something even more powerful? Today, along with releasing ksqlDB support for Supertubes, we’re engaging in even more speculation about what Istio and Kafka can do, together, to improve our lives. Over the years, Kafka has grown itself into a whole platform with an entire ecosystem built around it, consisting of components like Kafka Connect, Kafka Schema Registry, and Mirror Maker 2.

What is Zero Trust Security? Cisco Cloud Native Security Solution The “Trust no one, always verify!” motto is at the core of the zero trust security model and its popularity is growing as the zero trust framework addresses many of today’s common cybersecurity issues, such as: ‘‘‘Security perimeters are becoming porous’'': as the need for connectivity between systems increases, maintaining a tight security perimeter is increasingly difficult. ‘‘‘Cloud-native and hybrid infrastructure increase risks’'': one of the main advantages of migrating all or parts of a system to the cloud is its agility and flexibility.

We’ve already covered the details of SLO-based alerting for Request Latency (Duration) and Request Error Rate in our previous blog posts. However, if we take a look at Google’s Site Reliability Engineering book, we’ll see that it mentions four golden signals to be used as the basis of any system’s alerting strategy: Latency (or Duration): The time it takes to service a request. Errors: The rate of requests that fail, either explicitly (e.

Logs play several roles during an application’s lifecycle. They help to debug problems - some give us analytical data (like access logs) - and help provide an overview of an application’s state at a given moment. This requires different modes of access and visualization, and, because of the distinct behavior involved in each outcome, we often end up using different tools. For example: for tailing logs live, we use Loki, for complex visualizations we prefer Elasticsearch, and to archive logs, we prefer object storage (like Amazon S3).

We had quite a few Bank-Vaults releases recently where new features arrived in the webhook and operator. This is no coincidence, rather, these features were developed and implemented with the help of the community that has grown up around the project over the past two years. In this blog post, we would like to highlight and discuss some of these improvements (though, this is not an exhaustive list). Docker image registry changes ghcr.

If you’re an active Istio user, then there’s a good chance that Istio’s configuration reference is bookmarked in your browser, and that you’ve read the pages on VirtualServices, and ServiceEntries over and over, but still have to struggle to set up even simple configurations in your mesh. Istio’s custom resource configuration is very powerful and flexible, but infamous for being overly complex. At its best, its YAML consists of lists of lists, cross-references, conflicting fields, and wildcards.

Finding a root cause can be hard. You need to collect data from several (usually disparate) sources. Often, when we debug a cluster, we need the output of its logging Flows to validate the consistency of its Logging stack. The idea in such cases is to peek into the messages transported by fluentd. Let’s take a look at how we might accomplish that. If you’re not familiar with the Logging Operator, please check out our earlier posts.

Hybrid and multi-cloud infrastructure is becoming more and more common in enterprise IT environments, since it provides a superior set of capabilities than what might otherwise be gained from a single cloud. Specifically, it allows us to use different cloud services for different use-cases within a single environment, helping to avoid vendor lock-in. Using both on-prem and cloud environments makes it possible to use the best of both worlds, letting users transition to the cloud at their own pace; it’s very rare for businesses of significant size, nowadays, to exist exclusively on either.

Backyards provides, out-of-the-box, Service Level Objective-based monitoring and alerting solutions for monitoring HTTP/GRPC-based microservice latencies and error rate characteristics. The decision to only provide these Service Level Indicators as part of the base installation was due to the fact that only Istio itself is able to provide the metrics they require. On the other hand, as we had briefly discussed in the Tracking and enforcing SLOs webinar, the SLO-based measurement of systems can not and should not be limited to these three measurements.

Distributed applications Distributed applications have many definitions, but typically they are defined as applications that run on a network of multiple computers. Nowadays, that might well mean an application that runs on multiple Kubernetes clusters that are distributed over a large geographical area. Since such clusters are distributed, they have many benefits - like high-availability, speed, resiliency - but it can make developing, operating, and testing such applications tedious. The Multi-DC feature of Bank-Vaults is just such a distributed application; its testing was not automated and required us to manually test each time an important PR went into master.

We previously covered the basics of burn rates in our last blog post about enforcing SLOs and in our Tracking and enforcing SLOs webinar. Even if the basic concept is clear (if a burn rate is greater than 1, the SLO is in danger), it can be difficult to develop a strategy for alerts based on burn rate values. Of course, when implemented properly, burn rate-based alerting ensures that Service Level Objectives (SLOs) are enforced and met.

One of the main challenges enterprises currently face – whether they have an on-prem infrastructure or are running their workloads in the cloud – is application and network security. Data must be protected at all times, not just when it’s resting in a database or is placed in storage somewhere. Ensuring data protection while its moving means more than just encryption, it also means authentication and authorization. A service mesh is a transparent infrastructure layer that sits between a network and microservices, as such it’s the perfect place to ensure data encryption, authentication and authorization.

tl;dr On August 25th, one of our customers, who was using Supertubes to operate Kafka clusters on Kubernetes on Amazon, began to experience a service disruption in their Kubernetes cluster. Their previously configured Prometheus alerts notified the infra team what services were impacted. It turned out that Amazon was having some instance connectivity issues, in one of the London AZs. No data loss occurred thanks to the properly configured Kafka and Zookeeper clusters, which had both been distributed across multiple zones.

Microsoft rocks the market, announcing OSM. What’s the big deal about another service mesh option? Up until the beginning of this month, life seemed clear. And so did the clear choice for service mesh: Istio. Following on the heels of the Kubernetes acquisition in 2016 by CNCF came the development of Istio by Google, IBM and Lyft, an open source service mesh, and of Envoy by Lyft, an open source edge and service proxy.

Many of you are probably familiar with the moment in Life of Brian when John Cleese lists all the improvements the Roman Empire has brought to Judea then asks, “What have the Romans ever done for us?” I mention this because, one day recently, I was scrolling through twitter when I stumbled onto a treasure trove of users blaming Kubernetes for its complexity, claiming things were better before it existed. So, in the interest of fair play, I’ve decided to write a bit about the glorious past and our mediocre present, and to compare the two.

The last few versions of Istio have been released at a rate of about one every three months. Because of this rapid rate the upgrade of an Istio service mesh is an essential and recurring task to do when using Istio. Upgrading an Istio control plane in a production environment with as little downtime as possible has been one of the many challenges that service mesh operators have routinely faced. This is why the new Istio control plane canary upgrade feature is such a game changer.

Our mission at Banzai Cloud is to make the digital transformation of tech companies to the Cloud Native era as quick and efficient as possible. We’ve had the opportunity to meet a large audience and provide solutions in various use cases from simple ones (single cloud, multi-cluster workloads) to more complex ones (multi cloud, hybrid cloud), and recently we’ve successfully tested our platform in highly regulated environments, like AWS GovCloud.

Andrew Ng: “AI is a superpower” How many years has it been since Andrew Ng started teaching the world via Coursera about the power of machine learning and deep learning? Although I was an early adopter of GPUs and ML technologies over 10 years ago, I thought his course was outstanding and highly motivating. Andrew’s course helped explain the value of low-cost, GPU-based, high performance computing to many executives and engineers better than anything else I knew at the time, way back in the days before “Deep Learning” became a universally used term.

Proxy servers are out of fashion, but you may meet them at some places, or you might even realize that they are a good candidate to solve your problem. We’ll look into the history and the basic workings of HTTP proxies to understand their limitations, the environment where they worked well, and see how they can be best used today. Some networking history Networking protocols are one of the last thing you should expect to significantly change when designing a system or software.

Protecting your Kubernetes platform from Multiple Attack Vectors with the K8SHIELD™ Framework Cisco Cloud Native Security Solution As many know, the MITRE ATT&CK framework provides a threat matrix that guides administrators, developers, DevOps, security teams and others in protecting their networks, systems and endpoints from undesirable access and manipulation. However, this framework currently supports attacks primarily related to these platforms: Linux, Windows, and macOS-based attacks, as well as general cloud (Infrastructure as a Service for example (IaaS)) environments.

Today we’ve launched the 1.4 release of Backyards, Banzai Cloud’s production ready Istio distribution. While Backyards is first and foremost a production ready Istio distribution, the latest release is a shift towards an observability tool and dashboard for SREs and operation teams. The most important features of this release are: designing and tracking service level objectives and corresponding alerting rules seamless canary upgrades of the service mesh control plane support for Istio 1.

Istio 1.7 has just been released and it mostly focuses on improving the operational experience of an Istio service mesh. In this post we’ll review what’s new in Istio 1.7 and highlight a few gotchas to look for when upgrading to this new version. If you’re looking for a production ready Istio distribution based on the new Istio 1.7 release and with support for a safe and automated canary upgrade flow, check out the new Backyards 1.

Running a computing system at scale is hard. The goal of the whole Cloud Native transformation is to make it easier, but it still requires a deep expertise in observability and alerting tools to track the state of the infrastructure and the services. And it’s not only the tools, but the questions that arise at scale. Practice shows that running a complex system with a 100% reliability target is unrealistic. But in that case, how do you determine that certain level of errors that you can live with, or when do you need to trigger alerts to possibly wake an engineer up in the middle of the night?

Navigating the Service Meshes Map A “Service Mesh” is an infrastructure layer regulating the interactions and relationships between applications or microservices. Rather than a source of fundamentally new features, it provides a repackaging of functionalities such as request-level load balancing, circuit-breaking, retries, instrumentation, and others. When developing cloud-native or hybrid applications, DevOps increasingly relies on service meshes to abstract application network functions from the code. Born as a facilitator for orchestration in the wake of Kubernetes and other container technology, service meshes are rapidly becoming an indispensable tool for containerization.

A couple of weeks have passed since the release of One Eye version 0.3, which introduced some powerful UI and backend improvements. Our goal with One Eye is to provide a singular vantage point from which you can see what’s happening inside your Kubernetes cluster, centralizing our observability tools into one place and onto one dashboard. However, we’re not trying to reinvent the wheel by re-implementing observability, but to organize and configure existing tools to create a seamless experience.

Companies providing services for government, healthcare, or finance sectors must ensure that their products adhere to government approved security requirements that ensure the protection of sensitive data. The Federal Information Processing Standards (FIPS) outlines the rules for cryptographic modules employed as mechanism that implement security. A cryptographic module is a combination of hardware and/or software that handles cryptographic functions such as encryption, decryption, digital signatures, authentication techniques and random number generation.

Update: beta.banzaicloud.io has been shut down on November 3, 2020. You can use the Banzai Cloud Pipeline Enterprise Preview platform at try.pipeline.banzai.cloud instead, with extended feature set and the same conditions. Pipeline is Banzai Cloud’s Kubernetes container management platform, which allows enterprises to develop, deploy and securely scale container-based applications in multi- and hybrid-cloud environments. For the past two years, we have been running a freely available, hosted and managed Pipeline environment at beta.

Leveraging a Cloud Native technology stack, Banzai Cloud Supertubes is the ultimate deployment tool for setting up and operating production-ready Kafka clusters on Kubernetes. While Supertubes installs and manages all the infrastructure components of a production-ready Kafka cluster on Kubernetes (like Zookeeper, Koperator , Envoy, etc) it also includes several convenience components, like Schema Registry, multiple disaster recovery options, Cruise Control and lots more. The list of Supertubes features is very long and covers everything you need to self host and run a production-ready Kafka cluster on Kubernetes.

The API gateway pattern has been used as a part of modern software systems for years. A different concept, service mesh, has also emerged over the last couple of years. They share some similarities in their feature set, and service meshes soon started to introduce their own API gateway implementations. In this post, we’ll discuss the Istio ingress gateway, from an API gateway perspective. We’ll examine its feature set compared to typical API gateway features.

Government organizations and institutions have similar requirements and goals regarding their IT infrastructure as commercial enterprises: it must be flexible enough to adapt to the changing needs of the organization, easy to maintain and monitor, scalable to meet the changing workload requirements, highly available and resistent to errors, and of course secure to protect the various sensitive data such organizations must process. In addition, they must meet the requirements of various national and state-level regulations, like the Federal Risk and Authorization Management Program (FedRAMP), the Department of Defense (DoD) Cloud Computing Security Requirements Guide (SRG), the Federal Information Security Management Act (FISMA), and other legislation.

Recently, we blogged about certificate management on Kubernetes. Today, we’ll be returning to that topic, but we’ll be focusing on the differences an Istio service mesh makes. The primary difference is the method of solving the ACME HTTP-01 challenge. Solving this challenge involves routing an HTTP request from the ACME server (the Certificate Authority) to the cert-manager challenge solver pod. Cert management with Istio Ingress support As we saw in our previous blog post, you can route such a challenge request by using a Kubernetes Ingress gateway.

Can You Detect Kubernetes Runtime Vulnerabilities? Container images sometimes suffer from vulnerabilities ranging from negligible to critical. CVE-2019-11249, for example, is a Kubectl copy operation that, together with a malicious tar binary in the downloaded container, enables hackers to replace or create files on users’ machines even if the location is outside the destination directory of Kubectl copy operation. It can lead to a complete hostile takeover of the application pod, the Kubernetes node, and, in some cases, even the entire cluster.

When exposing services it’s generally a good idea to follow the industry standard and use HTTPS protocol. HTTPS requires a certificate issued by a trusted third party, called a Certificate Authority (or CA for short). There are several ways to acquire one, but a simple and effective method is to use Let’s Encrypt (a CA) by way of the ACME protocol. The ACME protocol is a communication protocol for interacting with CAs that makes it possible to automate the request and issuance of certificates.

Istio claims that it helps to connect, secure, control and observe services. We’ve blogged a lot about connect, even more about observe, and also had a few articles about secure. But so far, we haven’t really touched control. This post tries to fill that gap, and discusses Istio’s access control model, or more specifically AuthorizationPolicies. Architecture Istio Authorization can be used to enforce access control rules between workloads. It basically answers the question: who can access what, under which specific conditions?

It may not be a well-known fact, but a Kubernetes API server can proxy HTTP connections between a client and any service running on a cluster. A simple kubectl command exists that allows it: $ kubectl proxy Starting to serve on 127.0.0.1:8001 We use this for demo purposes or when we don’t want to expose APIs publicly, but need to access them from our computers. Moreover, this proxy can transport WebSocket connections.

Applications running in Kubernetes Pods are authenticated against the Kubernetes API with their corresponding ServiceAccount tokens. These JWT tokens are usually mounted into containers as files. JWT tokens are signed by the Kubernetes cluster’s private key, and can be validated only with the TokenReview API. This API is not widely recognized and, to access it, external systems must first authenticate against Kubernetes to review ServiceAccounts. This configuration and access review process is considerably more complex than necessary, not to mention that it leaves out widely accepted standards like OIDC.

A generation of system engineers has grown up who’ve never had to go to a data center as part of their job. It’s easy to forget that while cloud computing offers an abstraction over physical servers, they do still exist behind the scenes — more of them than ever before. There are hardly any companies anymore that host all of their IT infrastructure on premises, but many enterprises have reasons to continue managing physical servers as part of their infrastructure.

Companies frequently use proxies to act as a link between an internal network and the Internet. This is often frustrating for employees, even non-IT ones, when they can’t access a specific site from the company network. For engineers it’s even more obnoxious, since they have to configure all kinds of compute infrastructure to connect to external networks via these proxies. It’s debatable if this is the best way to harden corporate network security, but it’s still the most widely spread method to restrict outgoing traffic.

tl;dr: The Supertubes approach to handling Kafka ACLs in Kubernetes provides a clearer way of seeing what’s actually happening by introducing a logical separation of ACL components under the names: KafkaACL, KafkaRole and KafkaResourceSelector. That way we get reusable parts that help maintain the system in the long term, allowing us to handle ACLs with a declarative approach, and overcoming the difficulties inherent in handling ACLs in a Kubernetes environment.

More than a month ago, we announced One Eye, the observability tool for Kubernetes. This has been an ongoing project, and we release a new version of it about once per week. We’ve gathered the features included in those updates here to keep you up to speed. If you are not familiar with One Eye, check out our introductory blog post or browse the official documentation. Who is One Eye for?

Thanks to the gradual maturation of Istio over its last few of releases, it is now possible to run control plane components without root privileges. We often use Pod Security Policies (PSPs) in Kubernetes to ensure that pods run with only restricted privileges. In this post, we’ll discuss how to run Istio’s control plane components with as few privileges as possible, using restricted PSPs and the open source Banzai Cloud Istio operator.

One of the many best-practices for operating Kubernetes clusters is to frequently perform Kubernetes version upgrades in those clusters. This insures that you’ll be running your workload with a version of Kubernetes that contains the latest security fixes, stability improvements, and features. There are two ways of doing this: manually with, for example, kubeadm when you’re running on bare-metal and, if you’re using one of the many cloud-provider Kubernetes solutions, the assisted way They all have their benefits and downsides.

Last autumn we open-sourced the dast-operator which helps checking web applications for security vulnerabilities. The first version was able to initiate a simple dynamic application security test based on custom resources and service annotations. To read more about the first version please check our Dynamic application security testing in Kubernetes blog post. Today we are happy to announce that we are now extending the operator capabilities with a few new features to facilitate testing APIs as well.

We recently wrote a very detailed blog post about Kubernetes Ingress. It discusses the various ways of how to route traffic from external sources towards internal services deployed to a Kubernetes cluster. It mostly talks about basic ingress options in Kubernetes, but briefly mentions Istio as a different approach. In this post we examine Istio’s gateway functionality more thoroughly. We discuss the ingress gateway itself that acts as the common entry point for external traffic in the cluster, we take an in depth look into the configuration model, and we finish by talking about the advantages of using Backyards, Banzai Cloud’s production ready Istio distribution.

At Banzai Cloud we always strive to make things simpler and to make complex services available to our customers. We try to reduce the complexity of setting up components and services by automating as much setup as possible - to expose these for users in a transparent, easy to understand manner. This effort led us to introduce integrated services to the Banzai Cloud Pipeline platform. We have already written about what integrated services are, and we also have described a few of them, like automated public DNS management for Kubernetes clusters and cluster expiration.

We often find ourselved required to route traffic from external sources towards internal services deployed to a Kubernetes cluster. There are several ways of doing this, but the most common is to use the Service resource, or, for HTTP(S) workloads, the Kubernetes Ingress API. The latter is finally going to be marked GA in K8s 1.19, so let’s take this opportunity to review what it can offer us, what alternatives there are, and what the future of ingress in general could be in upcoming Kubernetes versions.

Backyards is Banzai Cloud’s widely popular production ready Istio distribution, which helps to install, upgrade, secure, operate, and observe an Istio service mesh. In this blog post, we will discuss the high-level architecture overview of Backyards, three different ways to start using Backyards. Introduction If you’re not familiar with Backyards, and want to know why we decided to build this product, we suggest reading the blog post about the first major release.

Helm version 3 has been out officially for some time (release blog post was published on Wed, Nov 13, 2019). We’ve been using Helm since the early days of Kubernetes, and it’s been a core part of our Pipeline container management platform since day one. We’ve been making the switch to Helm 3 for a while and, as the title of this post indicates, today we’ll be digging into some details of our experience in transitioning between versions, and of using Helm as a Kubernetes release manager.

Today we’ve launched the 1.3 release of Backyards, Banzai Cloud’s production ready Istio distribution. Along with some performance improvements and bug fixes, the 1.3 release is centered around three main topics: a brand new gateway management feature, a new declarative installation and configuration method, and support for Istio 1.6. If you’re not familiar with Backyards, and want to know why we decided to build this product, we suggest reading the blog post about the first major release.

Istio 1.6 is around the corner and it continues where 1.5 left off: it simplifies the architecture and improves the operational experience. In this post we’ll review what’s new in Istio 1.6 and dig deep on the important changes. The Backyards 1.3 release is already based on Istio 1.6. If you are interested in getting Istio up and running with Backyards make sure you register for the webinar! Istio 1.

If you’ve been reading our blog you already know that we’re passionate about observability. We are convinced that the key to operating a reliable system is to know what happens where, and the correlated ability to rapidly dissect issues as they emerge. In previous posts we’ve gone over the base components of our suggested stack, which includes Prometheus, Thanos, Fluentd, Fluentbit, and many others. We’ve created several tools and operators to ease the management of these components, like the Istio operator, Logging operator, Thanos operator as well as using some other very popular operators, like the Prometheus one.

At Banzai Cloud we write Kubernetes tools that, no surprise, require other tools to get installed. These tools handle resource lifecycle operations like the installation, upgrade, and removal of various components that need to be integrated together. Kubernetes Operators solve all these and even more for a given application’s domain, but if the focus is simply on installation, upgrade and removal of a set of components - which are in our case operators themselves (operating operators), then we need a tool with different characteristics at hand.

Network perimeter security is a focal point of any network admin. When it comes to network perimeter control, our first thought is always inbound security (ingress). However, securing what can leave the network (egress) and where is equally important. In this post, we’re not going to go into the theoretical details of discussing why, exactly, controlling egress traffic is so important or where possible exploitations points are, because there are quite a few posts already.

A few days ago we published a blog post about how to use Apache Kafka’s proprietary authentication and authorization (ACL) with Kubernetes Service Accounts via Supertubes. If you skimmed that post or have actually used it with Supertubes, you might think that this is (as Shakespeare put it) much ado about nothing, as things just work out-of-the-box. As the CTO of Banzai Cloud, and someone involved in the details of our product development, I can confirm that this has been our single biggest engineering push of 2020 so far (spoiler alert - that record won’t last long: there are three or four parallel efforts already challenging it).

Bank-Vaults already supports multiple KMS alternatives for encrypting and storing unseal-keys and root-tokens. However, during bootstrapping and configuring sometimes you need to source other secrets to configure Vault securely. In this post you will learn how to do that with the help of the valuable contributions of Pato Arvizu. Thank you! For those unfamiliar with Bank-Vaults, let’s do a quick recap. Bank-Vaults is a Vault Swiss Army knife, which makes enterprise-grade security attainable on Kubernetes.

Running Kafka on Istio with mTLS is, in of itself, an interesting topic, but before we can talk about how Banzai Cloud’s Supertubes allows us to do that, let’s take a step back and look at how SSL works in Kafka. Maybe then we can answer the question, why do we need Kafka in Istio with mTLS at all? Supertubes is Banzai Cloud’s Kafka as a Service, which runs on Kubernetes inside an Istio service mesh.

Pipeline is Banzai Cloud’s Kubernetes container management platform, which allows enterprises to develop, deploy and securely scale container-based applications in multi- and hybrid-cloud environments. While one of Pipeline’s core features is to automate the provisioning of Kubernetes clusters across major cloud providers, including Amazon, Azure, Google, Alibaba Cloud and on-premise environments (VMware and bare metal), we strongly believe that Kubernetes as a Service should be capable of much more. Pipeline has been a key enabler of multi- and hybrid-cloud strategies, providing both a unified cockpit for operations and a high level of workflow and workload portability for developers across major cloud and datacenters - in four different ways

Envoy is a high performance, programmable L3/L4 and L7 proxy that many service mesh implementations, such as Istio, are based on. At the core of Envoy’s connection and traffic handling are network filters, which, once mixed into filter chains, allow the implementation of higher-order functionalities for access control, transformation, data enrichment, auditing, and so on. You can add new filters to extend Envoy’s current feature set with new functionalities. There are two ways to go about doing this:

One of the Istio service mesh’s most popular and robust features is its advanced observability. Because all service-to-service communication is routed through Envoy proxies, and Istio’s control plane is able to gather logs and metrics from these proxies, the service mesh can provide us with deep insights about the state of the network and the behavior of services. This provides operators with unique ways of troubleshooting, managing, and optimizing their services, without imposing any additional burdens on application developers.

Kubernetes “kubectl cp” Command to Jeopardize Cloud-Based Host Instances A security issue was recently discovered with the Kubernetes kubectl cp command that could potentially enable a directory traversal to replace or delete files on a user’s workstation or cloud instance. This is a high severity issue, and it is recommended to upgrade kubectl to Kubernetes 1.11.9, 1.12.7, 1.13.5, or 1.14.0. See CVE-2019-1002101 and the fix. The latest security issue was initially found earlier this month by Ariel Zelivansky, a security researcher at Twistlock.

One of the core feature of Banzai Cloud’s container management platform, Pipeline, is the capability of building hybrid clouds with ease. The most important reason behind the introduction of our own CNCF certified Kubernetes distribution, Pipeline Kubernetes Engine (PKE) was to provide our customers a Kubernetes environment that behaves consistently across public cloud providers, on-premise, and the combination of those. A single approach does not cover all use-cases, so the Banzai Cloud Pipeline platform provides four different ways to build and use hybrid clouds.

When something goes wrong in your mesh, the first thing you’ll probably notice is an alert about your services: error rate or latency is increasing. But it’s only a symptom and the real root cause can be a whole bunch of different things, like underlying Kubernetes problems, application bugs or node failures. This blog post shows you how to track such an issue and find the root cause: in this example, a misconfiguration in a Kubernetes cluster.

Generally speaking, code generation is the process of producing code from some sort of abstract description. Although this is not a very expressive description, almost everyone has some sort of understanding of code generation. For example, in the cloud native ecosystem generating code from Protobuf or OpenAPI descriptors is quite common. Code generation, however, is much more than that. Besides specific, (very) high-level definition languages, program code is also such abstract description.

It’s no news that for quite a while our Apache Kafka on Kubernetes take, Supertubes has been happily running inside an Istio-based service mesh, in both single or multi-cluster setups across hybrid clouds. While we have touched on several aspects of the advantages Istio gave us, this post’s aim is to collect some of the issues, cornerstones and benefits. We see the service mesh as a key component of every modern Cloud Native stack.

In a perfect world, resources are infinite and readily available. But as we all know, things in the real world are never quite so simple. The struggle for resources is much the same in the world of computer programs, where CPU cycles, memory and storage space, or network bandwidth can easily become scarcities even when using the latest generation of devices and technology. Time-sharing, virtual memory and paging, logical volumes… These techniques are used by most operating systems today without the average user or software developer having to think about them, working hard to make the limitations of hardware disappear.

Kubei : A Kubernetes Runtime Vulnerabilities Scanner Today, Cisco Cloud Native Security is happy to announce Kubei, an open source Kubernetes runtime vulnerabilities scanner tool, as a contribution to the developer and DevOps community. The tool is available on GitHub. The idea behind Kubei is to offer the community a unique tool to help DevOps/SRE teams solve the pressing need for Kubernetes runtime scanning. While there are many tools available for vulnerability scanning, not all tools are created equal.

Today we are happy to announce the 1.2 release of Backyards, Banzai Cloud’s automated and operationalized service mesh product built on Istio. This is an announcement post describing the new features of [Backyards 1.2]. If you’re not familiar with Backyards yet, and want to know why we decided to build this product, we suggest reading the blog post about the first major release. Want to know more? Get in touch with us, or delve into the details of the latest release.

It’s been some time since we open sourced our Koperator , an operator designed from square one to take advantage of the full potential of Kafka on Kubernetes, and have built Supertubes on top to manage Kafka for our customers. One of the most requested enterprise feature has been the implementation of rolling upgrades. Built on the Banzai Cloud’s Koperator , Supertubes adds support and orchestrates these upgrades. Accordingly, in today’s blog we’re going to take a deep dive into the technical details of how the operator handles an update and how you can use Supertubes to seamlessly upgrade to the newest 2.

From the beginning, Bank-Vaults has been one of the core building blocks of Pipeline - Banzai Cloud’s container management platform for hybrid clouds. Today we are happy to announce the release of Bank-Vaults 1.0, and the official launch of Bank-Vaults as a product with commercial support. Additionally, we have taken the step of adding Bank-Vaults support for hardware security modules, usually abbreviated as HSMs. Bank-Vaults 1.0 Bank-Vaults was first released two years ago as a Vault operator for Kubernetes, a CLI tool and a Go library.

As the recent CNCF survey suggests (page 7), Istio is one of the most popular service mesh technologies on the market today. The biggest obstacle in Istio’s production adoption so far has probably been that the complexity and domain knowledge required to operate a mesh was too high. The Istio community has realized this and has taken multiple steps to improve the usability and reduce the complexity of Istio. In Istio 1.

In one of the earlier Kafka disaster recovery on Kubernetes with CSI post we discussed how brokers of an Apache Kafka clusters can be backed up and restored leveraging Kubernetes volume snapshots. While this solution provides a good enough disaster recovery option (and a super quick recovery), it doesn’t help when the entire Kubernetes cluster hosting the Kafka cluster is lost. In this post, we show you how to: back up a Kafka cluster to a remote Kafka cluster running on a separate Kubernetes cluster, and also how to recover the lost Kafka cluster with Supertubes.

Today we are happy to announce a new release of the Banzai Cloud logging operator. It’s been a long time from the first commits till today, and is always nice to look back, learn and reflect on the evolution of the project. The first major release, June 2018 This was the very first release, and among the first operators we made. The operator pattern was pretty new, and the goal of the first logging operator was fairly simple - automate the manual fluent ecosystem configurations we were doing for our customers with the Pipeline platform.

Istio, and in general the service mesh has changed the way of service to service communication (from dumb pipes and smart endpoints to sidecar-to-sidecar). While this has lots of benefits, it can increase the complexity of troubleshooting microservice to microservice communication. One of the typical places developers are checking when comes to troubleshoot is the Envoy sidecar proxy container’s access logs, both on the source and destination side. With the increased number of microservices deployed to distributed environments, this process can be tedious, and it’s very inconvenient trying to pair source and destination access log pairs.

There has been a lot of talk about multi- and hybrid-clouds over the past years. Some cloud vendors see these trends as a threat, others look at them as an opportunity. We think that beneath the buzzwords lie some very important use-cases driven by the needs of enterprises and SaaS providers. However, delivering and operating multi- and hybrid-clouds has been too complex for most organizations so far. One major area of focus for Banzai Cloud has been hybrid clouds, to create and automate a seamless and consistent operational experience for a concept that has a lot of underlying complexities.

One of the most popular feature of Bank-Vaults, the Vault swiss-army knife for Kubernetes is the secret injection webhook. With the growing popularity of Istio, recently the most requested feature was to support for running Bank-Vaults alongside Istio. We are big fans of Istio (a year ago we open sourced an Istio operator) and we have built an automated and operationalized service mesh, Banzai Cloud Backyards. As both components (Bank-Vaults and Backyards) are part of our hybrid cloud container management plaform, Pipeline, we went ahead and made them work together smoothly.

Apache Kafka was designed with a heavy emphasis on fault-tolerance and high-availability in mind, and thus provides different methods of ensuring enterprise-grade resiliency such as: replication factor - which defines how many partition replicas of a topic should be kept, each one being stored on a different broker. A replication factor of N allows for the loss of N-1 brokers. The new brokers will replace the lost brokers until the number of remaining broker(s) reaches the replication factor, or the number of copies of the partitions of a topic.

If you spend any of your time dealing with the cloud native world, you’ve probably already heard about Kubeflow. It’s something we’ve been playing with since we first began to explore the possibility of running Tensorflow in a distributed way. That was quite some time ago. Since then, Kubeflow has rapidly evolved, so that it now includes dozens of machine learning (ML) frameworks. The frameworks allow for the training and serving of all kinds of machine learning models.

Almost every blog post or lecture explaining how Istio service meshes route traffic takes the time to go over how sidecar containers capture outgoing traffic - how that traffic is routed to another service with another sidecar. However, in the real world, a large amount of network traffic passes through the boundaries of the service mesh itself. That traffic might be from a public facing app that receives traffic from the internet, an internal service that needs to connect to a legacy application running outside the mesh, or a workload that consumes an external, third party API.

Over the past few years, Kubernetes has become the de facto standard platform on which the world runs its cloud native applications. Although it has a great value proposition for use in all kinds of cases, its ecosystem is immensely complex and it requires a lot of expertise to operate. This is where the Banzai Cloud Pipeline platform comes in. Our mission is to help integrate Kubernetes into existing organizations, and to make the Kubernetes experience better for everyone:

In today’s blogpost we’re going to be discussing ingress and egress gateways. First, we’ll cover the basics, then we’ll go into detail and explore how they work through a series of practical examples. Ingress and egress gateways are load balancers that operate at the edges of any network receiving incoming or outgoing HTTP/TCP connections. Ingress gateways make it possible to define an entry points into an Istio mesh for all incoming traffic to flow through.

At Banzai Cloud we support and manage hybrid Kubernetes clusters for our customers across five clouds and on-prem (bare metal, VMware). Therefore, the ability and fluency required to observe these clusters is an absolute must. Very frequently, the Pipeline control plane is tasked with managing multiple Kubernetes clusters, which it does through our own CNCF certified Kubernetes distribution, PKE, or a cloud provider-managed distribution. When that happens, it’s important that we federate metrics, collect them into a single place for querying, analysis and long term storage.

Update: Looks like the solution described below isn’t going to land in Kubernetes after all. The developers went back to the drawing board and they will try to come up with a solution that’s best for everyone. The sidecar concept in Kubernetes is getting more and more popular, and for a good reason. In the container world, it’s a common principle that a container should address a single concern only, but do it well.

Readers of this blog may remember a post we made in January about Bank-Vaults that touched on the topic of disaster recovery with multi datacenter replication. In that post we dicussed replication, mostly in the context of it being used as a form of hot backup. Today we’ll be exploring cold backups, another but equally important form of disaster recovery. Why we use Velero We use a toolset for Kubernetes disaster recovery called Velero.

One of the challenges we repeatedly faced when using microservices-based solutions was how best to properly secure communication between participating services. One option was to manage security at the application layer, which meant implementing specific authentication mechanisms in the application code itself. This approach, however, would quickly become burdensome, eating up time for developers, who should be concentrating on implementing actual business logic. Wouldn’t it be awesome, we thought, if developers never had to worry about implementing authentication mechanisms in their application code, and, instead, there was a magical solution that would provide secure communication between their services?

Without a doubt Prometheus has become the de facto standard monitoring solution for Kubernetes, the same way it has become a core component of the Pipeline platform’s monitoring service. However, Prometheus already has a well defined mission with a focus on alerts and the storage of recent metrics. Prometheus’ local storage is limited by single nodes in its scalability and durability. Instead of trying to solve clustered storage in Prometheus itself, Prometheus has a set of interfaces that allow integration through remote storage systems.

Apache Kafka is a distributed streaming platform used to build reliable, scalable and high-throughput real-time streaming systems. Its capabilities, while impressive, can be further improved through the addition of Kubernetes. Accordingly, we’ve built an open-source Koperator and Supertubes to run and seamlessly operate Apache Kafka on Kubernetes through its various features, like fine-grain broker configuration, metrics based scaling with rebalancing, rack awareness, and graceful rolling upgrades - just to name a few.

Today we are happy to announce the 1.1 release of Backyards, Banzai Cloud’s automated and operationalized service mesh product built on Istio. This is an announcement post describing the new features of Backyards. If you’re not familiar with Backyards yet, and want to know why we decided to build this product, we suggest reading the blog post about the first major release. Want to know more? Get in touch with us, or delve into the details of the latest release.

Kubernetes is a highly extensible framework that is built from a bunch of loosely coupled components. This gives a very high level of flexibility, but adds some new challenges to the operation compared to monolithic solutions of similar systems of the past. One of these challenges is observability, especially log collection. This post describes how to collect the logs of Kubernetes components in detail, but does not discuss the collection of application (workload) logs.

In today’s post, we’ll be discussing multi-datacenter Vault clusters that span multiple regions. Most enterprises follow different replication strategies to provide scalable and highly-available services. One common replication/disaster recovery strategy for distributed applications is to have a hot standby replica of the very same deployment already setup in a secondary data center. When a catastrophic event occurs in the primary data center, all traffic is then redirected to the secondary datacenter.

In Kubernetes clusters, the number of Operators and their managed CRDs is constantly increasing. As the complexity of these systems grows, so does the demand for competent user interfaces and flexible APIs. At Banzai Cloud we write lots of operators (e.g. Vault, Istio, Logging, Kafka, HPA, etc) and we believe that whatever system you’re working with, whether it’s a service mesh, a distributed logging system or a centralized message broker operated through CRDs, you will eventually find yourself in need of enhanced observability and more flexible management capabilities.

Two months ago we announced the release of Backyards, Banzai Cloud’s multi- and hybrid-cloud enabled service mesh built on top of our Istio operator. One of [Backyards’] hallmarks is its ability to simplify building a production-ready Istio deployment down to a single command: backyards install -a - complete with enterprise grade security, monitoring, tracing, logs, audit, and features like canary releases, traffic management, circuit breaking and lots more, either through a convenient UI, CLI or a GraphQL API.

A while ago we published some benchmarks and sizing about our experience of running Apache Kafka over a service mesh with Koperator and Istio operator, orchestrated by our automated and operationalized service mesh, Backyards. The reasons for such a setup were many, and there are more details in the Running Apache Kafka over Istio - benchmark post, but let me recap some of our initial reasons, and how we evolved from there.

When someone first hears about Istio’s sidecar concept, the first two questions are usually about its effect on resource consumption and request latency or throughput. In this post we’ll discuss the first of those questions: resource consumption of Istio proxies and how it can be reduced with the new Sidecar custom resource. Previously we had a Kafka on Istio benchmark post that was about throughput, and we’ll take a deeper look at the most important HTTP request metrics in a forthcoming article (stay tuned).

Frequent readers of our blog and users of our hybrid cloud container management platform, Pipeline, will be familiar with the integrated cluster services that come with it. These services are automated end-to-end solutions for centralized logging, federated monitoring, security scans, advanced credential management, autoscaling, registries and lots more (see, for example, automated DNS management for Kubernetes). Providing an automated logging solution, and making sure it works seamlessly across multiple clusters, has always been part of Pipeline.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Istio has been rightfully praised for ushering in free observability and secure service to service communication. Other, more significant features, however, are what truly make Istio the Swiss army knife of service mesh operators; when it comes to meeting SLOs like uptime, latency and error rates, the ability to manage traffic between services is absolutely critical.

Two weeks ago we announced, the tech preview for Banzai Cloud Pipeline 2.0, the hybrid any-cloud platform. That new platform allows customers to build hybrid Kubernetes clusters in four different ways, across 5 cloud providers and on-premise datacenters and introduces a new acronym: BYOK, Bring your own Kubernetes! Day 2 operations Kubernetes has changed how workloads are run by providing a convenient abstraction layer over infrastructure. While many of our customers use Pipeline to create all their clusters, and want to kickstart their Day 0 and Day 1 experiences, others want to reuse the clusters that they have already deployed.

One of the exciting new features of Istio 1.4 is automatic mutual TLS support, which brings some long awaited convenience to Istio users configuring mTLS for their applications. In this post, we’ll be introducing the concept of Istio’s auto mTLS feature and demonstrating how it works using a demo application. Today, we’ll be using our open-source Banzai Cloud Istio Operator and our multi and hybrid-cloud enabled service mesh platform, Backyards, to install Istio 1.

With KubeCon North America 2019 drawing near, we are excited to share the latest developments from Banzai Cloud. Stop by our booth SE74 to see what exciting new products Banzai Cloud brings into the fold. Don’t miss the opportunity to connect with our team, register for 1:1 sessions, and experience the magic of Hungarian pálinka (it will be under the table, code word: pah-lyn-kah). Join us for a taste! BOOK A MEETING

UPDATE: For a newer Istio control plane upgrade method using the canary upgrade flow see the Safe and sound canary upgrade for your Istio control plane post. Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about.

Over the last few months we’ve been working on implementing a new internal Pipeline framework for managing integrated cluster services. While most of those services had already been supported as part of our cluster create flow, now they have been separated from that flow and have been componentized in Banzai Cloud Platform 2.0. This was done for two reasons: we wanted to give users the flexibility to install and uninstall these services at any time during a cluster’s lifecycle, and we wanted them to be able to do so on any Kubernetes cluster, even ones that were not created by Pipeline (note, with 2.

We are delighted to announce our tech preview of Banzai Cloud Pipeline 2.0, the hybrid any-cloud platform. A little under a year ago we released the first commercial version of Banzai Cloud Pipeline and made it available for development and PoCs by launching our Try Pipeline initiative. Banzai Cloud Pipeline 1.0 was the first incarnation of our vision for a productive and flexible multi-cloud container management platform (now supporting five public clouds and on-prem datacenters).

One of the earliest features of our hybrid cloud container management platform, Pipeline, was to standardize the running of applications on Kubernetes with a framework we called Spotguides. Spotguides allowed us to build domain specific knowledge into apps, avoid writing boilerplate code, and go from commit to a production ready deployment in minutes. All of this while meeting enterprise-grade requirements such as federated monitoring, centralized log collection, security scans, cluster lifecycle management and lots more.

There is a newer version of Backyards available, read more here: Announcing Backyards 1.1 Today we are happy to announce the first major release of Backyards, Banzai Cloud’s automated service mesh product. Before jumping into the details, we need to answer an obvious question: Why another service mesh product? The service mesh became one of the most hyped things lately, and it may seem that a new product is released every other week.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Service interruptions caused by outages can have severe business consequences, so it’s important that we build, run and test resilient systems. Resiliency can be implemented and tested at multiple levels, from the bottom infrastructure layer all the way to the application.

Our thinking that there was a hunger for an operator that makes easy the provisioning and operating Apache Kafka clusters on Kubernetes which is not based on Kubernetes stateful sets, proved to be correct as shortly after we released the first version our open-source Koperator a community started to build around it. We received lots of valuable feedback that helps to shape the future of Koperator and also feature contributions from the community.

On this blog we’ve already discussed our totally redesigned logging operator, which automates logging pipelines on Kubernetes. Thanks to the tremendous amount of feedback and the numerous contributions we received from our community, we’ve been able to rethink and redesign that operator from scratch, but the improvements aren’t going to stop coming any time soon. Our goal is to continue removing the burden from human operators, and to help them manage the complex architectures of Kubernetes.

At Banzai Cloud we work with cutting edge cloud technology every single day. Nowadays cloud and container related softwares are almost exclusively written in the Go programming language, so it’s no surprise most of our projects (Pipeline or Bank-Vaults to name a few) are also written in Go. In this post we will take a look at our favorite features of the latest (1.13) Go release. Reproducible builds — -trimpath Reproducible builds (or deterministic compilation) is the process of compiling a software in a way that ensures that the resulting artifact can be reproduced, given certain conditions are met.

Service interruptions caused by outages have severe consequences that range from harming a provider’s reputation to frustrating a customer to the point that they move to a competitor, so it’s extremely important to build and run resilient systems. Resiliency must be implemented at multiple levels from the bottom infrastructure layer to the application. For our customers, it’s a necessity that the EKS clusters they deploy their production applications onto be highly available.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. In production systems mostly when working in microservices architecture, it is soon realized that monitoring each service individually is often not enough to troubleshoot complex issues. It becomes a necessity to have a full picture of the whole call stack through the entire application with detailed information on service topologies, network latencies and individual request durations.

One of the key features of Pipeline, our hybrid cloud container management platform, is its ability to provision Kubernetes clusters across five different cloud providers (Alibaba, Azure, Amazon, Google), private datacenters (vmWare, baremetal, etc), or any combination thereof. It does this by using either cloud provider-managed Kubernetes, or our own CNCF certified Kubernetes distribution - PKE. Each cloud provider’s internal LB is different, and so is the way each is integrated with Kubernetes.

About a year ago we published the first release of our popular logging-operator. The initial version of that operator was designed to fit Pipeline, the Banzai Cloud hybrid cloud container management platform. However, since then, all kinds of people have found it to be an extremely useful tool that helps them manage their logs on Kubernetes. Initially, Fluent ecosystem automation was enough to support the disparate needs of our userbase, but, as the popularity of the logging-operator grew, different setups were put in place by our community that revealed some of its limitations.

UPDATE: For a newer Istio control plane upgrade method using the canary upgrade flow see the Safe and sound canary upgrade for your Istio control plane post. Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about.

Banzai Cloud’s Pipeline platform is an operating system that allows enterprises to develop, deploy and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operations teams alike. Strong security - multiple authentication backends, fine grained authorization, Vault-based dynamic secret management, automated secure communications between components using TLS, vulnerability scans, static code analysis, pod and network policies etc. - are a tier zero feature of the Pipeline platform, which we strive to automate and enable for all enterprises.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Istio has been rightfully praised for ushering in free observability and secure service to service communication. Other, more significant features, however, are what truly make Istio the Swiss army knife of service mesh operators; when it comes to meeting SLOs like uptime, latency and error rates, the ability to manage traffic between services is absolutely critical.

A key part of the Banzai Cloud Pipeline platform, has always been our strong focus on security. We incorporated Vault into our architecture early on in the design process, and we have developed a number of support components to be easily used with Kubernetes. We love what Vault enables us to do, but, as with many things security-related, strengthening one part of our system exposed a weakness elsewhere. For us, that weakness was K8s secrets, which is the standard way in which applications consume secrets and credentials on Kubernetes.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. At Banzai Cloud we work with Istio quite a bit and run a lot of Istio-based service meshes for our customers. Earlier this year we opensourced the Banzai Cloud Istio operator in order to simplify provisioning, management, upgrades, and multi-cluster scenarios.

One of the main features of the Banzai Cloud Pipeline platform is that it allows enterprises to run cost effective workloads by mixing spot and preemptible instances with regular ones, and without sacrificing overall reliability. First, let’s dig into some of the components that make spot instances so reliable, then we’ll provide an example of a Pipeline control plane installation, submit some workloads and simulate a spot instance termination. Availability in spot-instance clusters EKS or PKE clusters launched with Pipeline can mix in spot and on-demand instances (similarly, GKE can do this with preemptible instances).

At Banzai Cloud we develop a container management platform that provisions and orchestrates hundreds of Kubernetes clusters across six different cloud providers or hybrid clouds every day. Pipeline makes it easy for users to run applications on Kubernetes, but under the hood there is an insanely complex system, so it’s mission critical for us to have realtime information about what’s happening inside the platform and have access to historical data to investigate incidents our customers may have had.

Here at Banzai Cloud, we provision and manage Kubernetes clusters on multiple cloud providers (Alibaba, Amazon, Azure, and Google) and on-premise (bare metal or vmWare) with our container management platform, Pipeline. We support both cloud provider-managed K8s distributions (ACK, EKS, EKS, GKE) and our own lightweight, CNCF certified Kubernetes distribution, PKE. Both these approaches have their pros and cons, though that’s not what we’ll be talking about today (we’ve blogged about this several times already, see Deploying Pipeline Kubernetes Engine (PKE) on Azure).

At Banzai Cloud we do a lot more than work on Pipeline, our container management platform, and PKE, our lightweight CNCF certified Kubernetes distribution. In fact, we’re currently developing a variety of services that run on Kubernetes. These range from operators (Istio, Vault, Kafka, Logging, HPA to name a few), webhooks, K8s and cloud controllers to more general applications that we develop and test each day. During the development phase for these projects, we usually need to experiment and rapidly iterate applications, but, when using Kubernetes, this isn’t as easy as running each application inside a container.

About a year ago, we wanted to publish a central Helm chart repository. It seemed like the easiest way to do that was from a single source, so we migrated all of our Helm charts to a central Git repository. The idea was to use CircleCI to build every chart, then upload the resultant charts to S3 and serve them from there. It wasn’t a perfect solution, though, since it made coordination with application releases and tracking issues more difficult.

One of the key features of our container management platform, Pipeline is its ability to create multi- and hybrid-cloud Kubernetes environments using cloud provider-managed K8s or our own CNCF certified Kubernetes distribution, PKE. Recently, customers have been asking for a way to bring their existing Kubernetes clusters (upstream or other distributions) under Pipeline’s management, in order to benefit from the features our platform offers. During the peer review of our new cluster import feature, we realized the potential security risk created by the common practice of sharing kubeconfig files.

With Pipeline, we strive to provide a unified authentication and authorization experience across our multi- and hybrid-cloud environments. To accomplish this, we rely on dex, an identity service that uses OpenID Connect to drive authentication for apps. Dex and OpenID Connect use ID Tokens that are an OAuth2 extension, but not all the applications we use supports OAuth2 flows. Because of this, we searched for an OAuth proxy solution that handles authentication and basic policies that control access to these applications and services.

One of the Pipeline platform’s key open-source component is Bank-Vaults - the Vault swiss-army knife for Kubernetes. Feature requirements are a big part of the Pipeline platform, but a community has also built up around Bank-Vaults, and now it has its own use cases and requirements. We’ve received a lot of these external contributions and feature requests since our last blog update, which was about Bank-Vaults 0.4.7 in February. We’d like to walk you through some of these festures as 0.

One of the key features of our container management platform, Pipeline, as well as our CNCF certified Kubernetes distribution, PKE, is their ability to form and run seamlessly across multi- and hybrid-cloud environments. While the needs of Pipeline users vary depending on whether they employ a single or multi-cloud approach, they usually build upon one or more of these key features: Multi-cloud application management An Istio based automated service mesh for multi and hybrid cloud deployments Federated resource and application deployments built on Kubernetes federation v2 As Istio operator-based multi-cluster and multi/hybrid-cloud adoption increased, so did the demand for the ability to run distributed or decentralized applications wired into a service mesh.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Many organizations are embracing a more decentralized decision making process, allowing them to better build cloud native applications, and to deliver them more rapidly and safely. Often, different teams manage different microservices, with each team able to release their microservices independent of the others.

One of the key features of our container management platform, Pipeline, and our CNCF certified Kubernetes distribution, PKE, is their ability to seamlessly form and run federated clusters across multi- and hybrid-cloud environments. While users of the Pipeline platform often have different requirements depending on whether they take a single or multi-cloud approach, they’re usually built around two key features: Multi-cloud application management Backyards, an Istio-based automated service mesh for multi- and hybrid-clouds Today, we’re happy to announce that we’ve added support for Kubernetes federation v2, which is being made available as a beta feature in the Pipeline platform.

At Banzai Cloud we blog mostly about our container management platform, Pipeline. We frequently gloss over the workhorse underneath Pipeline, our CNCF certified Kubernetes distribution, PKE. That’s because our customers usually install and manage PKE with Pipeline, and thus benefit from all the enterprise-grade features we’ve already made available on that platform. However, PKE is also available on its own, and is one of the simplest ways to kickstart a Kubernetes cluster across multiple supported environments.

UPDATE: For a newer Istio control plane upgrade method using the canary upgrade flow see the Safe and sound canary upgrade for your Istio control plane post. Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about.

At Banzai Cloud we are passionate about observability, and we expend a great amount of effort to make sure we always know what’s happening inside our Kubernetes clusters. All clusters provisioned with Pipeline - our multi- and hybrid-cloud container management platform - are provided with, and rely upon, each of the three pillars of observability: federated monitoring, centralized log collection and traces. In order to automate log collection on Kubernetes, we opensourced a logging-operator built on the Fluent ecosystem.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. A few weeks ago we announced Backyards, Banzai Cloud’s automated service mesh built on top of our Istio operator, which greatly simplifies the complex management of service meshes across multi and hybrid-cloud environments.

If you’re reading this post, you’re likely already familiar with our container management platform, Pipeline, and our CNCF certified Kubernetes distribution, PKE: you probably already know how we make it possible to spin up clusters across several cloud providers and on-premise, in multi-cloud but also hybrid-cloud environments. But whether these are single or multi-cluster topologies, resilience is key. We at Banzai Cloud believe this is the case not just for infrastructural components but for entire managed application environments, like Apache Kafka.

At Banzai Cloud we work on a multi- and hybrid-cloud container management platform, Pipeline. As a result, we’ve opensourced quite a few Kubernetes operators. While writing some of the more complex operators, such as those for Istio, Vault or Kafka, we were faced with lots of unnecessary Kubernetes object updates. These updates are a byproduct of the fact that operators are typically used to manage a large number of resources. In each iteration, resource objects are first populated with their desired values, then sent to the Kubernetes API Server to ensure their desired state.

A strong focus on security has always been a key part of the Banzai Cloud’s Pipeline platform. We incorporated security into our architecture early in the design process, and developed a number of supporting components to be used easily and natively on Kubernetes. From secrets, certificates generated and stored in Vault, secrets dynamically injected in pods, through provider agnostic authentication and authorization using Dex, to container vulnerability scans and lots more: the Pipeline platform handles all these as a default tier-zero feature.

If you are a frequent reader of this blog or familiar with our products, you may already be aware that the control plane of our multi- and hybrid-cloud container management platform, Pipeline, is available not just as a free/developer service but can be run in any number of preferred envionments, whether cloud or on-prem. The control-plane’s only requirement is Kubernetes, its installation wholely automated by the banzai CLI tool alongside our own CNCF certified Kubernetes distribution, PKE.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. In the last few months we wrote a lot of different blog posts about the Istio service mesh. We started with a simple Istio operator, then went on with different multi-cluster service mesh topologies, Istio CNI and a telemetry deep dive.

This is the second part of a very popular post, Helm from basics to advanced. In the previous post (we highly suggest you read it, if you haven’t done so already) we covered Helm’s basics, and finished with an examination of design principles. In this post, we’d like to continue our discussion of Helm by exploring best practices and taking a look at some common mistakes. If you are looking for a place to securely store your Helm charts, remember that Banzai Cloud runs a free Helm Chart repository service.

A few weeks ago we opensourced our Koperator , the engine behind our Kafka Spotguide - the easiest way to run Apache Kafka on Kubernetes when it’s deployed to multiple clouds or on-prem, with out-of-the-box monitoring, security, centralized log collection, external access and more. One of our customers’ preferred features is the ability of our Koperator to react to custom alerts, in combination with the default options we provide: options like cluster upscaling, adding new Brokers, cluster downscaling, removing Brokers or adding additional disks to a Broker.

This post is part of a series that highlight the multi- and hybrid-cloud features of the Banzai Cloud Pipeline platform. Today, we will be focusing specifically on multi-cloud features. Before we take a deep dive into our technical content, let’s go over some of the key expectations an enterprise has when it embraces a multi-cloud strategy: Multi-cloud infrastructure provisioning: - the Pipeline platform and our CNCF certified Kubernetes distribution runs through a unified interface on 5 different cloud providers as well as on-prem.

Amid a growing number of increasingly sophisticated cyber attacks, enterprises are searching for ways to enable security wherever possible, in order to protect their data in transit and at rest. Big data processing is no exception; security is a very broad topic and to cover it in its entirety would be beyond the scope of this post. Instead, we will focus exclusively on those security capabilities that Spark on Kubernetes provides (by Spark on Kubernetes, we mean when Spark uses Kubernetes as an external cluster manager for creating and running executors).

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Service mesh probably needs no introduction. But, just to recap, let’s define it as a highly configurable, dedicated and low‑latency infrastructure layer designed to handle and provide reliable service-to-service communication, implemented as lightweight network proxies deployed alongside application code.

Yes, we opensourced yet another Kubernetes operator for Apache Kafka. This might seem bizarre, considering the alternatives that are already available (they exist but there are not too many), so you may be wondering, ‘Why?’ Well, keep reading and we’ll tell you: from design gaps and features we believe are necessary to operate Kafka on K8s, through my personal fix for Envoy, to some of our specific usage scenarios. tl;dr So to return to our initial question, ‘Why?

There has been a lot of talk about multi- and hybrid-cloud deployments over the past years. Some cloud vendors see these trends as a threat, others look at them as an opportunity. We think that beneath the buzzwords lie some very important use-cases driven by the needs of enterprises and SaaS providers. However, delivering and operating multi- and hybrid-clouds had been too complex for most organizations so far. The use-cases that we’ve seen at customers broadly relate to three main areas: flexibility, cost optimization, and compliance.

The Cloudinfo project has been a core part of the Pipeline platform since day one. We built and open sourced Cloudinfo as a unified interface that would allow us to access prices and services from all the cloud providers we support (AWS, Azure, Google, and Alibaba), resulting in a multi- and hybrid-cloud application platform. When a Pipeline user creates a single, hybrid- or multi-cloud Kubernetes cluster in one of the cloud providers or on-premise, they have the option of choosing between specifying their resource requirements (number of CPUs, memory, advanced network, I/O, etc) or simply selecting the cheapest cloud provider.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. One of the core features of the Istio service mesh is the observability of network traffic. Because all service-to-service communication is going through Envoy proxies, and Istio’s control plane is able to gather logs and metrics from these proxies, the service mesh can give you deep insights about your network.

The following is a guest blog post from Jürgen Weber, Bank-Vaults user and contributor extraordinaire. Here at hipages, we have a legacy approach to how we keep and maintain our ‘secrets’. The login details for some of our primary application resources are easy to obtain and with this carries great risk.. So to solve this we decided to embark on a ‘secrets’ project and implement Hashicorps Vault. As a part of this project, we looked at a variety of solutions.

Every major cloud provider offers a managed Kubernetes service that aims to simplify the provisioning of Kubernetes clusters in its respective environment. The Banzai Cloud Pipeline platform has always supported these major providers - AWS, Azure, Google, Alibaba Cloud - turning their managed k8s services into a single solution-oriented application platform that allows enterprises to develop, deploy and securely scale container-based applications in multi-cloud environments. While this was very appealing from the outset, we quickly realized there was demand among our enterprise users to implement more sophisticated use cases that were limited by our initial approach.

Update: Logging operator v3 (released March, 2020) We’re constantly improving the logging-operator based on feature requests of our ops team and our customers. The main features of version 3.0 are: Log routing based on namespaces Excluding logs Select (or exclude) logs based on hosts and container names Logging operator documentation is now available on the Banzai Cloud site. Check The Kubernetes logging operator reloaded post for details.

In one of our previous posts about creating Helm Charts for Kubernetes, we outlined what we consider the best practices for creating Helm charts. We’ve been using Helm in production and investing our time in creating Helm charts (available on the Banzai Cloud Charts GitHub repository) since Banzai Cloud’s inception. Creating Helm Charts is one thing; storing and serving them is another. We’d like to reduce the burden this places on the user, so today marks the launch of our Helm Chart repository service, which you can use to store and serve public Helm Charts for free.

In our last post about using Cadence workflows to spin up Kubernetes we outlined the basic concept of Cadence and walked you through how to use the Cadence workflow engine. Let’s dive into the experiences and best practices associated with implementing complex workflows in Go. We will use the deployment of our PKE Kubernetes distribution, from Pipeline to AWS EC2 as an example. Of course, you can deploy PKE independently, but Pipeline takes care of your cluster’s entire life-cycle , starting from nodepool and instance type recommendations, through infrastructure deployment, certificate management, opt-in deployment and configuration of our powerful monitoring, logging, service mesh, security scan, and backup/restore solutions, to the scaling or termination of your cluster.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Hybrid- and multi-cloud are quickly becoming the new norm for enterprises, just as service mesh is becoming essential to the cloud native computing environment. From the very beginning, the Pipeline platform has supported multiple cloud providers and wiring them together at multiple levels (cluster, deployments and services) was always one of the primary goals.

A strong focus on security has always been a key part of the Banzai Cloud’s Pipeline platform. We incorporated security into our architecture early in the design process, and developed a number of supporting components to be used easily and natively on Kubernetes. From secrets, certificates generated and stored in Vault, secrets dynamically injected in pods, through provider agnostic authentication and authorization using Dex, to container vulnerability scans and lots more: the Pipeline platform handles all these as a default tier-zero feature.

UPDATE: For a newer Istio control plane upgrade method using the canary upgrade flow see the Safe and sound canary upgrade for your Istio control plane post. Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about.

Two weeks ago we introduced our Kafka Spotguide for Kubernetes - the easiest way to deploy and operate Apache Kafka on Kubernetes. Since then, it’s been integrated into our application and DevOps container management platform, Pipeline, among other spotguides such as Spark on Kubernetes, Zeppelin, NodeJS and Golang, just to name a few. Because we’ve already met our goal of making it easy set up a Kafka cluster on Kubernetes with just few clicks, and in less than ten minutes - provisioning and operating its entire infrastructure, both in Kubernetes and Kafka - we’ve shifted our focus to Kafka security.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. At Banzai Cloud we are building a feature rich enterprise-grade application and devops container management platform, called Pipeline and a CNCF certified Kubernetes distribution, PKE. Security is one of our main areas of focus, and we strive to automate and enable those security patterns we consider essential for all the enterprises that use Pipeline.

One of the main goals of the Banzai Cloud Pipeline platform and PKE Kubernetes distribution is to radically simplify the whole Kubernetes experience and execute complex operations on behalf of the users. These operations communicate with a number of different remote services (from cloud providers to on-prem virtualization or storage providers) where we have little or no way to influence the result of these calls: how long will it take, will it ever succeed, whether it provides the desired result and so.

At Banzai Cloud we strive to enable a secure software supply chain which ensures that applications deployed with the Pipeline platform and Pipeline Kubernetes Engine are secure, without reducing developer productivity across all environments (on-premise, multi-, hybrid-, and edge-cloud). While we have our own internal processes and a dedicated security team working full time on hardening the entire application platform stack, it also makes sense to provide confidence to our customers following industry standard benchmarks.

One of the core features of our application and devops container management solution, the Banzai Cloud Pipeline platform, is multi-dimensional autoscaling based on default and custom metrics. Most such solutions gather Prometheus metrics and base Horizontal Pod Autoscaling only on basic CPU & memory metrics. However, scaling on these metrics is often not flexible enough to adapt to the increased load and traffic. window.HELP_IMPROVE_VIDEOJS = false; Therefore, Banzai Cloud Pipeline allows you to scale based on:

Banzai Cloud is proud to announce that our open source Pipeline Kubernetes Engine is now a CNCF Certified Kubernetes Distribution! PKE is an extremely simple Kubernetes installer and distribution, designed to work anywhere, and is the preferred run-time of Banzai Cloud’s cloud native application and devops container management platform, Pipeline. Banzai Cloud Pipeline supercharges the development, deployment and scaling of container-based applications with native support for multi-, hybrid-, and edge-cloud environments.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Two weeks ago we announced the first release of our Istio operator. Since then we’ve added support for Istio’s preliminary 1.1 release. This post will detail how and why you should try it.

One of the key features of the Pipeline platform is its ability to automatically provision, manage, and operate different application frameworks through what we call spotguides. Among the many spotguides we support on Kubernetes (Spark, Zeppelin, NodeJS, Golang, even custom frameworks - to name a few) Apache Kafka is among the most popular. We are heavily invested in making it as easy and straightforward as possible to operate Apache Kafka automatically on Kubernetes, and we believe that our current Apache Kafka Spotguide does just that.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Last week we released our open source Istio operator designed to help ease the sometimes difficult task of managing Istio. One of the main feature of the operator is its ability to manage a single mesh multi-cluster Istio.

At Banzai Cloud we try to provide our users with a unified, cloud and on-premise-agnostic authentication and authorization mechanism. Note that our Pipeline platform supports cloud provider-managed Kubernetes and, as of recently, our own Kubernetes distribution - the Pipeline Kubernetes Engine, PKE. We also recently introduced an open source project, JWT-to-RBAC (you can read more about that project, here), designed to solve authentication and authorization challenges within the Pipeline platform in a cloud provider-agnostic way.

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we’ve already blogged about. Service mesh has, without question, been one of the most vigorously debated and obsessed over topics of discussion in recent memory. It seems like, whichever way you turn, you run into heated arguments between those developers that are convinced that service mesh will outgrow even Kubernetes, and the naysayers convinced that, outside of use in a few large companies, service mesh is impractical to the point of uselessness.

Note: The Spotguides feature mentioned in this post is outdated and not available anymore. In case you are interested in a similar feature, contact us for details. If you’re a Node.js developer, it’s very likely that you are building microservices and have already come across Kubernetes. Kubernetes is a solid foundation for scalable container deployments but is also infamous for it’s steep learning curve. This post will explain how to containerize your Node.

At Banzai Cloud we are building a managed Cloud Native application and devops platform called Pipeline. Pipeline supercharges the development, deployment and scaling of container-based applications with native support for multi and hybrid-cloud environments. The Pipeline platform provides support for advanced scheduling that enables enterprises to run their workflows in an efficient way by scheduling workflows to nodes that meet the needs of the workflow (e.g.: CPU, memory, network, IO, spot price, etc).

Banzai Cloud’s Pipeline platform allows enterprises to develop, deploy and scale container-based applications on several cloud providers, using multiple Kubernetes distributions. One significant difference between the cloud providers that support Kubernetes (we support ACSK, EKS, AKS, GKE) and our own Banzai Cloud Pipeline Kubernetes Engine is our ability to access the Kubernetes API server, and to configure it. Whether our enterprise customers are using Banzai Cloud’s PKE distribution in a hybrid environment, or cloud provider-managed Kubernetes, they demand we meet the same high standards - the ability to authenticate and authorize (e.

One of the Banzai Cloud Pipeline platform’s key open-source projects is Bank-Vaults - the Vault swiss-army knife (and more) for Kubernetes. Feature requirements are part of the Pipeline platform, and the relatively large community around Bank-Vaults also has its own use cases and requirements. We’ve received lots of external contributions (thank you!), and we continue to find time to work on our community-driven features. While there have been many besides, these are the most sought-after features of the last few weeks.

As a tech startup our main focus is on delivering value to our users and customers which means that the Developer Experience often comes second. Since we launched our beta platform Pipeline, we received an enormous amount of attention not just from users and customers, but from developers as well. Based on your feedback we’ve started to formalize our development processes. This post contains instructions for setting up a basic development environment and explains our development workflow through a simple pull request as an example.

At Banzai Cloud we’re building a managed Cloud Native application and devops platform, called Pipeline. Pipeline supercharges the development, deployment and scaling of container-based applications with native support for multi- and hybrid-cloud environments. Pipeline’s built-in CI/CD solution is capable of creating Kubernetes clusters, running and testing builds, packaging and deploying applications as Helm charts, and lots more—all while its secrets are stored and managed by Vault. Note: The Pipeline CI/CD module mentioned in this post is outdated and not available anymore.

As those of you who are following us here at Banzai Cloud may or may not be aware, we are in the middle of releasing/certifying our own Kubernetes distribution — Pipeline Kubernetes Engine (PKE). PKE will be orchestrated the same way as other providers already supported by Pipeline, and will benefit from/inherit those features of the Banzai Cloud Pipeline platform that you already know and love. If you’re interested in learning more about PKE and our vision for buidling multi and hybrid cloud managed (application) environments, please read this post

Enterprises often use heterogenous clusters to deploy their applications to Kubernetes, because those applications usually have needs that involve special scheduling constraints. Pods may require nodes that have special hardware, that are isolated, or that are colocated with other pods running within a system. To help our customers solve these problems, the Banzai Cloud Pipeline platform uses nodepools. A nodepool is a subset of node instances within a cluster with the same configuration, however, the overall cluster can contain multiple nodepools as well heterogenous nodes/configurations.

Note: The Spotguides feature mentioned in this post is outdated and not available anymore. In case you are interested in a similar feature, contact us for details. Spotguides are one of the most useful features of the Banzai Cloud Pipeline platform. Spotguides are managed application environments, which provide an easy and scalable way of deploying applications, and programmatically take care of all the necessary “plumbing” that is critical for production (if you missed our introductory series on Spotguides, you can catch up here).

Suppose you’re working on a project which is running on Kubernetes - like we usually do - and you would like to test out this project on each and every pull request or commit. You can write many unit and integration tests, but at the end of the day, the proof of the pudding is in the eating. A real test would be to start up the application on the same platform where it will end up being deployed in production (in this case Kubernetes) and exercise it with some real workloads (aka end-to-end tests).

Note: The Spotguides feature mentioned in this post is outdated and not available anymore. In case you are interested in a similar feature, contact us for details. In December 2018 we released the public beta of Pipeline and introduced a Banzai Cloud terminology - spotguides. We have already gone deep into what Spotguides were and how they supercharged Kubernetes deployments of application frameworks (automated deployments, preconfigured GitHub repositories, CI/CD, job specific automated cluster sizing, Vault based secret management, etc.

The following is a guest blog post from Robbie Blaine, Site Reliability Engineer at EOH Big Data Lab. Contributions from the community are a key factor in driving our products forward. BIG thanks to all of you who have engaged with us by raising issues, giving feedback, or creating pull requests. Keep them coming, we love them! Automating Vault Deployment and Configuration on OKD with Bank-Vaults Hashicorp Vault is an Encryption-as-a-Service tool that is used to securely store and access secrets.

Our last two blog posts about the Kubernetes scheduler explained how taints and tolerations and different types of affinities are working. In today’s post we are going one layer deeper and we’ll discuss how to implement and deploy a custom Kubernetes scheduler. Writing a scheduler may sound intimidating at first, but if you’ll follow this article you’ll realise that creating something that works and schedules pods based on some simple rules is quite easy.

If you’re a frequent reader of this blog, you may have already seen a “short” description of what our platform does. It usually goes something like this: Banzai Cloud Pipeline is a solution-oriented application platform which allows enterprises to develop, deploy and securely scale container-based applications in multi- and hybrid-cloud environments. We frequently elaborate on this by providing a list of key features: Banzai Cloud Pipeline leverages best-of-breed cloud components, such as Kubernetes and adds a unified system architecture that enables a highly productive, yet flexible environment for developers and operations teams alike.

One of the main features of the Banzai Cloud Pipeline platform is that it allows enterprises to run workloads cost effectively by mixing spot or preemptible instances with regular ones, without sacrificing overall reliability. The platform allows enterprises to develop, deploy and scale container-based applications and it leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive yet flexible environment for developers and operation teams alike. tl;dr The Banzai Cloud Pipeline platform switched to a unified, cloud-aware spot instance termination handler to properly drain the cluster node and provide information to the monitoring system if an instance is going to be preempted from a cluster nodepool.

Note: The Spotguides feature mentioned in this post is outdated and not available anymore. In case you are interested in a similar feature, contact us for details. About a year ago we published a blog post on Spotguides, a core feature of the Banzai Cloud Pipeline platform. We spent a lot of time using and refining the original ideas, and as a result, many things changed since we first introduced the concept.

Banzai Cloud’s Pipeline platform and Kubernetes distribution tames the complexity inherent in the development, deployment, and scaling of modern containerized applications. The platform seeks to bring the power of cutting-edge cloud and containerization technologies to a wide range of enterprises. “Runners focus on the race they’re running, not the materials their shoes are made of; they trust that their shoes will get them to the finish line,” says Kris Flautner, CEO of Banzai Cloud.

We will be at KubeCon + CloudNativeCon, Seattle this week experiencing the magic and catching up with people from the broader community. This will be a special event for us as we will be launching the first public beta of our Banzai Cloud Pipeline platform. Let us know if you’d like to chat! Get in touch thru Twitter or LinkedIn. See you there!

Admission webhook series: In-depth introduction to Kubernetes admission webhooks Detecting and blocking vulnerable containers in Kubernetes (deployments) Controlling the scheduling of pods on spot instance clusters Banzai Cloud’s Pipeline platform uses a number of Kubernetes webhooks to provide several advanced features, such as spot instance scheduling, vulnerability scans and some advanced security features (to bypass K8s secrets - more to come next week). The Pipeline webhooks are all open source and available on our GitHub:

As of version 1.6, Kubernetes provides role-based access control (RBAC) so that administrators can set up fine-grained access to a variety of Kubernetes resources. It would take too long to fully explain why it makes sense to use RBAC in this post, but, in a nutshell, RBAC provides a level of control that most enterprises need to meet their security requirements within Kubernetes clusters. Processes and human operators that assume the identity of a Kubernetes Service Account will authenticate with said account and gain its associated access rights.

The Kubernetes scheduler can be constrained to place a pod on particular nodes using a few different options. One of these options is node and pod affinities. In a smaller homogeneous cluster they probably don’t make too much sense, because the scheduler is doing a good job spreading pods on different nodes, - well, that’s its job - but when you have a larger cluster with different types of nodes, maybe even spreading across availability zones, or multiple racks, then affinities may come in handy.

These days it seems that everyone is using some sort a CI/CD solution for their software development projects, either a third-party service, or something written in house. Those of us working on the Banzai Cloud Pipeline platform are no different; our CI/CD solution is capable of creating Kubernetes clusters, running and testing builds, of pulling secrets from Vault, packaging and deploying applications as Helm charts, and lots more. For quite awhile now (since the end of 2017), we’ve been looking for a Kubernetes native solution but could not find many.

Whenever we talk about Kubernetes monitoring and the topic of Prometheus comes up, so does Grafana; the two pretty much go hand in hand. Prometheus query language This article will not serve as an introduction to the powerful Prometheus query language. If you’re not comfortable with Prometheus, now would be a good time to familiarize yourself with the basics of querying and monitoring best practices with Prometheus on Kubernetes. We will only use simple queries with filters and labels in this post.

Banzai Cloud has closed its latest round of seed funding with a total investment of $2.5 million. The round was led by PortfoLion, a Central European venture capital and private equity fund, and included financing from FastVentures and Euroventures of Budapest, the latter being an angel investor that has been with the company since its foundation in 2017. “We’re very excited by this opportunity, by the exceptional team Banzai Cloud has brought together that will make them a force to be reckoned with in the world of enterprise computing.

The Banzai Cloud Pipeline platform allows enterprises to develop, deploy and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operation teams alike. One of the main features of the Pipeline platform is that it allows enterprises to run workloads cost effectively by mixing spot instances with regular ones, without sacrificing overall reliability. This requires quite a lot of behind the scenes magic to be built on top of core Kubernetes building blocks.

Banzai Cloud is on a mission to simplify the development, deployment, and scaling of complex applications and to bring the full power of Kubernetes to all developers and enterprises. Banzai Cloud’s Pipeline provides a platform which allows enterprises to develop, deploy and scale container-based applications. It leverages best-of-breed technology from the Cloud Native Foundation ecosystem to create a highly productive, yet flexible environment for developers and operation teams alike. One of the key tools we use from the Kubernetes ecosystem is Helm.

At Banzai Cloud we are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline - the operating system for your clouds. The platform itself consists of many building blocks - 30+ components - but they share one commonality: they are all developed in Golang. Dependency Injection is a hot topic in the Go community these days, mainly because of Google’s recently released dependency injection container: Wire.

Banzai Cloud’s Pipeline provides a platform which allows enterprises to develop, deploy and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operations teams alike. Strong security measures—multiple authentication backends, fine-grained authorization, dynamic secret management, automated secure communications between components using TLS, vulnerability scans, static code analysis, etc.—are a tier zero feature of the Pipeline platform, which we strive to automate and enable for all enterprises.

Observability is a key feature of the Banzai Cloud Pipeline platform - we put significant effort into monitoring and centralized log collection - and, recently, into tracing. Instrumentation Instrumentation is the process of monitoring/measuring software performance, writing logs and trace information and, more generally speaking, of collecting information that helps us to diagnose errors, run software with maximum availability, to understand how the software operates and, more importantly, how it impacts users and businesses.

Banzai Cloud’s Pipeline provides a platform which allows enterprises to develop, deploy and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operations teams alike. Strong security measures—multiple authentication backends, fine-grained authorization, dynamic secret management, automated secure communications between components using TLS, vulnerability scans, static code analysis, etc.—are a tier zero feature of the Pipeline platform, which we strive to automate and enable for all enterprises.

Banzai Cloud’s Pipeline platform is an operating system which allows enterprises to develop, deploy and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operations teams alike. Strong security measures — multiple authentication backends, fine grained authorization, dynamic secret management, automated secure communications between components using TLS, vulnerability scans, static code analysis, etc. — are a tier zero feature of the Pipeline platform, which we strive to automate and enable for all enterprises.

Business continuity is a key requirement for our enterprise users, and it becomes exponentially more important in a cloud native environment. The Banzai Cloud Platform operates in such an environment, deploying and managing large scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment, and provides a safety net to backup and restore cluster and application states during the lifecycle of the cluster.

Banzai Cloud’s Pipeline platform is an operating system which allows enterprises to develop, deploy and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operations teams alike. Strong security - multiple authentication backends, fine grained authorization, dynamic secret management, automated secure communications between components using TLS, vulnerability scans, static code analysis, etc. - is a tier zero feature of the Pipeline platform, which we strive to automate and enable to all enterprises.

Banzai Cloud’s Pipeline platform is an operating system which allows enterprises to develop, deploy and scale container-based applications. It leverages best-of-breed cloud components, such as Kubernetes, to create a highly productive, yet flexible environment for developers and operation teams alike. One of the main features of the Pipeline platform is that it allows enterprises to run workloads cost effectively by mixing spot instances with regular ones, without sacrificing overall reliability. This requires quite a lot of behind the scenes magic to be built on top of core Kubernetes building blocks.

Satellite is a Golang library and RESTful API that determines a host’s cloud provider with a simple HTTP call. Behind the scenes, it uses file systems and provider metadata to properly identify cloud providers. When we started to work on Pipeline and the Banzai Cloud Pipeline Platform Operators, we soon realized how frequently we would need to find out which cloud provider the service was actually running on. Note that Pipeline supports 6 different cloud providers

Monitoring series: Monitoring Apache Spark with Prometheus Monitoring multiple federated clusters with Prometheus - the secure way Application monitoring with Prometheus and Pipeline Building a cloud cost management system on top of Prometheus Monitoring Spark with Prometheus, reloaded Hands on Thanos Monitoring Vault on Kubernetes using Cloud Native technologies At Banzai Cloud we are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline.

The Banzai Cloud Cloudinfo service retrieves product and pricing information from cloud providers and exposes it through a RESTful API, and UI. Our Kubernetes based Pipeline platform and Telescopes recommendation engine make use of this information when they advise users on cluster layout and resourcing. Here’s a quick primer of how and why we utilize the Cloudinfo service: Pipeline platform users have the option of launching clusters or deploying applications based only on resource- and SLA-requirements (price, IO, memory, CPU, GPU, etc.

Monitoring series: Monitoring Apache Spark with Prometheus Monitoring multiple federated clusters with Prometheus - the secure way Application monitoring with Prometheus and Pipeline Building a cloud cost management system on top of Prometheus Monitoring Spark with Prometheus, reloaded Hands on Thanos Monitoring Vault on Kubernetes using Cloud Native technologies In depth application observability is a must-have feature for us; we put significant effort into monitoring all our customers’ Kubernetes deployments on the Pipeline platform, as well as the platform itself.

One of the main advantages of the Pipeline platform is that it allows users to use their infrastructure cost effectively; Telescopes helps with cluster and machine instance recommendations, Hollowtrees enables SLA-aware cost reduction using spot instances, and autoscalers allow for multi-dimensional autoscaling based on custom metrics. This post will highlight some new features of the Banzai Cloud Horizontal Pod Autoscaler Kubernetes Operator and the advanced automation supported by Pipeline - a new, forward-thinking way to operate Kubernetes clusters and autoscale deployments.

Two core features of the Pipeline platform are advanced security and observability for all Kubernetes deployments. We make sure that all secrets are securely stored, transported, images scanned, deployments monitored, and logs centrally collected. As you might expect from us, we use the industry standard in security components, such as Vault, Prometheus, Grafana, Fluent and more. This post is about monitoring Vault with Prometheus (on Kubernetes) and displaying metrics on Grafana.

Enterprises often use multi-tenant and heterogenous clusters to deploy their applications to Kubernetes. These applications usually have needs which require special scheduling constraints. Pods may require nodes with special hardware, isolation, or colocation with other pods running in the system. The Pipeline platform allows users to express their constraints in terms of resources (CPU, memory, network, IO, etc.). These requirements are turned into infrastructure specifications using Telescopes. Once the cluster nodes are created and properly labeled by Pipeline, deployments are run with the specified constraints automatically on top of Kubernetes.

Banzai Cloud announced today that it is collaborating with Oracle to bring its feature-rich application platform to Oracle Kubernetes Engine users. Banzai Cloud’s Pipeline deployment automation and execution engine enables developers to go from commit to scale in minutes by automating all the underlying tasks that provide convenient CI/CD flows, robust security, analytics and the ability to scale. The technology not only provides significant productivity gains to developers, but also increases operational efficiencies by aiding instance selection and automated introspection of large-scale workloads.

At Banzai Cloud we are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline. The platform itself consists of many building blocks - over 30 components - but they share one commonality: they are all developed in Golang. Obviously we are very fond of Go and like it quite a bit, so in this post we’d like to share the error handling practices that our team of 20+ developers adheres to while building the Pipeline platform.

Last year Alibaba joined CNCF and announced plans to create their own Kubernetes service - Alibaba ACK. The service was luanched more than a year ago, with its stated objective to make it easy to run Kubernetes on Alibaba Cloud without needing to install, operate, and maintain a Kubernetes control plane. At Banzai Cloud we are committed to providing support for Kubernetes on all major cloud providers, thus one of our priorities was to enable Alibaba Cloud’s Container Service for Kubernetes in Pipeline and take the DevOps experience to the next level by turning ACK into a feature-rich enterprise-grade application platform.

At Banzai Cloud we are building an application-centric platform for containers - Pipeline - running on Kubernetes to allow developers to go from commit to scale in minutes. We support multiple development languages and frameworks to build applications, with one common goal: all Pipeline deployments receive integrated centralized logging, monitoring, enterprise-grade security, autoscaling, and spot price support automatically, out of the box. In most cases we accomplish this in a non-intrusive way (i.

Apache Spark on Kubernetes series: Introduction to Spark on Kubernetes Scaling Spark made simple on Kubernetes The anatomy of Spark applications on Kubernetes Monitoring Apache Spark with Prometheus Spark History Server on Kubernetes Spark scheduling on Kubernetes demystified Spark Streaming Checkpointing on Kubernetes Deep dive into monitoring Spark and Zeppelin with Prometheus Apache Spark application resilience on Kubernetes Apache Zeppelin on Kubernetes series: Running Zeppelin Spark notebooks on Kubernetes Running Zeppelin Spark notebooks on Kubernetes - deep dive

Banzai Cloud is happy to announce that it is an Amazon EKS Platform Partner. Last year Amazon joined CNCF and announced plans to create their own Kubernetes service - Amazon EKS. The service has been launched this June, with the objective to make it easy for you to run Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes control plane. At Banzai Cloud we are committed to provide support for Kubernetes on all major cloud providers for our users thus one of our priority was to enable Amazon EKS in Pipeline and take the DevOps and user experience to the next level by turning EKS into a feature rich enterprise-grade application platform.

At Banzai Cloud we are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline. We have always been committed to supporting Kubernetes and our container based application platform on all major providers, however, we are also committed to making portability between cloud vendors easy, seamless and automated. Accordingly, this post will highlight a few important aspects of a multi-cloud approach we learned from our users, and the open source code we developed and made part of the Pipeline platform.

Update: Logging operator v3 (released March, 2020) We’re constantly improving the logging-operator based on feature requests of our ops team and our customers. The main features of version 3.0 are: Log routing based on namespaces Excluding logs Select (or exclude) logs based on hosts and container names Logging operator documentation is now available on the Banzai Cloud site. Check The Kubernetes logging operator reloaded post for details.

Continuing our commitment to support all major cloud providers, today we are adding support for Oracle’s Kubernetes-managed cloud service, OKE – Oracle Kubernetes Engine in Pipeline. We are building a feature rich enterprise-grade application platform on top of Kubernetes - called Pipeline - to deliver a better DevOps experience by automating the lifecycle management of applications. This experience is now available to OKE users as well; here, we will guide you through the first steps of using OKE and summarize the benefits of using Pipeline.

At Banzai Cloud we are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline. With Pipeline we provision large, multi-tenant Kubernetes clusters on all major cloud providers, specifically AWS, GCP, Azure, AliCloud, and BYOC - on-premise and hybrid - and deploy all kinds of predefined or ad-hoc workloads to these clusters. For us and our enterprise users authentication and authorization is absolutely vital, thus, in order to access the Kubernetes API and the Services in an authenticated manner as defined within Kubernetes, we arrived at a simple but flexible solution.

A few months ago the Kubernetes Operator SDK was released with one of its goals being the conversion of human operational knowledge into code. At Banzai Cloud we’ve been contributors and early adopters of this technology, since it provides a better standardized method of automating our processes and allows us to dramatically ease the lives of our customers. We are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline, wherein we endeavour to automate the DevOps experience and the lifecycle of deployments.

At Banzai Cloud we are building a feature-rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline. Applications deployed to Pipeline automatically inherit the platform’s features: enterprise-grade security, observability (centralized log collection, monitoring and tracing), discovery, high availability and resiliency, just to name a few - encapsulated in spotguides. One of the most popular spotguides we deploy is Spark. In the past few months we’ve been working and pushing many pull requests to make Spark a first class player on Kubernetes and to make it resilient.

A few weeks back we released Telescopes, our Kubernetes cluster layout recommender application. That application has evolved quite a bit, and in this post we’ll provide insight into some its new features and recent changes. Cloud cost management series: Overspending in the cloud Managing spot instance clusters on Kubernetes with Hollowtrees Monitor AWS spot instance terminations Diversifying AWS auto-scaling groups Draining Kubernetes nodes tl;dr: We added new features to Telescopes to provide support for blacklisting or whitelisting instance types Recommendation accuracies can now be checked There is now support that allows asking cloud instance types for CPU, memory and network performance.

One of our goals at BanzaiCloud is to make our customers’ lives easier by providing low barrier to entry, easy to use solutions for running applications on Kubernetes. To achieve this, we often rely on Kubernetes Operators to provide comprehensive solutions over the course of an application’s lifecycle. Here is a list of our operators, which we have already open sourced: Vault Operator Prometheus JMX Exporter Operator Wildfly Operator

At Banzai Cloud we are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline. Security is one of our main areas of focus, and we strive to automate and enable those security patterns we consider essential, including tier zero features for all enterprises using the Pipeline Platform. We’ve blogged about how to handle security scenarios on several of our previous posts. This time we’d like to focus on a different aspect of securing Kubernetes deployments:

We are excited to announce that Banzai Cloud has joined the Cloud Native Computing Foundation! The CNCF and The Linux Foundation are expending extraordinary effort in helping to standardize open source technologies that enable the development, deployment, management and operation of next generation Cloud Native software stacks. Our mission is to bring Cloud Native to enterprises and with this announcement, we strive to help push container and cloud native technology standardization and interoperability forward.

At Banzai Cloud we are building a feature-rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline. For an enterprise-grade application platform security is a must and it has many building blocks. Please read through the Security series on our blog to learn how we deal with a variety of security-related issues. Security series: Authentication and authorization of Pipeline users with OAuth2 and Vault Dynamic credentials with Vault using Kubernetes Service Accounts Dynamic SSH with Vault and Pipeline Secure Kubernetes Deployments with Vault and Pipeline Policy enforcement on K8s with Pipeline The Vault swiss-army knife The Banzai Cloud Vault Operator Vault unseal flow with KMS

At Banzai Cloud we are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline. With Pipeline we provision large, multi-tenant Kubernetes clusters on all major cloud providers such as AWS, GCP, Azure, Alibaba and BYOC, on-premise and hybrid, and deploy all kinds of predefined or ad-hoc workloads to these clusters. For us and our enterprise users, Kubernetes secret management (base 64) was not sufficient, so we chose Vault and added Kubernetes support to manage our secrets.

If you followed our blog series on Autoscaling on Kubernetes, you should already be familiar with Kubernetes’ Cluster autoscaler and the Vertical Pod Autoscaler used with Java 10 applications. This post will show you how to use the Horizontal Pod Autoscaler to autoscale your deployments based on custom metrics obtained from Prometheus. As a deployment example we’ve chosen our JEE Petstore example application on Wildfly to show that, beside metrics like cpu and memory, which are provided by default on Kubernetes, using our Wildfly Operator, all Java and Java Enterprise Edition / Wildfly specific metrics are automatically placed at your fingertips, available in Prometheus, allowing you to easily autoscale deployments.

Here at Banzai Cloud we blog a lot about Prometheus and how to use it. One of the problems we have so far neglected to discuss is the inadequate long term storage capability of Prometheus. Luckily a new project called Thanos seeks to address this. If you are not familiar with Prometheus, or are interested in other monitoring related articles, check out our monitoring series, here: Monitoring series: Monitoring Apache Spark with Prometheus Monitoring multiple federated clusters with Prometheus - the secure way Application monitoring with Prometheus and Pipeline Building a cloud cost management system on top of Prometheus Monitoring Spark with Prometheus, reloaded

When we started to work on our cluster infrastructure recommender, Telescopes, we soon realized how difficult it was to get instance type attributes and pricing information from cloud providers programatically. While EC2, Google Cloud, and Azure all provide some kind of API from which to query this information, in some cases these APIs respond with partially inconsistent data, or their responses are large chunks of JSON files that are very cumbersome to parse.

At Banzai Cloud we are building a feature rich enterprise-grade application platform, built for containers on top of Kubernetes, called Pipeline. With Pipeline we provision large, multi-tenant Kubernetes clusters on all major cloud providers such as AWS, GCP, Azure and BYOC, on-premise and hybrid, and deploy all kinds of predefined or ad-hoc workloads to these clusters. For us and our enterprise users, Kubernetes secret management (Base64) was woefully inadequate, so we chose Vault with native Kubernetes support to manage our secrets.

This post highlights how the Pipeline Platform enables Managed Service Identity (MSI) and assigns the Storage Account Contributor role to AKS cluster Virtual Machines. But wait, why? At Banzai Cloud we have a PVC Operator, which makes using Kubernetes Persistent Volumes easier on cloud providers by dynamically creating the required accounts and storage classes. That operator allows us to use the same Helm chart on all supported providers, thus there is no need to create cloud specific Helm charts.

At Banzai Cloud we provision all kinds of applications to Kubernetes and we try to autoscale these clusters with Pipeline and/or properly size application resources as needed. As promised in an earlier blog post, How to correctly size containers for Java 10 applications, we’ll share our findings on the Vertical Pod Autoscaler(VPA) used with Java 10. VPA sets resource requests on pod containers automatically, based on historical usage, thus ensuring that pods are scheduled onto nodes where appropriate resource amounts are available for each pod.

One of our goals with Pipeline is to support Java and Java Enterprise Edition deployments, allowing developers to iterate fast while building and deploying safe, and also pushing code to production. In order to do that, we place a lot of importance on different aspects of a Java/JEE application’s lifecycle - we allow engineers: To continuously integrate and deploy their Java apps to Kubernetes To deploy Java Enterprise Edition applications to Kubernetes Once the Java containers are deployed to K8s, to avoid OOMKills To correctly size Java containers And, once deployments are done and sized, to monitor them without any code modification Enter Infinispan - a distributed cache and data grid.

One of our goals at Banzai Cloud is to eliminate the concept of nodes, insofar as that is possible, so that users will only be aware of their applications and respective resource needs (cpu, gpu, memory, network, etc). Launching Telescopes was a first step in that direction - helping end users to select the right instance types for the job, through Telescopes infrastructure recommendations, then turning those recommendations into actual infrastructure with Pipeline.

For our Pipeline Platform, observability is an essential part of operating distributed applications in production. We put a great deal of effort into monitoring large and federated clusters, and automating the centralized log collection of these clusters with Pipeline. That way, all our users get out-of-the-box observability for free. Logging series: Centralized logging under Kubernetes Secure logging on Kubernetes with Fluentd and Fluent Bit Advanced logging on Kubernetes

If you are looking to try out an automated way to provision and manage Kafka on Kubernetes, please follow this Kafka on Kubernetes the easy way link. At Banzai Cloud we use Kafka internally a lot. We have some internal systems and customer reporting deployments where we rely heavily on Kafka deployed to Kubernetes. We practice what we preach and all these deployments (not just the external ones) are done using our application platform, Pipeline.

At Banzai Cloud we’re always open to experimenting with and integrating new software (tools, products). We also love to validate our new ideas by quickly implementing “proof of concept” projects. Even though we used five or so programming languages while building the Pipeline Platform, we love and use Golang the most. While these PoC projects are not intended for production use, they often serve as the basis for it. When this is the case, the PoC code needs to be refactored - or prepared for production.

At Banzai Cloud we place a lot of emphasis on the observability of applications deployed to the Pipeline Platform, which we built on top of Kubernetes. To this end, one of the key components we use is Prometheus. Just to recap, Prometheus is: an open source systems monitoring and alerting tool a powerful query language (PromQL) a pull based metrics gathering system a simple text format for metrics exposition Problem statement Usually, legacy applications are not exactly prepared for these last two, so we need a solution that bridges the gap between systems that do not speak the Prometheus metrics format: enter exporters.

During the development of the Pipeline Platform all of its key building blocks such as Pipeline, Hollowtrees and Bank-Vaults have relied on making extensive Kubernetes API calls. Often, we tried a quick K8s API call or ran a small PoC inside a cluster, while also wanting to avoid the usual deployment process. We quickly realized that we needed a shortcut. There are tools like telepresence that support slightly more complex scenarios.

Creating Kubernetes clusters in the cloud and deploying (or CI/CDing) applications to those clusters is not always simple. There are a few conventional options, but they are either cloud or distribution specific. While we were working on our open source Pipeline Platform, we needed a solution which covered (here follows an inclusive but not exhaustive list of requirements): provisioning of Kubernetes clusters on all major cloud providers (via REST, UI and CLI) using a unified interface application lifecycle management (on-demand deploy, dependency management, etc) preferably over a REST interface support for multi tenancy, and advanced security scenarios (app to app security with dynamic secrets, standards, multi-auth backends, and more) ability to build cross-cloud or hybrid Kubernetes environments This posts highlights the ease of creating Kubernetes clusters using the Pipeline API on the following providers:

At Banzai Cloud we continue to work hard on the Pipeline platform we’re building on Kubernetes. We’ve open sourced quite a few operators already, and even recently teamed up with Red Hat and CoreOS to begin work on Kubernetes Operators using the new Operator SDK, and to help move human operational knowledge into code. The purpose of this blog will be to take a dive deep into the PVC Operator.

At Banzai Cloud we’re building a feature rich platform, Pipeline, on top of Kubernetes. With Pipeline we provision large, multi-tenant Kubernetes clusters on all major cloud providers - AWS, GCP, Azure and BYOC - and deploy all kinds of predefined or ad-hoc workloads to these clusters. We wanted to set the industry standard for the way in which our users log in and interact with secure endpoints, and, at the same time, we wanted to provide dynamic secret management for each application we support.

At Banzai Cloud we run and deploy containerized applications to our PaaS, Pipeline. Java or JVM-based workloads, are among the notable workloads deployed to Pipeline, so getting them right is pretty important for us and our users. Java/JVM based workloads on Kubernetes with Pipeline Why my Java application is OOMKilled Deploying Java Enterprise Edition applications to Kubernetes A complete guide to Kubernetes Operator SDK Spark, Zeppelin, Kafka on Kubernetes

Note: The Spotguides feature mentioned in this post is outdated and not available anymore. In case you are interested in a similar feature, contact us for details. A good number of years ago, back at beginning of this century, most of us here at Banzai Cloud were in the Java Enterprise business, building application servers (BEA Weblogic and JBoss) and JEE applications. Those days are gone; the technology stack and landscape has dramatically changed; monolithic applications are out of fashion, but we still have lots of them running in production.

Banzai Cloud announced today that it will collaborate with Red Hat in an effort to help standardize the management of complex stateful applications on Kubernetes. Simultaneously, Red Hat announced the Operator Framework, an open source toolkit designed to manage application instances on Kubernetes in a more effective, automated, and scalable way. Banzai Cloud will work collaboratively with Red Hat on this open source project, which will focus on the creation of a new Software Development Kit (SDK) for the “operators” pattern.

Note: The Spotguides feature mentioned in this post is outdated and not available anymore. In case you are interested in a similar feature, contact us for details. Our last post about the Banzai Cloud CI/CD flow described how to build/deploy a Spring Boot application on Kuberbetes. This post does the same for a Node.js application. We’ve teamed up with our friends at RisingStack to streamline Node.js application deployments to Kubernetes (pushable to any cloud provider) with Pipeline, which does all the heavylifting and allows developers to focus on writing the code.

Cloud cost management series: Overspending in the cloud Managing spot instance clusters on Kubernetes with Hollowtrees Monitor AWS spot instance terminations Diversifying AWS auto-scaling groups Draining Kubernetes nodes A few months ago we posted on this blog about overspending in the cloud. We discussed how difficult it is to keep track of the vast array of instance types and pricing options offered by cloud providers, especially on AWS with spot pricing.

At Banzai Cloud we’re always looking for new and innovative technologies that support our users in their transition to microservices deployed on Kubernetes, using Pipeline. In recent months we’ve partnered with CoreOS and RedHat to work on Kubernetes operators. That project was opensourced today, and is now available on GitHub. If you read through the rest of this blog, you’ll learn what an operator is, and how to use the operator sdk to develop an operator through a concrete example we developed, here, at Banzai Cloud.

This week KubeCon + CloudNativeCon, Copenhagen will bring together over 4000+ developers, architects and other members of the cloud native and open source communities under one roof. We at BanzaiCloud are part of these communities, as well as the CNCF landscape since we’re certified experts in these technologies and operating under the umbrella of the Cloud Native Computing Foundation, so we can’t wait. Join us to learn about Kubernetes and its related technologies directly from industry experts.

We are excited to announce that Banzai Cloud is now a Kubernetes Certified Service Provider (KCSP). The KCSP program was started by the Cloud Native Computing Foundation in collaboration with the Linux Foundation and represents a milestone in the wide-spread adoption of a cloud native platform. It provides a strict set of rules and a battery of certified experts that guarantee only experienced partners be admitted to the program. This fosters trust, so enterprises can rely on Banzai Cloud and our flagship PaaS, Pipeline, to bring to bear the experience necessary to guide them on their Kubernetes and microservices journey to cloud native application platforms and production usage.

Bank-Vaults is a thick, tricky, shifty right with a fast and intense tube for experienced surfers only, located on Mentawai. Think heavy steel doors, secret unlocking combinations and burly guards with smack-down attitudes. Watch out for clean-up sets. Bank-Vaults is a wrapper for the official Vault client with automatic token renewal, built in Kubernetes support, dynamic database credential management, multiple unseal options, automatic re/configuration and more. At Banzai Cloud, we’ve been building an open source next generation platform as a service, called Pipeline, which is constructed on top of Kubernetes.

At Banzai Cloud we push different types of workloads to Kubernetes with our open source PaaS, Pipeline. There are lots of deployments we support for which we have defined Helm charts, however, Pipeline is able to deploy applications from any repository. These deployments are pushed on-prem or in the cloud, but many of these deployments share one common feature, the need for persistent volumes. Kubernetes provides abundant options in this regard, and each cloud provider also offers custom/additional alternatives.

Update: Logging operator v3 (released March, 2020) We’re constantly improving the logging-operator based on feature requests of our ops team and our customers. The main features of version 3.0 are: Log routing based on namespaces Excluding logs Select (or exclude) logs based on hosts and container names Logging operator documentation is now available on the Banzai Cloud site. Check The Kubernetes logging operator reloaded post for details.

While we build our open source, cloud agnostic Heroku/Cloud Foundry-like Paas - Pipeline - on top of Kubernetes, we continue to launch lots of clusters on different cloud providers. Most of these clusters are launched on spot or preemptible instances, and managed by Hollowtrees. However, there are many smaller development clusters, control planes, instances and PoCs we launch that are marginally related to, or launched with, Pipeline. Naturally, these have an associated cost that we want to keep tight control over.

In the past few weeks we’ve been blogging about the advanced, enterprise-grade security features we are building into our open source PaaS, Pipeline. If you’d like to review these features, please read this series: Security series: Authentication and authorization of Pipeline users with OAuth2 and Vault Dynamic credentials with Vault using Kubernetes Service Accounts Dynamic SSH with Vault and Pipeline Secure Kubernetes Deployments with Vault and Pipeline Policy enforcement on K8s with Pipeline The Vault swiss-army knife The Banzai Cloud Vault Operator Vault unseal flow with KMS Kubernetes secret management with Pipeline Container vulnerability scans with Pipeline Kubernetes API proxy with Pipeline

Apache Spark on Kubernetes series: Introduction to Spark on Kubernetes Scaling Spark made simple on Kubernetes The anatomy of Spark applications on Kubernetes Monitoring Apache Spark with Prometheus Spark History Server on Kubernetes Spark scheduling on Kubernetes demystified Spark Streaming Checkpointing on Kubernetes Deep dive into monitoring Spark and Zeppelin with Prometheus Spark Streaming Checkpointing on Kubernetes Deep dive into monitoring Spark and Zeppelin with Prometheus Apache Spark application resilience on Kubernetes

Apache Spark on Kubernetes series: Introduction to Spark on Kubernetes Scaling Spark made simple on Kubernetes The anatomy of Spark applications on Kubernetes Monitoring Apache Spark with Prometheus Spark History Server on Kubernetes Spark scheduling on Kubernetes demystified Spark Streaming Checkpointing on Kubernetes Deep dive into monitoring Spark and Zeppelin with Prometheus Spark Streaming Checkpointing on Kubernetes Deep dive into monitoring Spark and Zeppelin with Prometheus Apache Spark application resilience on Kubernetes Collecting Spark History Server event logs in the cloud

Update: Logging operator v3 (released March, 2020) We’re constantly improving the logging-operator based on feature requests of our ops team and our customers. The main features of version 3.0 are: Log routing based on namespaces Excluding logs Select (or exclude) logs based on hosts and container names Logging operator documentation is now available on the Banzai Cloud site. Check The Kubernetes logging operator reloaded post for details.

During the development of our open source Pipeline PaaS, we introduced some handy features to help deal with deployments. We deploy most of our applications as Helm releases, so we needed a way to interact programatically (using gRPC) and to use a UI (RESTful API) with Helm. In order to do that with Pipeline, we introduced a very useful feature that manages Helm repositories and deploys applications with Helm to Kubernetes, using RESTful API calls.

Cloud cost management series: Overspending in the cloud Managing spot instance clusters on Kubernetes with Hollowtrees Monitor AWS spot instance terminations Diversifying AWS auto-scaling groups Draining Kubernetes nodes Cluster recommender Cloud instance type and price information as a service Kubernetes was designed in such a way as to be fault tolerant of worker node failures. If a node goes missing because of a hardware problem, a cloud infrastructure problem, or if Kubernetes simply ceases to receive heartbeat messages from a node for any reason, the Kubernetes control plane is clever enough to handle it.

At Banzai Cloud, we’re building a feature rich platform as a service on Kubernetes, called Pipeline. With Pipeline, we provision large, multi-tenant Kubernetes clusters on all major cloud providers, such as AWS, GCP, Azure and BYOC, and deploy all kinds of predefined or ad-hoc workloads to these clusters. When we needed a way for our users to login and interact with protected endpoints and, at the same time, provide dynamic secrets management support, while simultaneously providing native Kubernetes support for all our applications, we turned to Vault.

Note: The Spotguides feature mentioned in this post is outdated and not available anymore. In case you are interested in a similar feature, contact us for details. This post showcases how to enable a simple Spring Boot application for the Banzai Cloud CI/CD flow, build and save the necessary artifacts, and deploy it to a Kubernetes cluster. We have already posted about our CI/CD flow several times, and have set up a few example projects to illustrate how it works; this time we’ll show you how to use it with an arbitrary Spring Boot application.

Update: Logging operator v3 (released March, 2020) We’re constantly improving the logging-operator based on feature requests of our ops team and our customers. The main features of version 3.0 are: Log routing based on namespaces Excluding logs Select (or exclude) logs based on hosts and container names Logging operator documentation is now available on the Banzai Cloud site. Check The Kubernetes logging operator reloaded post for details.

For some time we’ve been evangelizing the idea that the runtime fabric of big data workloads should be Kubernetes. In this post I’d like to walk through the thought process behind that change and discuss its benefits. Obliviously, this is a pretty large topic, and this post has no intention of covering it completely - also, it reflects the views and opinions that we at Banzai Cloud believe in and push others to adopt.

The adoption of serverless technologies is quickly progressing. According to this survey, it’s on par with the adoption of containers. And, even though ‘serverless’ is a very vague term (it can be argued that it’s rarely used in production, especially in complex applications), it seems set to be one of the most dominant trends in the near future in the cloud computing space. While, once, serverless referred specifically to early stage AWS Lambda, the category has matured rapidly.