When we started to work on our cluster infrastructure recommender, Telescopes, we soon realized how difficult it was to get instance type attributes and pricing information from cloud providers programatically. While EC2, Google Cloud, and Azure all provide some kind of API from which to query this information, in some cases these APIs respond with partially inconsistent data, or their responses are large chunks of JSON files that are very cumbersome to parse. So, it soon became clear that if we wanted to have a properly functioning recommender that delivered up to the minute pricing data, our first step would be to build a service that could reliably fetch this information. Our product info service uses cloud provider APIs to asynchronously fetch and parse instance type attributes and prices, while storing the results in an in-memory cache, and making them available as structured data through a REST API. In this post we will be introducing our Cloud Info service, and discussing the different cloud provider APIs for those interested in their technical details.

This service is part of the Pipeline ecosystem - a feature rich enterprise-grade application platform, built for containers on top of Kubernetes.

This post is part of our cloud cost management series:

Overspending in the cloud Managing spot instance clusters on Kubernetes with Hollowtrees Monitor AWS spot instance terminations Diversifying AWS auto-scaling groups Draining Kubernetes nodes Cluster recommender Cloud instance type and price information as a service

The Cloud Info Service

We built this service for our recommendation project, and, after the API was up and running, we decided to build a light UI around it and make it available online for everyone at https://banzaicloud.com/cloudinfo. This service is not just a UI, but a REST API as well. It can be used to build tools that help with cost management or to compare different providers, regions or instance types. Here's an example of how to use this API to get all the instance types available on EC2 in the EU (Ireland) region:

curl -ksLX GET https://banzaicloud.com/cloudinfo/api/v1/providers/amazon/services/compute/regions/eu-west-1/products | jq .
{
  "Products": [
    {
      "type": "i3.8xlarge",
      "onDemandPrice": 2.752000093460083,
      "cpusPerVm": 32,
      "memPerVm": 244,
      "gpusPerVm": 0,
      "ntwPerf": "10 Gigabit",
      "spotPrice": [
        {
          "zone": "eu-west-1b",
          "price": "0.981500"
        },
        {
          "zone": "eu-west-1c",
          "price": "1.261300"
        },
        {
          "zone": "eu-west-1a",
          "price": "2.752000"
        }
      ]
    },
    {
      "type": "g3.8xlarge",
      "onDemandPrice": 2.4200000762939453,
      "cpusPerVm": 32,
      "memPerVm": 244,
      "gpusPerVm": 2,
      "ntwPerf": "10 Gigabit",
      "spotPrice": [
        {
          "zone": "eu-west-1a",
          "price": "0.800600"
        },
        {
          "zone": "eu-west-1c",
          "price": "2.420000"
        },
        {
          "zone": "eu-west-1b",
          "price": "0.726000"
        }
      ]
    },
    {...}
  ]
}

Connection to the Pipeline infrastructure

This service is used by our cluster infrastructure recommender, Telescopes - with the up-to-date information about available instance types serving as the basis of its recommendations. Telescopes is used to recommend Kubernetes cluster layouts that can be started by Pipeline and then managed by Hollowtrees.

Collecting pricing and machine type info

All major cloud providers make their pricing information available in different formats and through different APIs. Some have tools to filter for specific products, others only have large JSON files that require parsing. In most cases, one API call is not enough to collect instance type characteristics, like CPUs or memory, and the related pricing or spot pricing. Our cloud info service collects all pricing information from each of the three major providers, caches it, updates it daily through the APIs and makes it available through a REST API. Caching all that data in memory makes the service fast, while daily updates keep it up-to-date, even when a cloud provider adds a new instance type or region. Additionally, having the same API for all providers and all regions makes it easy to programatically compare them, or to build cost management tools on top of the API. So let's examine what kinds of services allow us to collect what we need, and how these services are used by our own product info service.

EC2

Of the three major cloud providers, EC2 has the best tools for querying instance types and related pricing. Since the end of last year, they've maintained a Price List API that allows users to query product pricing in a detailed way. Before that, the only way to query the price of a service product was to download a large JSON or CSV file that listed all the products and their prices for a specific region (or a specific service) and parse that information manually. Since the API update, it's become easier to query this information, so let's take a look at an AWS CLI example that fetches product info for the c5.xlarge instance type in the EU (Ireland) region. We've added a few other filters to this example (the OS is Linux with no pre-installed software).

aws pricing get-products --region us-east-1 --service-code AmazonEC2 \
 --filters '[{"Field":"tenancy","Value":"shared","Type":"TERM_MATCH"},'\
'{"Field":"operatingSystem","Value":"Linux","Type":"TERM_MATCH"},'\
'{"Field":"preInstalledSw","Value":"NA","Type":"TERM_MATCH"},'\
'{"Field":"instanceType","Value":"c5.xlarge","Type":"TERM_MATCH"},'\
'{"Field":"location","Value":"EU (Ireland)","Type":"TERM_MATCH"}]'

{
    "FormatVersion": "aws_v1",
    "PriceList": [
        "{\"product\":{\"productFamily\":\"Compute Instance\",\"attributes\":{\"enhancedNetworkingSupported\":\"Yes\",\"memory\":\"8 GiB\",
        \"dedicatedEbsThroughput\":\"Upto 2250 Mbps\",\"vcpu\":\"4\",\"capacitystatus\":\"Used\",\"locationType\":\"AWS Region\",\"storage\":\"EBS only\",
        \"instanceFamily\":\"Compute optimized\",\"operatingSystem\":\"Linux\",\"physicalProcessor\":\"Intel Xeon Platinum 8124M\",\"clockSpeed\":\"3.0 Ghz\"
        ,\"ecu\":\"16\",\"networkPerformance\":\"Up to 10 Gigabit\",\"servicename\":\"Amazon Elastic Compute Cloud\",
        \"instanceType\":\"c5.xlarge\",\"tenancy\":\"Shared\",\"usagetype\":\"EU-BoxUsage:c5.xlarge\",\"normalizationSizeFactor\":\"8\",
        \"processorFeatures\":\"Intel AVX, Intel AVX2, Intel AVX512, Intel Turbo\",\"servicecode\":\"AmazonEC2\",\"licenseModel\":\"No License required\",
        \"currentGeneration\":\"Yes\",\"preInstalledSw\":\"NA\",\"location\":\"EU (Ireland)\",\"processorArchitecture\":\"64-bit\",\"operation\":\"RunInstances\"},
        \"sku\":\"39VH8W4ANB3RS4BD\"},\"serviceCode\":\"AmazonEC2\",\"terms\":{\"OnDemand\":{\"39VH8W4ANB3RS4BD.JRTCKXETXF\":
        {\"priceDimensions\":{\"39VH8W4ANB3RS4BD.JRTCKXETXF.6YS6EN2CT7\":{\"unit\":\"Hrs\",\"endRange\":\"Inf\",\"description\":\"$0.192 per On Demand Linux c5.xlarge Instance Hour\",
        \"appliesTo\":[],\"rateCode\":\"39VH8W4ANB3RS4BD.JRTCKXETXF.6YS6EN2CT7\",\"beginRange\":\"0\",\"pricePerUnit\":{\"USD\":\"0.1920000000\"}}},
        ...
    ]
}

This response contains all the required information (instance type characteristics along with EBS support, network performance, on demand pricing, reserved pricing), though it's a bit hard to parse because it's not a standard JSON response, instead, the PriceList field is a giant text field with escaped quotes. The only pieces of information we need that aren't included in the response are spot prices. Our service can query spot prices from two different sources. It can query EC2 directly, in which case the spot prices are fetched using a DescribeSpotPriceHistory request, and the current spot prices are stored. Spot prices change continuously, so they are updated every minute. The other way to query spot prices is by using Prometheus and our spot price exporter. The exporter uses the same EC2 API, but stores the prices as time series data that can be queried for averages, maximums and predictions. If Prometheus is configured with product info service, the spot price average of the last 24 hours is reported by the API, by default. The code to parse the getProducts output along with spot price handling can be found on Github.

Google Cloud

On Google Cloud we use two different APIs to collect the product information we need: the Cloud Billing API and the Compute Engine API. The Cloud Billing API can be used to programatically manage billing for different accounts, or to retrieve the full list of billable SKUs for a service. To get the pricing information for specific instance types, the compute service's SKU list must be parsed. Authentication to the catalog API is done through an API key that can be generated on the Google Cloud Console. If you have an API key, billing is enabled for the project. The Cloud Billing API is also enabled, so you can start using it. You can try to list all the services, first, to get the Compute Engine's service ID:

curl -s "https://cloudbilling.googleapis.com/v1/services?key=$GOOGLE_API_KEY"
{
  "services": [
    {
      "name": "services/6F81-5844-456A",
      "serviceId": "6F81-5844-456A",
      "displayName": "Compute Engine"
    },

Once you have the service ID, you can fetch the full list of billable SKUs in the Compute service. Here's what it looks like with an example SKU for a standard instance type running in the Japan region:

curl -s "https://cloudbilling.googleapis.com/v1/services/6F81-5844-456A/skus?key=$GOOGLE_API_KEY"
{
  "skus": [
    {
      "name": "services/6F81-5844-456A/skus/0186-15D9-9CAD",
      "skuId": "0186-15D9-9CAD",
      "description": "Standard Intel N1 32 VCPU running in Japan",
      "category": {
        "serviceDisplayName": "Compute Engine",
        "resourceFamily": "Compute",
        "resourceGroup": "N1Standard",
        "usageType": "OnDemand"
      },
      "serviceRegions": [
        "asia-northeast1"
      ],
      "pricingInfo": [
        {
          "summary": "",
          "pricingExpression": {
            "usageUnit": "h",
            "usageUnitDescription": "hour",
            "baseUnit": "s",
            "baseUnitDescription": "second",
            "baseUnitConversionFactor": 3600,
            "displayQuantity": 1,
            "tieredRates": [
              {
                "startUsageAmount": 0,
                "unitPrice": {
                  "currencyCode": "USD",
                  "units": "1",
                  "nanos": 952000000
                }
              }
            ]
          },
          "currencyConversionRate": 1,
          "effectiveTime": "2018-06-20T04:04:30.503Z"
        }
      ],
      "serviceProviderName": "Google"
    },
  ]
}

This list doesn't exclusively contain instance type information, but all available SKUs, like licensing fees for specific software, or network components. It doesn't contain exact machine type ids, only resource groups and a description. Our cloud info service parses all the machine type pricing info from the SKU list, and stores it in an in-memory cache. The above cURL example contains a response entry for machine type and, as you can see, it only contains billing info. It doesn't contain anything about machine type characteristics, like the cpu or memory available, so that information should be queried from a different API. The standard Compute Engine API has an endpoint that lists all the available machine types in a specific zone. This API performs authentication in the standard Google Cloud way with service accounts instead of API keys. It can be tried out with gcloud CLI by running this command:

gcloud compute machine-types list
NAME             ZONE                       CPUS  MEMORY_GB  DEPRECATED
f1-micro         us-central1-f              1     0.60
g1-small         us-central1-f              1     1.70
n1-highcpu-16    us-central1-f              16    14.40
n1-highcpu-2     us-central1-f              2     1.80
n1-highcpu-32    us-central1-f              32    28.80
n1-highcpu-4     us-central1-f              4     3.60
...

Our product info service parses this list along with the SKUs, merges the information, and stores it. The code to parse Google Cloud APIs can be found on Github.

Azure

Azure provides the required information in a manner very similar to that of Google Cloud, but makes it a bit harder to parse and merge responses. There are two different APIs here, as well, that provide machine type information and SKUs respectively. Machine types can be queried through the Compute API's list virtual machine sizes request. Using the Azure CLI, the command looks like this for the centralus region:

az vm list-sizes -l centralus
[
  {
    "maxDataDiskCount": 1,
    "memoryInMb": 768,
    "name": "Standard_A0",
    "numberOfCores": 1,
    "osDiskSizeInMb": 1047552,
    "resourceDiskSizeInMb": 20480
  },
  {
    "maxDataDiskCount": 2,
    "memoryInMb": 1792,
    "name": "Standard_A1",
    "numberOfCores": 1,
    "osDiskSizeInMb": 1047552,
    "resourceDiskSizeInMb": 71680
  },
...

Pricing info can be queried through the Rate Card API. The Rate Card API is very similar to the Google Cloud SKU list; it doesn't offer filtering or querying, so a large list of SKUs are returned for a specific Azure offer (in our case it's the Pay-as-You-Go offer with offer number 0003P). An example GET request and a sample of its response, which will return the SKUs with USD as its currency, looks like this:

https://management.azure.com/subscriptions/{subscription-Id}/providers/Microsoft.Commerce/RateCard?api-version=2015-06-01-preview&$filter=OfferDurableId eq ’MS-AZR-0003p’ and Currency eq ’USD’ and Locale eq ’en-US’ and RegionInfo eq ’US’
{
   "OfferTerms": [],
   "Meters": [
      {
         "MeterId": "1822fcc4-6059-4cbb-a132-54a187aaac46",
         "MeterName": "Compute Hours",
         "MeterCategory": "Virtual Machines",
         "MeterSubCategory": "Basic_D6 VM (Non-Windows)",
         "Unit": "Hours",
         "MeterTags": [],
         "MeterRates": {
            "0": 3.136
         },
         "EffectiveDate": "2015-02-01T00:00:00Z",
         "IncludedQuantity": 0.0
      },

This list contains a lot of entries about other SKUs, like networking solutions, so it must be parsed and filtered before being stored in our cache. The other problem with this list is that it's MeterSubCategory field is not entirely consistent with the machine size list API's instance type names. Some machine types have different naming conventions, like BASIC.A0 instead of Basic_A6 and some "sub-instance types" are missing from this list, like premium storage types that are suffixed with an s. The code to parse the Azure APIs can be found on Github.

Future plans

Some useful information is missing from our service, but we're planning to add it soon. We currently have only on-demand and spot prices, and are missing reserved instance pricing. We are also missing Windows pricing that differs from the Linux prices on every cloud provider. We are also planning to add more information about instance storage and networking, and to add filters and support for instance type comparisons between cloud providers and regions. If you'd like to learn more about Banzai Cloud check out our other posts on this blog, the Pipeline, Hollowtrees and Bank-Vaults projects.