Skip to main content

Estimate and monitor your cloud spending

Context​

Nowadays, the cloud is used to reduce our total cost of ownership of our IS/IT assets. Transferring operational activities to cloud providers or transversal teams, or benefiting from dynamic sizing of the infrastructure resources. Rather than performing capacity planning, the challenge is nowadays to track costs and try to anticipate the cloud spent.

K8SaaS provides a billing API that allows you to get insight on the pricing:

  • v2: you get the raw cloud consumption of k8saas per day, without the "added value" price of the trustnest platform

This API is exposed in our API portal: here

here

Use case​

note

This feature is only accessible to @thalesdigital.io account for the moment.

  • Get the price of the k8saas for a given day
  • Get data every day to monitor the trends
  • Get cost for a period of days to estimate his bill
  • Understand your trustnest billing

What to do ?​

First you need to know you TDFAccountID

Then, go to TrustNest K8SaaS Service catalog - Access to my KaaS billing API. Here are the information requested:

  • the TDFAccountID
  • the name of the k8saas instance
  • the email address of the people who need access to the billing API

Once the configuration done, follow the steps:

note

Once the ticket is sent it will need to be approved by the TDF accountID Owner before any access are given. You will also need to sign in at least once in the API Management portal to create your user so permission can be linked the user provided for access. Follow the documentation bellow.

Access to the queries using API Portal​

Access to API portal using API Management portal. This portal shows all the transversal APIs that k8saas exposes. You should see Billing API v2 and Customer Apis.

warning

Billing v2 are restricted to Trustnest Billing Team. Only Customer Apis Access will be granted to you.

Sign in​

Click on sign in, then Azure Active Directory. Indeed, the API Management portal is integrated with the Azure Active Directory of the platform where all the Thales employees are registered.

img

Access to the API​

Once allowed on a specific product, you have access to all APIs inside, especially "Customer Apis". API Management portal provide a way to test the APIs using the button "Try it" in green.

tip

Tip: When you click on "Try it", by default it shows you HTTP request, but you have a menu to see other examples of code to query the API (Curl, Java, Python, Javascript etc...)

img

note

Note: Date format is: YYYY-MM-DD

Create your Product Subscription Key​

You will need to create your own Subscription key to test and perform any request on the API. Click on "Products" at the top of the page. Click on "Billing v2". Enter your subscription name in the Box mention "Your new product subscription name" and click on the button "Subscribe". Your key will be generated and they must be validated by the KAAS Team. Once allowed, you will have access to selected API.

img

Access to the queries using Curl​

API portal provides a way to automate the monitoring of the costs:

export APIM_SUBSCRIPTION_KEY="xxxxx"
export BILLING_DATE="YYYY-MM-DD"
curl -X GET "https://k8saas-internal-tooling-apim.azure-api.net/Consumer/billing/$BILLING_DATE" -H "Cache-Control: no-cache" -H "Ocp-Apim-Subscription-Key: $APIM_SUBSCRIPTION_KEY"

Supported request​

https://k8saas-internal-tooling-apim.azure-api.net/Consumer/billing/YYYY-MM-DD

Reserved instance​

It's possible to use Azure reserved instances feature to reduce your bill (available only in industrialize service offer). Please raise a postIT ticket to ask for it !

There are two types of reserved instance:

  • Reserved 1 year: Reserve one or several Azure machines for 1 year.
  • Reserved 3 years: Reserve one or several Azure machines for 3 years.

Of course, using of this option binds you with a 1 or 3 years contract. However the longer you commit yourself, the more money you save.

warning

If you reserved a machine for 1 or 3 years, you will pay for it even if you stop using it.

On the billing API response, there is a list of all reserved instances for each cluster, with the following fields:

"reserved_instances": [
   {
      "end_date": "2024-11-01T00:00:00Z",
      "start_date": "2021-11-02T00:00:00Z",
      "type": "RESERVED_INSTANCE_3_YEARS"
   }
]

with for each reservation:

  • end_date: end date for the reservation instance
  • start_date: start date for the reservation instance
  • type: either "RESERVED_INSTANCE_1_YEAR" or "RESERVED_INSTANCE_3_YEARS"
tip

The reduction has been already applied to the billing api output, the information is only here to inform you on the active reservations.

Detailed billing​

The billing API main endpoints allow you to easily know the price for each of your clusters for a specific date or time range. However, this output is short to easily get an overview of each clusters' price.

The goal of the detailed endpoints is to provide a better understanding of the clusters cost by showing the price for every resource groups and every Azure service.

On the billing API response, the following fields are added:

"meters_details": [
   ...
   "resource_groups": [
      {
            "name": "k8saas-resource-group",
            "services": {
               "Event Hubs": "XX.XXXXX",
               "Log Analytics": "XX.XXXXX",
               "Network Watcher": "XX.XXXXX",
               "Service Bus": "XX.XXXXX",
               ...
            },
            "total_cost": "XXX.XXXXX",
      },
      ...
   ]
]

with for every cluster a field resource_groups that list all of the cluster resource groups.

And for every resource group:

  • name: the resource group name;
  • services: the list of every Azure service price;
tip

The list is dynamic: Only the Azure services that actually cost something during the requested period are displayed.

  • total_cost: the total cost for the resource group.

Monitoring with Grafana​

warning

This feature is currently under development and made available to you here in early access. There's currently an issue with version of kube state metric and the datasource use to display the dashboard, the issue should be fixed whenever your cluster will be updated to AKS >= 1.29

In Grafana, in the folder K8saas, there is the dashboard K8saas / Billing.

This dashboard sends requests to the billing API on your behalf and displays your detailed costs in graphs.

However, in order to send data to the API, you need to fill in your subscription key.

There is a parameter field on the top-left corner of the dashboard:

Billing Grafana Dashboard

You can find your own key value when requesting the k8saas billing API with the Billing API Portal:

Billing API Portal

Once Activated with the subscription keys, the panels will load and you'll have your detailed billing information by day for the last month or by month for the last year.

Here is an example of costs for one cluster for the last month:

Billing Grafana Dashboard last month

The panels are in order:

  • Azure resource groups cost by day over the last month;
  • Azure service cost by day over the last month. The list of Azure service is dynamic and depends of the services used during the time range;
  • Total cost over the last month and the currency;
  • Distribution of resource group cost over the last month;
  • Distribution of Azure service cost over the last month.

And for the information about last year consumption:

Billing Grafana Dashboard last year

The panels are the same as for the section "last month", but the information is gathered by month over the last year. In this section there is also one extra panel at the bottom: any active reserved instance during the year will be display here.

How to save your subscription key​

  1. Get OcpApimSubscriptionKey from the previous step
  2. Load K8saas / Billing dashboard
  3. Edit Url by adding the following value 
    &var-OcpApimSubscriptionKey=<OcpApimSubscriptionKey_value>
  4. Save a bookmark of the edited url

Next steps​