Compute Engine Cost Optimization

Tudip

15 April 2020

Google Compute Engine (GCE) is the Infrastructure as a Service (IaaS) component of Google Cloud Platform. It lets you create and run virtual machines on Google infrastructure.

On Google Cloud Platform (GCP), a customer’s first step is often to adopt Compute Engine, which makes it easy to procure and set up virtual machines (VMs) in the cloud. Compute Engine offers multiple machine types, many various features (such as OS, customized machine types and more) and is available in 20 regions and 61 zones!

Here in this blog, we see some best practices for the cost optimization of Compute Engine.

• Compute Engine Rightsizing

  Compute Engine provides machine type rightsizing recommendations to help you optimize the resource utilization of virtual machine (VM) instances. These recommendations are generated automatically based on system metrics gathered by the Stackdriver Monitoring service over the previous eight days. Use these recommendations to resize your computer instance’s machine type to more efficiently use the instance’s resources.

  It also recommends Custom Machine Types which is a feature of Compute Engine that lets you easily create a customised machine type to your needs. Using Custom Machine Types, you can create virtual machines with the optimal amount of CPU and memory for your workloads.

  Compute Engine makes viewing, resizing and other actions easier from the Cloud Console as shown below.

  

• Purchase Committed Discount

  Committed Use Discounts on Compute Engine offers the ability to purchase committed use contracts in return for deeply discounted prices for VM usage.

  If you have a predictable steady-state workload then Committed Use Discounts are ideal, as you can purchase a one or three year commitment in exchange for a substantial discount on your VM usage.

  These Committed discounts are ideal for workloads with predictable resources. When you purchase a committed use contract, you purchase a certain amount of resources such as memory, vCPUs GPUs, and local SSDs at a discounted price. In this, you are committing to paying for those resources for 1 year or 3 years. The discount is ~57% for most resources like machine types or GPUs. Also, for the memory-optimized machine types the discount is up to 70%.

  Once purchased, you are billed monthly for the resources you purchased.
  

• Auto Cost Optimizations

  The one of the best practices for cost optimization is to automate the tasks and reduce manual intervention.

  Automation is simplified using a label – which is a key-value pair applied to various Google Cloud services. You can attach a label to each resource (such as Compute instances), then filter the resources based on their labels. For example, you could label instances that only testers use during business hours with “env: testing”. Then, you could use Cloud Scheduler to schedule a serverless Cloud Function to shut them down after business hours or over the weekend and then restart them when needed.

  To automate the cleanup of other Compute Engine resources, using Cloud Functions can also save you a lot of time and money.

  For example, customers often forget to delete unattached persistent disks, or about unused IP addresses. They will accumulate costs, even if they are not attached to a virtual machine instance. If the “deletion rule” option is set to “keep disk”, then VMs retain persistent disks even after the VM is deleted. That’s nice if you require to save data for later time, but those orphaned persistent disks will quickly add up, and are often overlooked!

• Use of Preemptible VMs

  You can create and run preemptible VM instances at a much lower cost than normal instances. However, if it requires access to those resources for other events, Compute Engine might preempt these instances at any time. It might vary from day to day and from zone to zone depending on current conditions of the resources.
  If your apps are fault-tolerant and can resist instance preemptions, then preemptible.

  e instances can reduce your Compute Engine costs significantly. For batch processing, you can run jobs on preemptible instances. During processing, some of the instances terminate but it does not completely stop the jobs. Preemptible instances complete batch processing tasks without placing additional workload on existing instances and you no need to pay full price for additional normal instances.

  Preemptible instances always run for a maximum of 24 hours. It gets terminated by sending 30 second preemption notice to the instance, so you can use a shutdown script to clean up in that 30 second period. Preemptible instances might not always be available as they are finite Compute Engine resources.

• Autoscaling

  Another way to save on costs is when you need to run just as much capacity as you need. Usually about 70 percent of the capacity is used for steady-state use, but it’s important to have it available when you need extra capacity. You need to buy the extra capacity in advance in an on-prem setting. In the cloud, autoscaling can only be leveraged to automatically flex to increase capacity when you need it.

  Instance groups managed by compute engine are what offer you this autoscaling capability in Google Cloud. You can gracefully scale up to handle an increase in traffic, and then scale down again automatically when the need for instances (downscaling) is smaller. Based on CPU usage, HTTP load balancing power, or Stackdriver Monitoring metrics, you can scale.

Billing Reports

As discussed above, there are many ways to optimize your Compute Engine costs. Addition to this, another important step to check your Compute Engine cost is using Billing reports in the Google Cloud Console. You can customize your views based on filtering and grouping by projects, labels and more.

Along with this, you can export Compute Engine usage details to BigQuery for more granular analysis. Cloud Billing export to BigQuery enables you to export detailed billing data (such as usage and cost estimate data) automatically to a BigQuery dataset and allows you to query on the dataset.

If you have set a billing threshold amount, you set budget alert rules that are used to trigger notifications (such as emails, messages), so that you will be notified about how your spend is tracking against your budget.