CloudWatch Custom Metrics

What Are Custom Metrics?

Standard/default CloudWatch metrics are ones that AWS makes available out-of-the-box: for EC2 instances, these are metrics related to your CPU and network utilization. However, to get better insight into your resources, you can enable custom metrics that will give you insight into, for example, items such as your EC2 instance’s memory utilization and memory used/available.

How Do I Get Custom Metric Data Into CloudCheckr?

Overview

To access custom metric data, you will need to create the mechanism by which the metric is generated, such as by installing an agent on an EC2 instance to generate memory metrics. Once your resource is generating the desired metric, the data is sent to CloudWatch and can next be ingested by CloudCheckr where it is used for various reporting and alerting features.

Some of the Most Popular Custom Metrics

Some of the most popular custom metrics are the ones specific to EC2 Memory. These help you see the big picture with regards to how your instances is truly being utilized. Without the memory metrics, you will only have insight into the default CloudWatch metrics concerning CPU and Network data. Having the third piece of Memory is key to a holistic view.

The EC2 Custom Memory metrics are:
System/Linux MemoryAvailable
System/Linux MemoryUsed
System/Linux MemoryUtilization
System/Windows MemoryAvailable
System/Windows MemoryUsed
System/Windows MemoryUtilization

Workflow

The general workflow for custom metrics is as follows:

cwm01_workflow

Getting EC2 Memory Metrics into CloudWatch

AWS provides various procedures for setting up and generating your custom metrics.

Using EC2 as an example, you will need to manually prepare your instance to send the data by installing an agent or monitoring scripts.

PART 1

Setup for Linux Instances

You will need to install monitoring scripts on your Linux EC2 instances, which will send the memory metrics to CloudWatch. AWS provides documentation here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html

Setup for Windows Instances

You will need to use the EC2Config service to export performance counter data to CloudWatch. You will want to send the available memory data. AWS provides documentation here: http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/UsingConfig_WinAMI.html

PART 2

After you have configured your instances to export memory metrics to CloudWatch, go to the AWS Console and access the CloudWatch section. Choose EC2 and verify that you are seeing your new memory metrics.

cwm02_SearchForEC2MemMetric

PART 3

Lastly, from within CloudCheckr navigation column, go Utilization > Manage Custom Metrics. You will be taken to the default tab EC2 Memory Metrics. Within this tab, turn on the Memory metrics that you have just enabled within AWS Console. Linux and Windows memory metrics each have three possible selections.

CloudCheckr will begin to download the EC2 memory metrics from AWS and will populate various reports with the data.

CloudCheckr Reports and Features That Use Custom Metrics

In general, memory metrics will help inform any reports that alert you to under-utilized or idle resources, giving you a fuller understanding of your EC2 instances behavior so you are aware of CPU, network, and memory activity. Some of the reports that use this data are:

EC2 Instance Rightsizing Report

You will get data on your instances’ 30-day peak and average memory usage, indicating whether it is doing memory intensive operations – even if CPU or network usage is low – and thus giving you a true understanding of the instance’s utility.

cwm03_Ec2RightSizing

CloudWatch Historical Export

While the AWS Console stores only 14 days’ worth of data, CloudCheckr stores all of your CloudWatch metrics for the life of your account. This is accessible here.

cwm04_CWHistExport