What Are Custom Metrics?
Standard/default CloudWatch metrics are ones that AWS makes available out-of-the-box: for EC2 instances, these are metrics related to your CPU and network utilization. However, to get better insight into your resources, you can enable custom metrics that will give you insight into, for example, items such as your EC2 instance’s memory utilization and memory used/available. Additionally, CloudCheckr can ingest all of these CloudWatch metrics and store them for you for the life of your account, maintaining a searchable record of all your CloudWatch metrics.
How Do I Get Custom Metric Data Into CloudCheckr?
For most custom metrics, you will need to turn on Detailed Monitoring within AWS CloudWatch and then set up CloudCheckr to look for the exact metrics you wish to have.
For custom metrics related to EC2 instance memory, you will need to install an agent on the EC2 instance to generate the memory metrics. Once your resource is generating the desired metric, the data is sent to CloudWatch and can next be ingested by CloudCheckr where it is used for various reporting and alerting features.
How To Add Default Custom Metrics from CloudWatch
As mentioned above, in addition to metrics that need a separate agent installed on a resource, AWS has a set of “default” custom metrics that offer extra insight with relatively little setup.
AWS has a list of custom metrics that can be used for EC2 instances:
For our example we will collect CloudWatch data for the custom metric called CPUCreditUsage.
- Turn on detailed monitoring for your specific instance within AWS Console.
- Make sure you have your CloudWatch permissions up-to-date within CloudCheckr. If not, see this page to update your account. If your CloudCheckr yellow notification icon has a number next to it, you’ll know there are certain permissions your account needs to have updated.
- Go to the Manage Custom Metrics page located at Utilization > Alerts > Manage Custom Metrics.
- Select the Custom Metrics tab.
- Add a Namespace, Metric, and Dimension. These entries must exactly match the entries in AWS. In our example, use the following:
- Namespace: AWS/EC2
- Metric: CPUCreditBalance
- Dimension: * (Note: inputting an asterisk makes the app input all dimensions for this metric.)
- Click the Add Custom Metric button. Your Custom Metric will be added and you will see that its status switch is set to ON.
- Your CloudWatch Custom Metrics will populate within the next 24 hours, depending on when your last CloudCheckr report update occurred. Alternatively, you can manually trigger a report update by clicking on the Reports Updated notification button at the top right of your screen. Depending on the size of your account, the update will take a few hours.
How To Add EC2 Memory Metrics into CloudWatch
To get custom CloudWatch memory metrics, you will need to install an agent/monitoring script on your EC2 instance (Note: the role of the agent is to specifically monitor memory — this is a functionality that is not available by default). This agent will send send the data to CloudWatch, and once within CloudWatch the data can be ingested by CloudCheckr.
The general workflow for custom metrics that need an installed agent is as follows:
The CloudWatch Custom Metrics that can report on instance memory are:
- System/Linux MemoryAvailable
- System/Linux MemoryUsed
- System/Linux MemoryUtilization
- System/Windows MemoryAvailable
- System/Windows MemoryUsed
- System/Windows MemoryUtilization
Setup for Linux Instances
You will need to install monitoring scripts on your Linux EC2 instances, which will send the memory metrics to CloudWatch. AWS provides documentation here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html
Setup for Windows Instances
You will need to use the EC2Config service to export performance counter data to CloudWatch. You will want to send the available memory data. AWS provides documentation here: http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/UsingConfig_WinAMI.html
- After you have configured your instances to export memory metrics to CloudWatch, go to the AWS Console and access the CloudWatch section.
- Choose EC2 and verify that you are seeing your new memory metrics.
- From within CloudCheckr, go to Utilization > Manage Custom Metrics. You will be taken to the default tab EC2 Memory Metrics.
- Turn on the Memory metrics that you have just enabled within AWS Console. Linux and Windows memory metrics each have three possible selections.
- CloudCheckr will begin to download the EC2 memory metrics from AWS and will populate various reports with the data.
CloudCheckr Reports and Features That Use Custom Metrics
In general, memory metrics will help inform any reports that alert you to under-utilized or idle resources, giving you a fuller understanding of your EC2 instances behavior so you are aware of CPU, network, and memory activity. Some of the reports that use this data are:
EC2 Instance Rightsizing Report
You will get data on your instances’ 30-day peak and average memory usage, indicating whether it is doing memory intensive operations – even if CPU or network usage is low – and thus giving you a true understanding of the instance’s utility.
CloudWatch Historical Export
While the AWS Console stores only 14 days’ worth of data, CloudCheckr stores all of your CloudWatch metrics for the life of your account. This is accessible here.