Understanding AWS Billing
Reserved Instance Overview
To understand AWS billing, we need to explain how consolidated billing and reserved instances work. Think about buying a RI as buying a coupon. Let’s suppose you are running a m3.medium on Linux. The on-demand rate for this is $0.067 per hour. In this case, running the instance for 1 month costs you $48.24 in a month with 720 hours. If you purchase a 1-Year Partial Upfront RI, you will pay $211 upfront to get a discounting hourly rate of $0.017. For the first month you pay $211 + $12.41 but each month after that you only pay $12.41 instead of the on-demand month rate of $48.24. After 6 months, you will break even and each month after that you will be saving $35.83. However, there is one caveat about purchasing RIs – they are a commitment. You pay the $12.41 per month even if you don’t use the instance for the lifetime of the RI.
Another way that RIs are like coupons is that if you don’t use them, you lose them. You can use 1 RI coupon for 1 instance per hour. If you use 0 instances in one hour and 2 instances in the next hour, you get to apply 0 of 1 available RI instance hours for the first hour and 1 of 1 RI instances hours for the second hour. You effective lose an RI instance hour even though you have 2 hours. This can get quite complicated tracking, purchasing, and staying on top of environments that are ephemeral, auto scaling, and elastic (which are 3 defining characteristics of the cloud).
Consolidated Billing Overview
Consolidated billing is how AWS allows you to tie multiple AWS accounts together into groups. Within a consolidated billing family, you have one payer and one or more payees. The payer is responsible for paying the bill, and the Detailed Billing Report (the record of all charges) is generated and deposited in this AWS account only. Using consolidated billing is valuable because the billing engine treats all the AWS accounts as a single account for billing purposes, resulting in the lowest possible bill. For instance, RIs purchased in one AWS account can share their RIs with other AWS accounts in the billing family. As well, pricing tiers are applied across the entire consolidated billing family. S3 pricing is based on usage. 0-10TB is $0.10 per GB. By consolidated AWS accounts, billing is based on the combined S3 usage which compresses the pricing tiers and results in lowering the bill.
As an AWS partner, it is fairer to charge your customers based on AWS “list” pricing as if the account is stand alone. A single AWS account should pay its fair share of AWS resources. Under the AWS billing model, AWS accounts could be charged less because they get to “borrow” unused RIs others used, or they get the lower pricing tiers because of the usage of other customers. While this ensures everyone always pays the lowest rate possible, in a Reseller or MSP model individual accounts can end up with lower rates than they should.
Blended rates are based on averaging the usage rate of similar resources across all the AWS accounts. This factors in the average on-demand and Reserved pricing into a common rate that is applied to all like-usage across the consolidated billing family. This means a few things – an AWS account that purchases RIs will show a blended rate HIGHER than the rate they purchased for the RI. AWS accounts that did not purchase RIs will see lower rates than on-demand because the RIs are averaging/lowering the on-demand rate. From the perspective of a cloud broker, this blended rate is not something you can use to bill your customers.
Blended rate is useful in a scenario in which an organization purchases RIs in a central account and wants to share that benefit with all the usage across the entire consolidated billing family. If an account purchases an RI within their account using their own budget, they will recognize very little benefit in the blended rate. Note, the organization overall sees the benefit, but the individual project or teams purchasing the RIs see very little benefit.
Unblended rate are more useful. They show the rates of the actual usage. However, unblended costs do create effects on individual payees lowering their bill by sharing RIs across AWS accounts and compressing tiered costs. Unblended costs are a better method of charging individual payees, but still is not accurate.
CloudCheckr’s List Cost
CloudCheckr allows you to translate the usage charges within AWS into costs as if each payee account were stand-alone. This allows an AWS partner, such as reseller or managed service provider, to create invoices that correctly represent the cost had the payee account not been in the AWS partner’s consolidated billing family. This is a more “accurate” cost to report to a payee over blended or unblended costs.
AWS consolidated billing families create many discounts within payees. CloudCheckr list costs reserves many of these discounts and creates an invoice that an AWS partner can fairly present to an end customer that is a payee of the consolidated billing family. The AWS partner still pays AWS the lowest possible cost, but the benefit of consolidating the accounts is given to the AWS partner rather than arbitrarily to the payee accounts.
List cost provides the following features:
1) Reverses out EC2 Reserved Instance charges that a stand-alone account should not receive.
CloudCheckr adjusts the EC2 RI usage costs in payees. First it changes any charges for EC2 usage within the payee for RIs that are not in the payee. If the payee purchased a reserved instance, CloudCheckr considers that the RI price should be honored and does not modify the price of that usage. AWS has account affinity for RIs meaning that it first attempts to apply the RI to any EC2 usage inside the payee that purchased an RI. If there is not a match for the hour for the RI within the payee, AWS then tries to apply the RI to other EC2 usage in other payees. There is no guarantee of how or to which EC2 usage or payees to which it will be applied.
If CloudCheckr finds that a payee received a discounted rate because the AWS billing engine gave it the RI rate from another payee that had an unused RI, it will recalculate that charge reversing the lower rate. If the payee had no matching RIs of its own, it would be recalculated as the on-demand rate.
Correcting RI usage can be complex, as there are many situations to handle. For instance, if the payee purchased a No Upfront RI within the payee account, the AWS billing engine could use All Upfront or Partial Upfront pricing for unused RIs in other payee accounts. In this case, List price should NOT modify the usage cost to the on-demand rate but must honor the No Upfront hourly rate. All the various scenarios and permutations must be considered and handled to create an accurate invoice for the payee of the end customer.
CloudCheckr also checks for any Heavy Utilization, No Upfront, Partial Upfront, and All Front RI usage for which the payee purchased but did not use. Often payees purchase an RI and then do not fully use the RI. When this happens, it is the obligation of the purchaser of the RI to pay for the hour whether they are used or not. This is a risk accepted for the lower RI usage rate.
In a consolidated billing family, if a payee has unused RIs those could be used by other payees with matching EC2 usage. In the Detailed Billing Report, the hourly rate for the usage will be included in the Blended and Unblended costs in the other payee that used the RI. This results in the payee that purchased the RI not being charged for all hours committed to. However, List cost will ensure the cost of unused hours for the payee are included in the List cost (even though they are not included in the Blended and Unblended costs). This enforces the principle that the payee that purchased the RI is obligated to pay for all hours used and unused by the RI, even if another payee used the hours.
2) Reverses out RDS Reserved Instance charges that a stand-alone account should not receive.
The same ideas apply to purchasing Reserved DB Instances for RDS. AWS initially offered Heavy Utilization, Medium Utilization, or Light Utilization Reserved DB instances. Heavy Utilization came with the commitment to run for every hour of the month. However, Light Utilization and Medium utilization does not commit the purchaser to use every hour. In summer of 2015, AWS added the capability to allow new purchases of All Upfront, Partial Upfront, and No Upfront RDS Reserved Instances. This new model is based on committing to 24×7 usage. Even if you shutdown the DB Instance, you will pay for the hours for the RI regardless. You should be able to properly allocate either the old generation RI types or the new generation RI types.
Just like EC2 RI usage, List cost in CloudCheckr will convert RI usage based on other RIs from other payees and will commit the payee to be charged for the RIs purchased even if not consumed in the payees account.
3) Reverses out tiered pricing that a stand-alone account should not receive.
AWS provides decreasing pricing for high levels of usage for several services. This is referred to as tiered pricing. The most common of these services are S3, CloudFront, and Data Transfer. Tiered pricing is applied across the usage for the entire billing family. What this means is that is one payee’s usage for a service can effectively decrease the usage rate for all other payees. As an AWS partner this is useful to maximize the overall price. But each payee should be charged based on their own specific usage of the service.
As an example, consider S3 pricing here.
First 1 TB / month $0.0300 per GB
Next 49 TB / month $0.0295 per GB
Over 5000 TB / month $0.0275 per GB
For complete details on S3 pricing, check here: https://aws.amazon.com/s3/pricing/
If one payee used over 1 TB of S3 storage, other payees end up paying starting at the $0.0295 rate. List cost in CloudCheckr sets the pricing for S3 for each payee to the actual usage of the payee only. If the payee’s own usage crosses discount tiers, those discounts will be applied. This idea is applied through CloudCheckr list price across S3, data transfer, and CloudFront.