When we combine the cloud with IaC tools like Terraform and continuous deployment we get the almost magical ability to create resources on demand. For all its benefits, however, the cloud is not without its share of problems β like estimating cloud costs accurately. Can we automate this tedious task? Yes, thanks to open source Infracost.
Why Cloud Costs Are Hard
Cloud providers have complex cost structures that are constantly changing. AWS, for example, offers more than 700 types of Linux machines on EC2. Many of them have similar names and features. Take for example “m6g.2xlarge” and “m6gd.2xlarge” (one comes with an SSD while the other doesn’t). Making a mistake in a Terraform file can cause your bill to balloon unexpectedly.
β

β
We can set up billing alerts, but they are reactive and with no guarantee it they will work. It has already happened multiple time to developers and companies.
What is Infracost?
Infracost is an open-source project that helps us understand how and where weβre spending our money. It gives a detailed breakdown of our infrastructure costs and calculates the impact of changes. In a nutshell, Infracost is a git diff
for cloud billing.
Infracost has two versions: a VSCode extension and a command line program. Both do the same thing: parse Terraform or Terragrunt, pull the current cost price points from a the Infracost cloud pricing API, and output an estimate.
β

β
With the extension, we can see the estimates right in the IDE as we make changes.
β

β
The command line tool lets us post comments in pull requests and commits in our repos.
β

β
Infracost also has an optional (a paid) Infracost Cloud, which acts as a central hub for cost management. Here, we can set usage estimates, set custom price books and keep track of costs over time.
β

Getting the Infracost Cloud also gives us some extra perks such as Slack Integration, setting up FinOps policies and establishing guardrails to prevent deployments beyond a preset budget.

Installing up Infracost
To try out Infracost, weβll need the following:
- An Infracost API key. You can get one by signing up for free by installing the Infracost CLI and running:
infracost auth login
- Some Terraform files.
- A GitHub account to post estimates messages.
The first command weβll try is infracost breakdown
. It analyzes Terraform plans and prints out a cost estimate. The --path
variable must point to the folder containing your Terraform files. For example, imagine we want to provision an “a1.medium” EC2 instance with the following:
provider "aws" {
region = "us-east-1"
skip_credentials_validation = true
skip_requesting_account_id = true
}
resource "aws_instance" "myserver" {
ami = "ami-674cbc1e"
instance_type = "a1.medium"
root_block_device {
volume_size = 100
}
}
When we run `infracost breakdown –path .` we get an estimate for this instance:
$ infracost breakdown --path .
Evaluating Terraform directory at .
β Downloading Terraform modules
β Evaluating Terraform directory
β Retrieving cloud prices to calculate costs
Project: TomFern/infracost-demo/ec2
Name Monthly Qty Unit Monthly Cost
aws_instance.myserver
ββ Instance usage (Linux/UNIX, on-demand, a1.medium) 730 hours $18.62
ββ root_block_device
ββ Storage (general purpose SSD, gp2) 100 GB $10.00
OVERALL TOTAL $28.62
ββββββββββββββββββββββββββββββββββ
1 cloud resource was detected:
β 1 was estimated
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββββ
β Project β Monthly cost β
β£ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ«
β TomFern/infracost-demo/ec2 β $29 β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ»βββββββββββββββ
βIf we add some extra storage (600GB of EBS), the cost increases to $89, as shown below:
infracost breakdown --path .
Evaluating Terraform directory at .
β Downloading Terraform modules
β Evaluating Terraform directory
β Retrieving cloud prices to calculate costs
Project: TomFern/infracost-demo/ec2-disk
Name Monthly Qty Unit Monthly Cost
aws_ebs_volume.extra_storage
ββ Storage (general purpose SSD, gp2) 600 GB $60.00
aws_instance.myserver
ββ Instance usage (Linux/UNIX, on-demand, a1.medium) 730 hours $18.62
ββ root_block_device
ββ Storage (general purpose SSD, gp2) 100 GB $10.00
OVERALL TOTAL $88.62
ββββββββββββββββββββββββββββββββββ
3 cloud resources were detected:
β 2 were estimated
β 1 was free
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββββ
β Project β Monthly cost β
β£ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ«
β TomFern/infracost-demo/ec2-disk β $89 β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ»βββββββββββββββ
Infracost can also calculate usage-based resources like AWS Lambda. Let’s see what happens when we swap the EC2 instance for serverless functions:
provider "aws" {
region = "us-east-1"
skip_credentials_validation = true
skip_requesting_account_id = true
}
resource "aws_lambda_function" "my_lambda" {
function_name = "my_lambda"
role = "arn:aws:lambda:us-east-1:account-id:resource-id"
handler = "exports.test"
runtime = "nodejs12.x"
memory_size = 1024
}
Running infracost breakdown
yields a total cost of 0 dollars:
$ infracost breakdown --path .
Evaluating Terraform directory at .
β Downloading Terraform modules
β Evaluating Terraform directory
β Retrieving cloud prices to calculate costs
Project: TomFern/infracost-demo/lambda
Name Monthly Qty Unit Monthly Cost
aws_lambda_function.my_lambda
ββ Requests Monthly cost depends on usage: $0.20 per 1M requests
ββ Ephemeral storage Monthly cost depends on usage: $0.0000000309 per GB-seconds
ββ Duration (first 6B) Monthly cost depends on usage: $0.0000166667 per GB-seconds
OVERALL TOTAL $0.00
ββββββββββββββββββββββββββββββββββ
1 cloud resource was detected:
β 1 was estimated
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββββ
β Project β Monthly cost β
β£ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ«
β TomFern/infracost-demo/lambda β $0.00 β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ»βββββββββββββββ
That canβt be right unless no one uses our Lambda function, which is precisely what the tool assumed here. We can fix this by providing an estimate via a usage file. Run this command to create the usage file:
$ infracost breakdown --sync-usage-file --usage-file usage.yml --path .
We can now provide estimates by editing usage.yml
. The following example consists of 5 million requests with an average runtime of 300 ms:
# usage.yml
resource_usage:
aws_lambda_function.my_lambda:
monthly_requests: 5000000
request_duration_ms: 300
Weβll tell infracost to use the usage file with --usage-file
to get a proper cost estimate:
$ infracost breakdown --path . --usage-file usage.yml
Evaluating Terraform directory at .
β Downloading Terraform modules
β Evaluating Terraform directory
β Syncing usage data from cloud
ββ Synced 0 of 1 resource
β Downloading Terraform modules
β Evaluating Terraform directory
β Retrieving cloud prices to calculate costs
Project: TomFern/infracost-demo/lambda
Name Monthly Qty Unit Monthly Cost
aws_lambda_function.my_lambda
ββ Requests 5 1M requests $1.00
ββ Duration (first 6B) 1,500,000 GB-seconds $25.00
OVERALL TOTAL $26.00
ββββββββββββββββββββββββββββββββββ
1 cloud resource was detected:
β 1 was estimated
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββββ
β Project β Monthly cost β
β£ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ«
β TomFern/infracost-demo/lambda β $26 β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ»βββββββββββββββ
Thatβs much better. Of course, this is accurate as long as our usage file is correct. If youβre unsure, you can integrate Infracost with the cloud provider and pull the utilization metrics from the source.
Estimating The Impact of Changes
Infracost can save results in JSON by providing the --format json
and --out-file
options. This gives us a baseline file we can check in source control:
$ infracost breakdown --path . --format json --usage-file usage.yml --out-file baseline.json
We can now compare changes by running infracost diff
. Letβs see what happens if the Lambda execution time goes from 300 to 350 ms:
$ infracost diff --path . --compare-to baseline.json --usage-file usage.yml
Evaluating Terraform directory at .
β Downloading Terraform modules
β Evaluating Terraform directory
β Retrieving cloud prices to calculate costs
Key: * usage cost, ~ changed, + added, - removed
ββββββββββββββββββββββββββββββββββ
Project: TomFern/infracost-demo/lambda
~ aws_lambda_function.my_lambda
+$4 ($26 β $30)
~ Duration (first 6B)
+$4 ($25 β $29), +250,000 GB-seconds (1,500,000 β 1,750,000)*
Monthly cost change for TomFern/infracost-demo/lambda
Amount: +$4 ($26 β $30)
Percent: +16%
ββββββββββββββββββββββββββββββββββ
Key: * usage cost, ~ changed, + added, - removed
1 cloud resource was detected:
β 1 was estimated
Infracost estimate: Monthly cost will increase by $4 β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββ³βββββββββββββββββββ
β Project β Cost change β New monthly cost β
β£ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ«
β TomFern/infracost-demo/lambda β +$4 (+16%) β $30 β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββ»ββββββββββββββ»βββββββββββββββββββ
As you can see, the impact is a 16% increase.
Running Infracost on CI/CD
Weβve seen how this tool can help us estimate cloud costs. Thatβs valuable information, but what role does Infracost take in continuous integration? To answer that, we must understand what infracost comment
does.
The comment command takes a JSON file generated by infracost diff
and posts its contents directly into GitHub, Bitbucket, Azure Pipelines, or Gitlab. Running Infracost inside CI makes relevant cost information available to everyone on the team.
β

β
How to run Infracost on Semaphore
β οΈ Before going into pipeline configuration, commit any files youβve created into the repo. Also, if this is your first time using Semaphore, I suggest signing up for a free Infracost.io account and going through the getting started guide.
In this section, weβll add two cost-control jobs to our CI pipeline. They will:
- Analyze pull requests and post a comment in GitHub with the cost difference.
- Comment on every commit that changes the infrastructure. The job will fail if it breaks the pre-established policy.
β

β
For this section of the tutorial, youβll need to create a GitHub token (choose the classic token) or a Bitbucket app password with read+write access to the repository.
Weβll save the Git provider password or token as a Semaphore Secret. To do that, go to your Organization menu in the top right corner and click Settings. Then, go to Secrets > Create new.
Define the following secret environment variables:
INFRACOST_API_KEY
: you can retrieve it by runninginfracost configure get api_key
in your machine.GITHUB_API_KEY
orBITBUCKET_API_KEY
: an API token with read and write access to the repository. Note that, for Bitbucket, the key takes the form ofusername:api-key
.
β

Commenting commits
We’ll begin by adding a job that comments on every commit changing a Terraform file.
β

β
To do so, open or add your project to Semaphore. To keep things simple, we’ll assume we already have a CI pipeline that builds and tests the project.
β

β
Add a new block with the following commands:
curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh
checkout
infracost diff --path . --usage-file usage.yml --compare-to baseline.json --format json --out-file /tmp/infracost-diff-commit.json
infracost comment github --path=/tmp/infracost-diff-commit.json --repo=$SEMAPHORE_GIT_REPO_SLUG --commit=$SEMAPHORE_GIT_SHA --github-token=$GITHUB_API_KEY --behavior=update
Letβs see what the job does:
- The first two commands install Infracost and clone the repository into the CI Machine.
- The third line compares the current costs with the ones stored in
baseline.json
(which should have been already committed to the repository). - The last line compares the changes and posts a comment to GitHub.
If youβre using Bitbucket instead of GitHub, the commands should be:
curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh
checkout
infracost diff --path . --usage-file usage.yml --compare-to baseline.json --format json --out-file /tmp/infracost-diff-commit.json
infracost comment bitbucket --path=/tmp/infracost-diff-commit.json --repo=$SEMAPHORE_GIT_REPO_SLUG --commit=$SEMAPHORE_GIT_SHA --bitbucket-token=$BITBUCKET_API_KEY --behavior=update
Remember to enable the secret you created earlier to ensure that the job has access to your API Keys.
β

β
Conditional execution
Our infracost job does not need to run on every commit. Only when a Terraform file changes, which we can detect with change-based conditions.
To turn on conditional execution, open the Skip/Run conditions section of the block and type: change_in('/**/*.tf') or change_in('/**/*.tfvars')
so the job does not run unless a file ending with a tf
or tfvars
extension changes in the codebase.
β

β
β οΈ If your project’s main branch is not called master
, you need to provide additional options. For example, if the main trunk is called main
use: change_in('/**/*.tf',{default_branch: 'main'}) or change_in('/**/*.tfvars',{default_branch: 'main'})
.
Commenting on pull requests
Instead of comparing the costs against a baseline, we can compare changes across branches on pull requests. This will give the reviewer a summary of the new costs.
β

β
To add comments on pull requests, weβll create a new block with the following commands:
curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh
checkout
git checkout master
infracost breakdown --path . --format json --out-file /tmp/infracost-master.json
git checkout FETCH_HEAD
infracost diff --path . --format json --compare-to /tmp/infracost-master.json --out-file /tmp/infracost-diff-master.json
infracost comment github --path=/tmp/infracost-diff-master.json --repo=$SEMAPHORE_GIT_REPO_SLUG --pull-request=$SEMAPHORE_GIT_PR_NUMBER --github-token=$GITHUB_API_KEY --behavior=update
Like in the previous job, we use infracost comment
to post a comment, but this time we reference a pull request number and compare changes between the trunk and the committed branch.
To finish configuring the block:
- Enable the infracost secret
- Set the run condition to
pull_request =~ '.*'
. As before, we can enable change detection with:pull_request =~ '.*' and (change_in('/**/*.tf') or change_in('/**/*.tfvars'))
.
β

β
Check out Semaphoreβs built-in support for monorepos to learn more about this feature.
Infracost with Monorepos
You will likely have separate Terraform files for each subproject if you work with a monorepo. In this case, you should add an infracost config file at the project’s root. This allows you to specify the project names and where Terraform and usage files are located. You can also set environment variables and other options.
# infracost-config.yml
version: 0.1
projects:
- path: dev
usage_file: dev/infracost-usage.yml
env:
NODE_ENV: dev
- path: prod
usage_file: prod/infracost-usage.yml
env:
AWS_ACCESS_KEY_ID: ${PROD_AWS_ACCESS_KEY_ID}
AWS_SECRET_ACCESS_KEY: ${PROD_AWS_SECRET_ACCESS_KEY}
NODE_ENV: production
When the config file is involved, you must replace the --path
argument with --config-file
in all your commands.
Setting up policies
One more trick the Infracost CLI has has up its sleeve is enforcing policies. Policies are rules that evaluate the output of infracost diff
and stop the CI pipeline if a resource goes over budget. This feature allows managers and team leads to enforce limits. When the policy fails, the CI/CD pipeline stops with an error, preventing the infrastructure from being provisioned.
β

β
Infracost implements policies using Open Policy Agent (OPA), which uses the Rego language to encode policy rules.
Rego has a ton of features, and itβs worth digging in to learn it thoroughly, but for our purposes, we only need to learn a few keywords:
deny[out]
defines a new policy rule that fails if theout
object hasfailed: true
msg
: defines the error message shown when the policy fails.out
: defines the logic that makes the policy pass or fails.input
: references the contents of the JSON object generated withinfracost diff
.
The following example shows a policy that fails when the total budget exceeds $1,000:
# policy.rego
package infracost
deny[out] {
# define a variable
maxMonthlyCost = 1000.0
msg := sprintf(
"Total monthly cost must be less than $%.2f (actual diff is $%.2f)",
[maxMonthlyCost, to_number(input.totalMonthlyCost)],
)
out := {
"msg": msg,
"failed": to_number(input.totalMonthlyCost) >= maxMonthlyCost
}
}
This is another example that fails if the cost difference is equal to or greater than $500.
package infracost
deny[out] {
# maxDiff defines the threshold that you require the cost estimate to be below
maxDiff = 500.0
msg := sprintf(
"Total monthly cost diff must be less than $%.2f (actual diff is $%.2f)",
[maxDiff, to_number(input.diffTotalMonthlyCost)],
)
out := {
"msg": msg,
"failed": to_number(input.diffTotalMonthlyCost) >= maxDiff
}
}
You can experiment and try several examples online on the OPA playground. To enforce a policy, you must add the --policy-path
option in any of the infracost comment
commands like this:
curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh
checkout
infracost diff --path . --usage-file usage.yml --compare-to baseline.json --format json --out-file /tmp/infracost-diff-commit.json
infracost comment github --path=/tmp/infracost-diff-commit.json --repo=$SEMAPHORE_GIT_REPO_SLUG --commit=$SEMAPHORE_GIT_SHA --github-token=$GITHUB_API_KEY --policy-path policy.rego --behavior=update
Conclusion
The power to spin up resources instantly is a double-edged knife: a typo in a Terraform file can be a costly mistake. If youβre already automating deployment and managing services with Terraform, you may as well add Infracost to the mix to help you make more informed decisions and avoid surprises. Setting this up takes only a few minutes and can save thousands of dollars down the road.
Other posts that might be relevant for you:
- A Guide to CI/CD Costs Optimization
- A Developer’s Guide to Terraform
- Top 10 Must-Have Tools for Kubernetes Engineers
Thanks for reading!
β