How to Monitor Disk Usage with CloudWatch Metrics for an EC2 Instance

by Irina Butaru

AWS CloudWatch is helpful if you want to monitor AWS applications in the cloud or on-premises or optimize system resources. But one of CloudWatch’s major downsides is that it doesn’t offer metrics for specific use cases, including memory metrics on EC2 instances, which are essential if you don’t want your server to crash. 

This is where custom metrics can save the day. Here’s how you can monitor the disk usage of an EC2 instance running Ubuntu 20.04.

Install and Configure CloudWatch Agent

Step 1: Configure CloudWatch Agent

  • Connect to EC2 instance as root
  • Download CloudWatch Agent
cd /tmp
wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/[arm64-or-amd64]/latest/amazon-cloudwatch-agent.deb

*Note: You’ll need to choose the arm64 or amd64 version of CloudWatch agent, depending on the configuration of the EC2 instance.

Step 2: Install CloudWatch Agent

dpkg -i -E ./amazon-cloudwatch-agent.deb

Step 3: Create and Edit CloudWatch Agent’s config file

sudo vi /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

Enter the following command written in JSON:

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "cwagent"
  },
  "metrics": {
    "append_dimensions": {
        "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "disk": {
        "measurement": [
          "used_percent"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "/"
        ]
      }
    }
  }
}

*Note: Find out more about CloudWatch available configuration parameters.

Enable Permissions to Write CloudWatch Metrics

You’ll need to create IAM roles and users with the CloudWatch agent, so you can access AWS resources.

  1. To create the IAM Role:
  • After signing in to the ‘AWS Management Console,’ go to the IAM console.
  • Choose ‘Roles’ from the left navigation panel, and select ‘Create role.’
  • Choose’ AWS service’ under ‘Select type of trusted entity.’
  • Choose EC2 under ‘Common use cases.’
  •  Select ‘Next: Permissions.’
  • You’ll then be prompted to the list of policies. Select the check box next to CloudWatchAgentServerPolicy. Use the search box if you don’t find the policy you’re looking for.

Create IAM User with CloudWatch Agent

  • Select the box next to ‘AmazonSSMManagedInstanceCore’. This AWS-managed policy enables an instance to use Systems Manager’s service main functionality. Use the search box if you don’t find the policy.

*Note: selecting this policy is unnecessary if you start and configure the agent using only the command line.

  • Choose ‘Next: Tags.’
  • (Optional) Attach one or more tag-key value pairs to organize, track, or control access for this role, and then choose ‘Next: Review.’
  • Enter a name for your new role (e.g., CloudWatchAgentServerRole or any name you’d prefer).
  • (Optional) Enter a role description.
  • Confirm the ‘CloudWatchAgentServerPolicy’.
  • (Optional) Choose ‘AmazonSSMManagedInstanceCore’ that appears next to Policies.Choose ‘Create role.’

2. To create the IAM User

  • After signing in to the ‘AWS Management Console,’ go to the IAM console. 
  • Choose ‘Users’ from the left navigation panel.  
  • Select ‘Add user’ and enter a name you want to use.
  • For Access type, select ‘Programmatic access’. Choose ‘Next: Permissions.’
  • Choose ‘Attach existing policies directly’ within ‘Set permissions.’
  • Select the check box next to ‘CloudWatchAgentServerPolicy’ from the list of policies. Use the search box if you don’t find the policy you’re looking for.

Run CloudWatch Agent using AWS System Manager

  • Select the box next to ‘AmazonSSMManagedInstanceCore’. This AWS-managed policy enables an instance to use Systems Manager’s service main functionality. Use the search box if you don’t find the policy.

*Note: selecting this policy is unnecessary if you start and configure the agent using only the command line.

  • Choose ‘Next: Tags’.
  • (Optional) Attach one or more tag-key value pairs to organize, track, or control access for this role.
  • Choose ‘Next: Review’.
  • Check and confirm the listed policies are correct. Then, choose ‘Create user’.
  • Choose ‘Show’  from the row for the new user. 
  • Copy the access key and secret key to a file or download them as csv.file, so that you can use them when installing the agent. Click on ‘Close’.

Last Step: Install CloudWatch Agent on EC2 Instances

You have now completed the CloudWatch agent configuration.  You’ll need to attach IAM roles to each Amazon EC2 instance, so you can read and write information to CloudWatch.

  1. Attach IAM role to EC2 instance
  • Open the Amazon EC2 console.
  • Choose ‘Instances’ from the navigation pane.
  • Select the instance, choose ‘Actions, Security, Modify IAM role’.
  • Select the IAM role to attach to your instance (the role created at the previous step). 
  • Choose ‘Save.’

2. Install AWS CLI

apt  install awscli

Specify IAM credentials and AWS Region

sudo aws configure --profile AmazonCloudWatchAgent

Enter required credentials for IAM User created at step 1:

AWS Access Key ID:

AWS Secret Access Key:

Default region name: 

Default output format:

 Restart the agent and check the agent’s logs:

sudo systemctl restart amazon-cloudwatch-agent
tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

Wait a few minutes, and your new set metric will show up in CloudWatch. It will now send metric data on disk space usage to CloudWatch, so you can increase storage size in time if needed.

by Irina Butaru Senior Software Engineer

Related Posts