Content CloudWatch Logs What is an accomplishment by using this automation? Requirements How does Automation work? How to Create an...
Read More
CloudWatch
Amazon CloudWatch allows you to monitor their AWS applications in the cloud, in real-time. CloudWatch is configured to monitors, store on request counts, latency, CPU usage and, provide metrics and log files. The data and reports CloudWatch provides that you can use to analyze and keep track of operational issues, resource use, and monitor system performance.
Users also can send their custom metrics and logs to the CloudWatch for monitoring. We can create custom dashboards and provides a real-time presentation of operational data in the form of a metric like CPU utilization, disk storage, etc.CloudWatch provides an Alarm facility, by which you can include rules, triggers high-resolution alarms, and send notifications.
Metrics
Metrics represents a set of data points and is the fundamental concept in CloudWatch. A name, dimensions, namespace uniquely define metrics. Each data point has a time-stamp.
Alarm
Create an Alarm for triggers, and send notification alerts. It can watch metrics over the specified time and perform an action. Alarms can be configured against any upper or lower threshold and will trigger whenever the aggregated and selected metric, across that threshold for a set amount of time.
CloudWatch Agent
You can use the CloudWatch agent to collect both system metrics and log files from Amazon EC2 Instances and on-premises servers.
In this article, we are using EC2 Instance and learn to collect metrics and including sub-resource metrics such as Memory Utilization, metric per-CPU core, disk utilization, etc.
For collecting the Memory Utilization metric, disk Utilization metric, we need to install the CloudWatch Agent on EC2 Instance.
Install and configure CloudWatch Agent on EC2 Instance Ubuntu
We are creating an example of collecting Memory, CPU, disk Metrics and sending Apache logs to CloudWatch Logs.
Create an IAM Role
Using IAM, Attach CloudWatchAgentServerPolicy permission to the IAM role of the EC2 Instance.
Configure EC2 Instance
Now assign the IAM role you just created to the EC2. This allows it to communicate with CloudWatch.
Now ssh to your EC2 Instance running Ubuntu and use the following commands.
Download the CloudWatch Unified Agent. Find the right agent link for your OS. Use the below command to install CloudWatch Agent on the Ubuntu server.
$ wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
Install the dpkg package. If you downloaded an RPM package on a Linux server, change to the directory containing the package.
$ sudo dpkg -i amazon-cloudwatch-agent.deb
For collecting metrics through collectd; you must install collectd.
$ sudo apt-get install collectd
You’ll need the JSON config file in your directory. Use the below command to configure JSON file for sending logs and collect metrics.
You can configure the CloudWatch agent by customizing the JSON config file.
$ sudo -e /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json
{
"agent": {
"metrics_collection_interval": 60,
"logfile": "/var/log/amazon-cloudwatch-agent.log",
"debug": false
},
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/apache2/access.log",
"log_group_name": "apache-access",
"log_stream_name": "{instance_id}"
},
{
"file_path": "/var/log/apache2/error.log",
"log_group_name": "apache-error",
"log_stream_name": "{instance_id}"
}
]
}
}
},
"metrics": {
"append_dimensions": {
"AutoScalingGroupName": "${aws:AutoScalingGroupName}",
"InstanceId": "${aws:InstanceId}"
},
"aggregation_dimensions": [["AutoScalingGroupName"]],
"metrics_collected": {
"collectd": {
"metrics_aggregation_interval": 60
},
"statsd": {
"metrics_aggregation_interval": 60,
"metrics_collection_interval": 10,
"service_address": ":8125"
},
"cpu": {
"measurement": [
"cpu_usage_idle",
"cpu_usage_iowait",
"cpu_usage_user",
"cpu_usage_system"
],
"totalcpu": true
},
"disk": {
"measurement": [
"used_percent",
"inodes_free"
],
"resources": [
"*"
]
},
"diskio": {
"measurement": [
"io_time",
"write_bytes",
"read_bytes",
"writes",
"reads"
],
"resources": [
"*"
]
},
"mem": {
"measurement": [
"mem_used_percent"
]
},
"netstat": {
"measurement": [
"tcp_established",
"tcp_time_wait"
]
},
"swap": {
"measurement": [
"swap_used_percent"
]
}
}
}
}
or
If you want to configure the Cloudwatch agent with the help of a setup wizard, run the below command:
$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
Explanation of Code
“metric_collection_interval”: Default metrics collection interval is 60 seconds. You can change according to requirements.
“logfile”: is the path of the log file where the CloudWatch Agent writes its logs.
“debug”: True or False, whether CloudWatch Agent runs in debug mode.
“file_path”: File where logs are written that should be forwarded to CloudWatch Logs. Here we are using the file as access.log file, which is then forwarded to CloudWatch logs.
“log_group_name”: Name of Log group to save logs to.
“log_stream_name”: Log stream to save logs to.
“append_dimensions”: Dimensions to add to all metrics.
“aggregation_dimensions”: Dimensions on which metrics are to be aggregated
“collectd”: Through which collect the metrics.
“statsd”: CloudAgent will act as a statsd server to allow the collection of additional metrics.
“totalcpu”: true, Set to `false` to show each CPU, `true` to aggregate all CPUs.
To start the CloudWatch agent, run the following:
$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s
Note: this will convert the config file JSON into a TOML file used by the CloudWatch Agent. The original JSON file will be deleted.
The CloudWatch agent will start automatically after a reboot because it is integrated with systemd and, and you can restart it like so:
$ sudo systemctl restart amazon-cloudwatch-agent
Setup CloudWatch Alarm
We are creating an alarm to notify when the Memory Utilization metric of the Instance is above 80%.
From the Amazon console, under the Management & Governance Services, select CloudWatch.
From the left panel, select Alarms, and click on Create alarm for creating a new alarm.
Click on the Select metric button for selecting the CloudWatch metric for which you want to configure the alarm.
Click on CWAgent. This is the metric send by the Unified CloudWatch agent.
Click on the InstanceId for selecting the memory metric.
Enter the name of the metric in the search box and select the Instance for which you want to configure the high memory utilization alert. Click on a Select metric.
On the Specific metric and conditions page, edit the condition of the threshold, For example: If memory utilization of Instance is greater than 80 percent.
Click on Next.
For sending the notification, select Alarm state trigger, select an SNS topic, and enter the email address of the subscriber to which you want to send the notification. Then Click on Next.
Enter the Alarm name and Alarm description(optional) and click on Next.
On the Preview page, Click on Create alarm.
From the Alarm page, you can see the alarm created successfully.
Create a Custom Dashboard
From the Amazon Console, under the CloudWatch service, from the left panel, select the Dashboards. Click Create dashboard.
Enter the name of the Dashboard and click on the Create dashboard button.
Select a type of widget (a type of graph or text). Click on Next.
Select the source from which you will like to create the widget. Click on Configure.
From the All metrics option, select CWAgent–>>InstanceId. In the search box, search for mem_used_percent and select all Instances. Click on the Create widget.
Your custom Dashboard with a memory utilization graph is ready.
Conclusion
You can collect different metrics, logs with the help of CloudWatch Agent. From the Amazon CloudWatch, you can create custom CloudWatch Metrics, create custom dashboards that will help you to analyze and keep track of operational issues, resource use, monitor system performance, etc.
Related Posts
Docker Containers V/s Virtual Machines
Contents Introduction Docker Containers Virtual Machines Differences between Container and Virtual Machine Comparison – Which one is better? Conclusion Introduction...
Read More
by Zehntech