Content Introduction Prerequisites Virtual Host Redirect to HTTPS Conclusion Introduction To secure the data transfer redirecting the HTTP traffic to...
Read More
Introduction
Linux system administrators often need to look at log files for troubleshooting purposes. In fact, this is the first thing any sysadmin would do.
Linux and the applications that run on it can generate all different types of messages, which are recorded in various log files. Linux uses a set of configuration files, directories, programs, commands and daemons to create, store and recycle these log messages. Knowing where the system keeps its log files and how to make use of related commands can therefore help save valuable time during troubleshooting.
In this tutorial, we will have a look at different parts of the Linux logging mechanism.
Disclaimer
The commands in this tutorial were tested in plain vanilla installations of CentOS 6.4, Ubuntu 12 and Debian 7.AWS SNS stands for Simple Notification Service. SNS is easy to send messages to the endpoints. Using Amazon SNS, the user can send messages or Emails to distributed systems and services and mobile devices.
SNS notifications can also trigger a Lambda function. When any message is published to an SNS topic with a Lambda function, the Lambda function is invoked with the message’s payload as an input parameter and then sends the message to SNS topics.
Here, we walk through the code in which we use the Amazon SNS service to send an email notification, and the Lambda function will send a notification if the snapshot creation succeeds or fail.
Know How to Send AWS SNS Notification fom Lambda
Prerequisites
- AWS Account
- IAM user with appropriate permission to access Lambda function and SNS Service
- SNS Topic
- SNS Subscription
- Lambda Function
Create an IAM Role
Log in to the AWS console with a root account or with an IAM user who has permission to create Roles.
- Click on Services and select IAM services.
- IAM services page will open from the left panel click on Create Role.
- Create Role page will open select lambda service(Because we need permission to access lambda function).
- Selecting the lambda function will invite you to attach permission and policies to the role. But we will create our customer policy.
Give permission to our function, able to perform the following:
- AmazonEC2ReadOnlyAccess
- Create and Delete Snapshots
- Create and access Cloudwatch Event.
- AmazonSNSFullAccess
- AWSLambdaBasicExecutionRole
For the above policies, you can also use JSON.
Tagging
Lambda function uses EC2 instance tags to identify for which instance the snapshot will create.
- From EC2 Services, select the EC2 instance for which we want to create a snapshot.
- From the Tags, the section adds the below tag.
- The name of the tag is “auto_snapshot”, value for the tag is “true.”
- Save the changes by clicking the Save button.
How to Use Amazon SNS?
Create a topic
- From the AWS Console, switch to the services under the application Integration services select Simple Notification Service(SNS)
- From the left panel, click on Topics, or you don’t have any yet, create one directly from the home page.
Click on Create topic to create a new Topic.
- Enter a topic name. For now, leave everything as default and click on create topic.
- Enter Topic ARN, Protocol, and Endpoint (Email address) on which you want to notification.
- Subscriber will get a Subscription confirmation mail. Click on the Confirm subscription link.
Create a Lambda Function
- Navigate to the AWS services,Lambda Management Console.
- Create a function and named it “create-ebs-snapshot.”
- Choose Python 3.7 as a runtime.
- Under Permissions, change the Execution role to “Use an existing role” and select the role you created earlier.
- Create a Lambda Function for creating snapshots.
https://github.com/ztck0064/LambdaFunction/blob/main/Create-ebs-snapshot.py
Sending a message to a topic
Create a Lambda function to create a snapshot and add the SNS code block to your code snapshot Lambda function.
The sample code to create a snapshot of EC2 Instances and send notification mail for the successful or failed snapshot is as below:
import boto3
import collections
import datetime, json, sys, logging
from pprint import pprint
h = logging.StreamHandler(sys.stdout)
FORMAT = ' [%(levelname)s]/%(asctime)s/%(name)s - %(message)s'
h.setFormatter(logging.Formatter(FORMAT))
logger.addHandler(h)
logger.setLevel(logging.INFO)
ec = boto3.client('ec2')
instance_list = []
def lambda_handler(event, context):
try:
reservations = ec.describe_instances(
Filters=[
{'Name': 'tag-key', 'Values': ['automatic-snapshot', 'Backup']},]
).get(
'Reservations', []
)
instances = sum(
[
[i for i in r['Instances']]
for r in reservations
], [])
print("Found {0} instances that need backing
up".format(len(instances)))
to_tag = collections.defaultdict(list)
for instance in instances:
pprint(instance)
try:
retention_days = [
int(t.get('Value')) for t in instance['Tags']
if t['Key'] == 'Retention'][0]
except IndexError:
retention_days = 7
for dev in instance['BlockDeviceMappings']:
if dev.get('Ebs', None) is None:
continue
vol_id = dev['Ebs']['VolumeId']
print("Found EBS volume {0} on instance {1}".format(
vol_id, instance['InstanceId']))
snap = ec.create_snapshot(
VolumeId=vol_id,
)
to_tag[retention_days].append(snap['SnapshotId'])
for tags in instance['Tags']:
if tags["Key"] == 'Name':
instancename = tags["Value"]
instance_list.append(instancename)
print("Retaining snapshot {0} of volume {1} from instance {2} for {3} days for {4}".format(
snap['SnapshotId'],
vol_id,
instance['InstanceId'],
retention_days,
instancename ))
delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)
delete_fmt = delete_date.strftime('%Y-%m-%d')
print("Will delete {0} snapshots on {1}".format(len(to_tag[retention_days]), delete_fmt))
print("instance id now ")
ec.create_tags( Resources=[snap['SnapshotId']],Tags=[
{'Key': 'automatic-ebs-snapshot-delete-on', 'Value': delete_fmt},
{'Key': 'Name', 'Value': instancename},
{'Key': 'Instance ID', 'Value': instance['InstanceId']}
])
sns = boto3.client('sns')
try:
response = sns.publish(
TopicArn = 'arn:aws:sns:us-east-2:510548384854:automatic-snapshot-status',
Message ="Hello Team,"+"\n\n"+ "The total number of instances found to perform the backup are {0} instances for Backup".format(len(instances)) + "\n\n" + "The list of instances includes {0}".format(instance_list)+ "\n\n" + "Thanks and best regards"+ "\n" + "PowerX",
Subject = 'EC2 Backup Status'
)
logger.info('SUCCESS: Pushed AMI Baker Results to SNS Topic')
return "Successfully pushed to Notification to SNS Topic"
except KeyError as e:
logger.error('ERROR: Unable to push to SNS Topic: Check [1] SNS Topic ARN is invalid, [2] IAM Role Permissions{0}'.format( str(e) ) )
logger.error('ERROR: {0}'.format( str(e) ) )
except:
sns = boto3.client('sns')
response = sns.publish(
TopicArn = 'arn:aws:sns:us-east-2:510548384854:automatic-snapshot-status',
Message ="Hello Team,"+"\n\n" + "Error occurred: failed to create Snapshot" + "\n\n" + "Thanks and best regards"+ "\n" + "PowerX",
Subject = 'EC2 Backup Status'
)
Import boto3 client to use the sns publish and send the notification mail to the subscriber.
TopicArn
The topic you want to publish to.
If you don’t specify a value for the TopicArn parameter, you must specify a value for the TargetArn parameters.
Type: String
Required: No
Replace the TopicARN from the code with your TopicARN, which you created earlier.
Message
Enter the message body, which is required to send to the subscriber.
The total size limit for a single SMS Publish action is 1,600 characters.
Type: String
Required: Yes
Enter a topic name. For now, leave everything as default and click on create topic.
Subject
The subject for mail notification.
Type: String
Required: Yes
Configure CloudWatch Rule to trigger the Lambda Function
- From the AWS Services Navigate to CloudWatch Management Console.
- From the left panel, under the Events, select the Rules tab.
- Click Create Rule.
The above settings create an EC2 snapshot between every 12 hours (timezone GMT) interval.
You can also use the cron expression according to your requirement.
Targets are used to select the Lambda function for which the rule is created.
Conclusion
In this article, we learn about AWS Simple Notification Service (SNS), create the topic, and subscribe topic to send SNS notifications for Backup status using Lambda function python script.
Download code from git hub: https://github.com/ztck0064/LambdaFunction/blob/main/Create-ebs-snapshot.py
Related Posts
How to Build and Deploy Angular Code to S3 Bucket using GitHub Actions
Content Introduction Requirement Getting Started Conclusion Introduction Angular is an open-source web application framework. It is a TypeScript-based free and development...
Read More
by Zehntech