RDS Autoscaling using AWS lambda

This post outlines the process of cost optimization in AWS cloud, i have introduced the autoscaling of RDS instance class to optimize the cost of cloud infrastructure, i have observed the metrics and finalised to do scale down during specific time period.

When you are using cloud services to host your application, it is important to take care of your budget in costing of cloud services, cost optimization plays an important role for every business. i am going to implement auto scaling using AWS lambda, AWS Eventbridge.

Solution Overview

In the cost optimization process i have created one lambda function which is using python 3.11 and the boto3 library is used to scale down and scale up the RDS instance class. AWS Eventbridge schedule is created based on cronjob it trigger to AWS lambda function. After the successful lambda execution of lambda function RDS scale down and scale up activity is initiated and the same informed via email using AWS SNS.

Implementation of RDS autoscaling

Prerequisite

I consider you should have knowledge about AWS Eventbridge, AWS lambda, AWS SNS and boto3 library.

Creation and configuration of AWS lambda function

I have created AWS lambda using the below steps, please follow every step carefully.

GO to the AWS lambda function and click on the create button, after that you will get as below, please enter specific name what you need, here i am using python 3.11 because my code is developed and tested on python 3.11.

Here I am using pre-created IAM role , which is used to provide essential permissions to AWS lambda function which is required to do activity on RDS instance, AWS SNS and AWS cloud watch. After that click on create function which will create a AWS lambda function.

I have created the code below and uploaded it to the AWS lambda function.

import boto3
import os
from botocore.config import Config
import logging

# Configure logger for logging to both CloudWatch Logs and terminal

logger = logging.getLogger()
logger.setLevel(logging.INFO)


# Create a handler for logging to terminal
# terminal_handler = logging.StreamHandler()
# terminal_handler.setLevel(logging.INFO)
# formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
# terminal_handler.setFormatter(formatter)
# logger.addHandler(terminal_handler)


# AWS SDK configuration
my_config = Config(
    region_name = os.environ['aws_region_name']
)


# Initialize RDS client
rds_client = boto3.client('rds', config=my_config)


# Main handler function
def lambda_handler(event,context):

    # Get environment variables for scaling operations
    down_instance_class = os.environ['down_instance_class']
    instance_identifier = os.environ['instance_identifier']
    up_instance_class   = os.environ['up_instance_class']
    
    # Log the event received
    logger.info(f"Event fired: {event}")

    # Perform scaling based on event details
    if (event["resource"] == "rds" and event["activity"] == "scale down"):
        scaleDownDBInstance(instance_identifier,down_instance_class)
    if (event["resource"] == "rds" and event["activity"] == "scale up"):
        scaleUpDBInstance(instance_identifier,up_instance_class)
        
    



def scaleDownDBInstance(DBInstanceIdentifier,DBInstanceClass):
    
    try:
        # Modify the RDS instance to scale down
        response = rds_client.modify_db_instance(
            DBInstanceIdentifier=DBInstanceIdentifier,
            DBInstanceClass=DBInstanceClass,
            ApplyImmediately=True
            )
        logger.info(f"Scaling down RDS instance {DBInstanceIdentifier}...")
        logger.info("Response: %s", response)
    except Exception as e:
        logger.error("Error scaling down RDS instance: %s", str(e))
        
        
def scaleUpDBInstance(DBInstanceIdentifier,DBInstanceClass):
    
    try:
        # Modify the RDS instance to scale up
        response = rds_client.modify_db_instance(
            DBInstanceIdentifier=DBInstanceIdentifier,
            DBInstanceClass=DBInstanceClass,
            ApplyImmediately=True
            )
        logger.info(f"Scaling up RDS instance {DBInstanceIdentifier}...")
        logger.info("Response: %s", response)
    except Exception as e:
        logger.error("Error scaling up RDS instance: %s", str(e))

The above code in AWS lambda function requires environment variables to execute well, so i have configured environment variables as below.

Creation and configuration of AWS Eventbridge schedule

I have created a schedule in AWS Eventbridge as a proposal which sends payload. It also helps lambda function to decide whether the trigger is related to scale up or scale down.I have created four schedules as below.

RDS_scaleDown_Schedule

This schedule will trigger the lambda function at 02:15 AM on every saturday and it will send json payload as { “resource”: “rds”, “activity”: “scale down” }.

RDS_scaleUp_Schedule

This schedule will trigger the lambda function at 09:00 AM on every saturday and it will send json payload as { “resource”: “rds”, “activity”: “scale up” }.

RDS_scaleDown_Sunday

This schedule will trigger the lambda function at 02:15 AM on every sunday and it will send json payload as { “resource”: “rds”, “activity”: “scale down” }.

RDS_scaleUp_Sunday

This schedule will trigger the lambda function at 11:00 PM on every sunday and it will send json payload as { “resource”: “rds”, “activity”: “scale up” }.

Creation and configuration of AWS SNS

AWS SNS is required to send email notification about the status of AWS lambda function execution to scale up and scale down of RDS instance class. Whenever the AWS lambda function is failed or success it sends notification via email.

Creation of SNS Topic

SNS topic is required to send email to the user which has subscribed the email to a related topic, it is basically a communication channel which gets notification from source AWS services and sends via different medium here we are using email.

Enter the name of the SNS topic and Display name(Subject of message), make it default and create.

Creation of SNS Subscription

After the creation of SNS topic, go to subscription and click on create subscription, it will open a form where you have to select the Topic to which you have to subscribe, select the protocol email and your email id where you want to get the notification.

After the creation of subscription, you will get an email to confirm your subscription, click on Confirm Subscription then you will get notification via email.

You can save your infrastructure cost upto 40% or want to optimize the productivity of development team. Contact me via email id firsttalk26@gmail.com