AWS for Beginners – How DMS Helps in Migrating Databases to AWS Cloud – Part 98

What is AWS Database Migration Service?

AWS Database Migration Service (AWS DMS) is a cloud service that makes it possible to migrate relational databases, data warehouses, NoSQL databases, and other types of data stores. You can use AWS DMS to migrate your data into the AWS Cloud or between combinations of cloud and on-premises setups. With AWS DMS, you can discover your source data stores, convert your source schemas, and migrate your data.

Migration Process

Plan → Assess → Convert → Migrate

At a high level, when using AWS DMS, you do the following:

1. Create a replication server
2. Create source and target endpoints that have connection information about your data stores
3. Create one or more migration tasks to migrate data between the source and target data stores

A task can consist of three major phases:

1. The full load of existing data
2. The application of cached changes
3. Ongoing replication

How DMS Works?

AWS Database Migration Service (AWS DMS) is a managed migration and replication service that helps move your database and analytics workloads to AWS quickly, securely, and with minimal downtime and zero data loss. AWS DMS supports migration between 20-plus database and analytics engines.

The only requirement to use AWS DMS is that one of your endpoints must be on an AWS service. You can’t use AWS DMS to migrate from an on-premises database to another on-premises database.

There are two types of Database Migration topologies

Homogeneous: The source and destination databases are same type. Example: Oracle to Oracle, Postgres to Postgres.

Heterogeneous: The source and destination databases are different type. Example: Oracle to Postgres, MySql to Postgres.

Prerequisite

1. VPC and subnets for EC2 Replication instance
2. Security Group to allow access to Replication EC2 Instance. There is no specific port requirement
3. Create the source Postgres database instance and store the credentials in the secret manager
4. Create a IAM role that grants Amazon DMS permissions to access the specified secret (and any required KMS encryption key)
5. Create the target Postgres database instance and store the credentials in the secret manager
6. S3 bucket and folder to store pre migration assessment report
7. IAM role to access S3 bucket

In this demo we are migrating from Postgres to Postgres. There is no requirement for SCT (Schema conversations tool).

For Heterogeneous migration, AWS SCT or DMS Schema Conversion to convert your source database schemas.

Steps to Migrate data from Postgres to Postgres

Step 1: Create a replication instance using the AWS DMS console

A replication instance performs the actual data migration between source and target endpoints. Your instance needs enough storage and processing power to perform the tasks that migrate data from your source database to your target database. How large this replication instance should be depends on the amount of data to migrate and the tasks your instance needs to do.

To create a replication instance using the console

• Login to AWS Account, Navigate to AWS DMS console
• On the navigation panel choose replication instance, click on create replication instance
• Enter the name and description

• Enter the instance class, engine version and Multi AZ requirement

• Click on advanced security and network configuration, Allocate the storage, provide the VPC and subnet details

• Leave all other settings as default and click on create

Step 2: Specify source and target endpoints

To specify a source endpoint and database endpoint using the AWS DMS console

• Navigate to endpoint in DMS, choose create endpoint
• Choose endpoint type (source or destination), if it’s a RDS DB instance then tick the checkbox of RDS DB instance
• Choose the database from the dropdown
• Provide endpoint identifier (name)

• Choose the Target engine or Source engine , choose AWS secret manager as Access to database endpoint , provide the database name

• Test endpoint connection tab, provide the VPC and replication instance to test the connectivity to database

Both source and destination endpoints are created

Step 3: Create a task and migrate data

To create a migration task and start your database migration:

• Navigate to Database migration task in DMS, click on create task
• Provide a task identifier, choose the replication instance created earlier

• Choose the source database type and target database type
• Choose the migration type from drop down (3 types)
• Choose do nothing in target table preparation mode

• Tick the checkbox of Turn on validation and turn on CloudWatch logs. Keep rest all default

• Click on table mapping tab, Add a selection rule to include or exclude your schemas

• In pre migration assessment tab, enable pre migration assessment run
• Fill the bucket name, bucket folder and IAM role to use


• Click on migration task startup tab, choose manual later. Click on create


Migration successful details



Step 4: Test replication

To test replication

1. While your migration task is running, you can monitor the progress of your database migration as it happens:
   • In the DMS console navigation pane, choose Database migration tasks
   • Choose Task Name created before
   • Choose Table statistics

2. After replication completes, query the target database for the records. Verify

Conclusion

AWS Database Migration Service is a handy tool for Migrating Databases from On-premises to AWS Cloud, Other Cloud to AWS Cloud and AWS Cloud to AWS Cloud. The migration method can be homogeneous or heterogeneous. It can help you for Full load, Full load and On-going replication or On-going replication type migrations. For heterogeneous migrations you can use SCT(Schema conversion tool) or DMS Schema Conversion for source server schema changes.

Follow our Twitter and Facebook feeds for new releases, updates, insightful posts and more.

Rate this post