On Jun 14, 2021 we saw Migrate AWS RDS to GCP Cloud SQL in Five Simple Steps in our Evonence GCP’s Blog Series. As a continuation, this blog will explore how AWS S3 contents can be easily transferred to GCP’s Cloud Storage using Storage Transfer Service for Cloud, a Free Service except the charges associated with network usage(AWS Network Egress) and operations performed by the transfer service at GCP Cloud Storage’s Class A PUT operation charge and AWS S3 Bucket’s storage cost.

Prerequisite

Existing AWS S3 Bucket with some data and User with Programmatic access to AmazonS3FullAcess

Make a note of Access Key ID and Secret Access Key.

Lastly, create a S3 and have some objects in it

For the demonstration purpose three video files each with ~400MB size are uploaded to AWS S3 bucket.

The game starts now...

Make sure you have a GCP Project with Billing enabled and Cloud Storage API enabled.

1. Create a Cloud Storage Bucket.

2. Now let’s create a Transfer Job using Storage Transfer Service wizard with source as AWS S3 bucket with the Access Key ID and Secret Access Key which was generated in the prerequisite step.

3. Define Cloud Storage Bucket as the destination

4. Provide an apt description for the Job

5. Select the preferred Scheduling option

Finally, 1.2GB of data got transferred is 5 min 55 sec with an average speed of 3.67 MB/s

Benefits

1. Easy-to-use and Serverless Migration tool

2. Built-in scheduler hence not need of any additional scheduler solutions

GCP Facts

1. You can use Cloud Secret Manager to save this sensitive information like AWS Access Key ID & Secret Access Key

2. Alway use Regional/ Standard Region Storage Class for any Data Processing 

3. Storage Transfer Service for cloud currently does not support CLI option. Only GCP Console & REST API options are available

Conclusion

Easily migrate AWS S3 Bucket contents to GCP Cloud Storage Bucket in a simple and straightforward fashion using Storage Transfer Service in a wizard-style. 

Now, we have seen how to migrate AWS RDS & AWS S3. The next target would be to migrate AWS EC2 to Containers using Migrate for Anthos...Stay tuned...