Transfer Data From AWS S3 to GCP Cloud Storage

Introduction

When cloud computing becomes much more popular, cloud migration is important. In general, AWS is more mature than GCP. However, the pricing of AWS is also higher than GCP in most cases. This article we will talk about how to migrate data from AWS S3 to GCP Cloud Storage.

Solution Diagram

How to transfer files from Google Cloud Storage (GCS) into Amazon S3 bucket  without downloading the files - Pro Analyst

Well, in fact this is not really like solution diagram because we just have to configure on S3 and Cloud Storage.

Create Demo S3 Bucket

If you have existing S3 bucket, you can skip this step.

Search for S3.

Create bucket.

Give it a name and then move down to create bucket.

click for the bucket.

Drag something to upload, and you will see it.

Set IAM

Go to IAM.

Click Users and then add user.

Give it a name and allow programmatic access.

Click Next and select attach existing policies directly, and search for AmazonS3ReadOnlyAccess, then click Next.

Click Next until the review page, then create user.

In this page we will see the access key and click show we will see the secret access key. Please copy it for further use.

Create Cloud Storage Bucket

In GCP, click Storage => Browser.

Create Bucket.

Input a name then create.

Select Data Transfer => Transfer Service | cloud.

Click create transfer.

Select S3 bucket as source, then type your S3 bucket name, paste the access key and secret access key that we copy before. And click continue.

Choose the destination bucket we created on GCP, and click continue.

Finally click create.

Wait for few secound, it will show completed which means our job is done.

Back to our bucket in GCP, we can see the file is transferred.

Clean up

Delete the GCP bucket.

Also delete the S3 bucket.

And finally delete the user we created.

Leave a Reply