Replicating Encrypted S3 Objects Across AWS Accounts – VMware Cloud

Having multiple AWS accounts within an organization is a common strategy. Numerous factors play a crucial role in deciding the appropriate number of AWS accounts required for an organization, such as resource isolation, security isolation, cost allocation, billing, business unit separation, audit, compliance, etc. However, managing multiple AWS accounts also comes with some additional complexity. 

Usually, data stored in S3 is replicated primarily for reliability, performance, and compliance reasons. How can we replicate objects to a bucket owned by a different AWS account? What if the objects in the source bucket are encrypted? 

This article discusses a method to configure replication for S3 objects from a bucket in one AWS account to a bucket in another AWS account, using server-side encryption using Key Management Service (KMS) and provides policy/terraform snippets. 

Setup Requirements 

Two AWS accounts: We need two AWS accounts with their account IDs.  

Source and destination buckets: We need an S3 bucket in the source account where the objects are created/uploaded and an S3 bucket in the destination account to store the replicated objects.  

Source and destination KMS keys: We need KMS keys created in both source and destination accounts.  

Some of the requirements for configuring replication are: 

• Both source and destination buckets must have versioning enabled. 
• S3 service must be allowed permissions to replicate objects from the source bucket to the destination bucket on your behalf. 

Let’s refer to the source AWS account as Alice’s account and the destination AWS account as Bob’s account. 

Configuration in Alice’s AWS account 

1. Create AssumeRole and allow S3 service 

{ 
   "Version": "2012-10-17", 
   "Statement": [ 
     { 
       "Action": "sts:AssumeRole", 
       "Principal": { 
         "Service": "s3.amazonaws.com" 
       }, 
       "Effect": "Allow", 
       "Sid": "VisualEditor0" 
     } 
   ] 
 } 

2. Create IAM policy allowing KMS keys to encrypt and decrypt

{ 
   "Version": "2012-10-17", 
   "Statement": [ 
     { 
       "Action": [ 
         "s3:ListBucket", 
         "s3:GetReplicationConfiguration", 
         "s3:GetObjectVersionForReplication", 
         "s3:GetObjectVersionAcl", 
         "s3:GetObjectVersionTagging", 
         "s3:GetObjectVersion", 
         "s3:ObjectOwnerOverrideToBucketOwner" 
       ], 
       "Effect": "Allow", 
       "Resource": [ 
         "<Alice-S3-Bucket-ARN>", 
         "<Alice-S3-Bucket-ARN>/*" 
       ] 
     }, 
     { 
       "Action": [ 
         "s3:ReplicateObject", 
         "s3:ReplicateDelete", 
         "s3:ReplicateTags", 
         "s3:GetObjectVersionTagging", 
         "s3:ObjectOwnerOverrideToBucketOwner" 
       ], 
       "Effect": "Allow", 
       "Resource": "<Bob-S3-Bucket-ARN>/*" 
     }, 
     { 
       "Action": [ 
         "kms:Decrypt" 
       ], 
       "Effect": "Allow", 
       "Resource": "<Alice-KMS-Key-ARN>" 
     }, 
     { 
       "Action": [ 
         "kms:Encrypt" 
       ], 
       "Effect": "Allow", 
       "Resource": "<Bob-KMS-Key-ARN>" 
     } 
   ] 
 } 

3. Set up replication configuration on S3 bucket and add replication rule. You can configure this using AWS console UI, but for simplicity, below is the terraform snippet

...
...
   versioning { 
     enabled = true 
   } 
 server_side_encryption_configuration { 
     rule { 
       apply_server_side_encryption_by_default { 
         kms_master_key_id = "<Alice-KMS-Key-ARN>" 
         sse_algorithm = "aws:kms" 
       } 
     } 
   } 
 replication_configuration { 
     role = "<Alice-IAM-Role-ARN>" 
     rules { 
       id = "Replicate_to_Bob_bucket" 
       status = "Enabled" 
       source_selection_criteria { 
         sse_kms_encrypted_objects { 
           enabled = "true" 
         } 
       } 
       destination { 
         bucket = "<Bob-S3-Bucket-ARN>" 
         replica_kms_key_id = "<Bob-KMS-Key-ARN>" 
         access_control_translation { 
           owner = "Destination" 
         } 
         account_id = "<Bob-AWS-AccountID>" 
       } 
     } 
   } 
...
...

Configuration in Bob’s AWS account 

1. Configure KMS key policy to allow S3 service to encrypt data in Bob’s bucket during replication 

{ 
   "Version": "2012-10-17", 
   "Statement": [ 
     { 
       "Sid": "VisualEditor0", 
       "Effect": "Allow", 
       "Principal": { 
         "AWS": [ 
           "<Alice-IAM-Role-ARN>" 
         ] 
       }, 
       "Action": [ 
         "kms:Encrypt", 
         "kms:ReEncrypt*", 
         "kms:GenerateDataKey*", 
         "kms:DescribeKey" 
       ], 
       "Resource": [ 
         "*" 
       ] 
     }, 
     { 
       "Effect": "Allow", 
       "Principal": { 
         "AWS": [ 
           "arn:aws:iam::<Bob-AWS-AccountID>:root" 
         ] 
       }, 
       "Action": [ 
         "kms:*" 
       ], 
       "Resource": [ 
         "*" 
       ] 
     } 
   ] 
 } 

2. Configure S3 bucket policy to grant Alice permissions to perform replication actions 

{ 
   "Version": "2012-10-17", 
   "Statement": [ 
     { 
       "Sid": "VisualEditor0", 
       "Effect": "Allow", 
       "Principal": { 
         "AWS": [ 
           "arn:aws:iam::<Alice-AWS-AccountID>:root" 
         ] 
       }, 
       "Action": [ 
         "s3:GetBucketVersioning", 
         "s3:PutBucketVersioning", 
         "s3:ReplicateObject", 
         "s3:ObjectOwnerOverrideToBucketOwner" 
       ], 
       "Resource": [ 
         "<Bob-S3-Bucket-ARN>", 
         "<Bob-S3-Bucket-ARN>/*" 
       ] 
     } 
   ] 
 } 

Summary 

In the walkthrough above, I have shown how to configure replication to copy objects across AWS accounts. If these topics excite you and you have a passion for building highly scalable, fault-tolerant, reliable SaaS services, join us in building foundational infrastructure components for Cloud Services Engagement Platform. We are hiring at all levels across multiple geographical locations. You can reach out to me directly or look for open roles.