S3 & Snow

S3 Cost Savings

Use S3 Select to save on Network and CPU cost

Set LifeCycle roles to move / delete objects - compress object if possible

Enable Requester Pays (pay only for storage) - add the IAM ARN of the client to the policy

Storage Classes - S3

Standard (3 copies) - Frequent data , low latency (BigData, Applications, Content distribution

Infrequent Access (pay on retrieval) - DR - Backup - 30+ days

One Zone - 1 copy (Secondary copy )

Storage Classes - Glacier

Low cost Object storage - pay per storage + retrieval

Storage Lens

Optimize storage across Organization (apply best practice , find costs) get report in the ORG level

• Free = 28 metrics for 14 days

• Paid = more metrics, CW data, prefix aggregation . for 15 months

Storage Class Analysis

Recommendations to move objects to the right storage class( not for One Zone IA or Glacier)

a daily report that we can view on QuickSight

Performance

Latency: 100 - 200 ms

At least 3500 put / copy / delete - 5500 get : per prefix per second per bucket - no limit on prefix

Upload:

can use Multi Part upload (recommended 100mb, must use from 5gb and up) to parallelize uploads. use Transfer acceleration (upload only) to use Edge locations to upload data into S3 - can combine.

can remove in-completed parts using Lifecycle policy

Download:

use Byte-Range Fetches to parallelize GET requests - but only for parts of data (header of a file)

use S3- Select to query server side SQL filtering (rows and columns)

Event Notification

can trigger SNS/SQS/Lambda for events (replication. created, removed, restored)

can trigger Event Bridge for other services (using JSON rules) , can set multiple destinations

S3 Replication - need to enable versioning

CRR- Cross Region Replication

SRR - Same Region Replication

• does not affect storage class - need to configure “Lifecycle rule”

• can set S3 RTC (replication time control) to speed replication - will be available within 15 min

S3 Access Points

Each point has its own DNS and policy and limit access - need to create a VPC gateway endpoint on the VPC side and link it to the VPC access point - can secure for a specific (VPC/subnet)

Can create a Multi-Region Access Point that provides a global endpoint to multiple buckets in multiple regions - will dynamically route traffic including failover

S3 Object Lambda

Associate a GET request to a Lambda function - configure S3 Object Lambda access point

(Good for file conversion or PII detection )

Security

User based security - IAM policy

Resource based policy (bucket Policy)- for cross account access , to force encryption, set conditions

Object ACL - not recommended - use policy

Can block object version deletion

Can set conditions on policies SourceIp | VpcSourceIp | CloudFront Origin

in Glacier - can lock policy for future edits (compliance) WORM

Encryption Keys:

SSE - S3 - managed by AWS

SSE-KMS - get cloud trail audit - good for public buckets ,

SSE-C - costumer managed keys

Client side - client management keys

Glacier = all data is encrypted by AWS

• can set HTTPS endpoint - recommended, can enforce by bucket policy aws:SecureTransport

Events

S3 Access Logs - can be sent to another bucket

Event Bridge - need to enable cloud trail at the object - can trigger Lambda etc’

VPC Gateway Endpoint for S3

To direct connect your VPC to S3 within AWS network

Pre-Signed URL’s

Can generate URL’s for premium download's (valid for 3600 sec - can change)

Object lock / vault lock

WORM (block delete / edit ) for objects = compliance