FSx & Data sync

Owner
Verification
Tags
Last edited time	@April 12, 2023 7:42 PM

A managed file system on AWS

FSx For Windows

A managed Windows File Server with SMB, NTFS, Active Directory, and ACLs, can be mounted on Linux EC2. It supports DFS to join an on-premises file server, requiring a DX/VPN connection.

Storage options include SSD/HDD, with the ability to set up Multi AZ.

S3 integration allows for both reading and writing to S3 as if it were a file system.

FSx For Luster

A distributed file system for HPC and ML with sub-millisecond latency.

Storage:

SSD (IOPS) / HDD (Throughput) , can read S3 as a file system and write to it

Deployment options:

Scratch (high performance, no backup) - for short term data

Persistent (data is replicated in the same AZ) - long term data

FSx for ONTAP

mixed file system (NASS compatible) storage auto shrink or grow

data de-duplication , point in time cloning

OpenZFS

NFS only , to move ZFS workloads to the cloud

📌

To migrate from Single AZ to Multi AZ, create a new file system. Use Data Sync to retain availability or use AWS Backup (with downtime). To change size, create a new file system with a smaller size and migrate with Data Sync.

✍🏻

Data Lazy Loading enables processing of data without loading it. This helps reduce memory usage and improves efficiency.

Data Sync

to move large data (need agent outside of AWS) in or out of AWS , replication is scheduled. keeps permissions and meta data , can set line limit

can be use with AWS snow family

AWS Transfer Family

FTP, FTPS, and SFTP access to S3 or EFS is available, with multi-AZ support for authentication with AD and OKTA. DNS (Route 53) and IAM roles, as well as third-party solutions, can be used. Endpoints include public DNS (AWS managed) and VPC endpoints for private access with static IPs, and hybrid VPC endpoints with internet access.