HPC in AWS
| Owner | |
|---|---|
| Verification | |
| Tags | |
| Last edited time |
Data management and transfer
Direct connect & snowball & Data Sync
Compute
EC2 (CPU/GPU optimized & SPOT and cluster placement group (same rack, same AZ)
Networking
EC2 enhanced networking - lower latency with:
- ENH (elastic network adapter) - up to 100 gbps
- Intel 82599 VF - up to 10 gbps (Legacy)
- EFA - improved (linux only - inter node communication)
Storage
EBS - up to 256k with io2
Instance store - up to millions of iops
S3 , EFS & Lustre (HPC optimized)
Automation
- AWS Batch (multi-node parallel jobs - one task on many EC2)
- AWS Parallel Cluster (opensource) cluster management tool to deploy HPC
automated VPC Subnet and cluster creation