Copy a large number of files from a server to AWS S3

TL/DR: Use s3-parallel-put, a Python script to copy to S3 — I recently worked on an EC2 instance which needed a large number of files, 200gb+ to be copied over to AWS S3. The overall goal was to move all data over to s3 to facilitate future growth and allow for multiple instances in different …

Continue reading ‘Copy a large number of files from a server to AWS S3’ »