Certopolis

AWS Certified Solutions Architect – Associate

Snowball vs DataSync vs Transfer Family: moving data into AWS

Use AWS DataSync to copy large datasets over the network, once or on a schedule. Use AWS Transfer Family when partners or users must keep sending files over SFTP, FTPS, FTP or AS2. Snowball Edge shipped data on a physical device, but AWS says it is no longer available to new customers and points them to DataSync or AWS Data Transfer Terminal.

AWS DataSync

DataSync copies file and object data to, from and between AWS storage services. On-premises sources include NFS, SMB, HDFS and object storage. AWS targets include Amazon S3, Amazon EFS and the four Amazon FSx file systems. It can also read from other clouds, such as Google Cloud Storage and Azure Blob Storage.

It handles encryption and data integrity validation, and uses a parallel, multi-threaded protocol. It can transfer through VPC endpoints so the data does not cross the public internet. It can write straight into S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive, so cold data needs no second copy step.

Choose DataSync over a home-built copy script when you need scheduling, verification and speed without running the infrastructure yourself.

AWS Transfer Family

Transfer Family is a fully managed file transfer endpoint in front of Amazon S3 or Amazon EFS. It supports SFTP, FTPS, FTP, AS2 and browser-based transfers through web apps.

Partners and users keep their existing clients and configurations, and you run no file transfer servers. Pick it when the requirement is about the protocol others use to send you files, not about a one-time bulk copy.

Snowball Edge and offline transfer

Snowball Edge is a rugged device with storage and compute. AWS ships it to you, you load data, and you ship it back for import into Amazon S3. It comes as Storage Optimized 210 TB or Compute Optimized.

The Snowball Edge guide now says: Snowball Edge is no longer available to new customers. New customers should use AWS DataSync for online transfers, AWS Data Transfer Terminal for secure physical transfers, or AWS Partner solutions. For edge computing, AWS points to AWS Outposts.

AWS Data Transfer Terminal is a physical location where you bring your own storage devices and upload them to AWS over a fast network link.

Where the exam catches people

  • Believing DataSync only reads NFS: it reads NFS, SMB, HDFS, object storage and several other clouds' storage.
  • Believing you must land data in S3 Standard and then add a lifecycle rule: DataSync can write directly to a Glacier storage class.
  • Believing Transfer Family is a bulk migration tool: it is a managed SFTP, FTPS, FTP and AS2 endpoint for ongoing file exchange.
  • Believing a new customer can order a Snowball Edge today: AWS says it is no longer available to new customers.

Written from Amazon Web Services's documentation

Try 3 questions on this

Pick an answer to see why each option is right or wrong.

  1. A company must copy 200 TB from an on-premises NFS server into Amazon S3 over a 10 Gbps link, then keep the two in step with a nightly incremental copy that verifies what it wrote. A team proposes a scripted copy on a cron job maintained in house. Which approach will meet these requirements?

    Right. In the game that’s Renown for your Design Cost-Optimized Architectures building.

    Not that one: A is right. In the game your Design Cost-Optimized Architectures building would lose a little Renown, and this question would come back later.

    Written from docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html

  2. A company is leaving a third-party cloud and must move 90 TB of objects from that provider's object storage into Amazon S3, and 4 TB from an on-premises SMB share into Amazon FSx for Windows File Server. A solutions architect wants one service for both moves. Which solution meets these requirements?

    Right. In the game that’s Renown for your Design Cost-Optimized Architectures building.

    Not that one: A is right. In the game your Design Cost-Optimized Architectures building would lose a little Renown, and this question would come back later.

    Written from docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html

  3. A company is shutting down a legacy on-premises NFS filer holding 40 TB of completed project archives. The data is read perhaps once a year, a 10 Gbps link to AWS is available overnight, and the company wants the archives to land in their long-term storage class without a second copying step. Which approach will meet these requirements?

    Right. In the game that’s Renown for your Design Cost-Optimized Architectures building.

    Not that one: B is right. In the game your Design Cost-Optimized Architectures building would lose a little Renown, and this question would come back later.

    Show the full explanation and doc quotes

    The service "works with the following on-premises storage systems: Network File System (NFS)", and the use case is named: "Archive cold data - Move cold data stored in on-premises storage directly to durable and secure long-term storage classes such as S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive. Doing so can free up on-premises storage capacity and help you shut down legacy systems." The transfer is managed rather than scripted: "DataSync automates both the management of data-transfer processes and the infrastructure required for high performance and secure data transfers", with "encryption and data integrity validation to help make sure that your data arrives securely, intact, and ready to use."

    Written from docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html