Certopolis

AWS Certified Solutions Architect – Associate

S3 performance: prefixes, multipart upload, Transfer Acceleration

Amazon S3 supports at least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per partitioned prefix, so spread heavy traffic across prefixes and parallel connections. Use multipart upload for objects of 100 MB or more so parts upload in parallel and a failed part is resent alone. Use S3 Transfer Acceleration when clients far from the bucket's Region upload or download large amounts of data.

Request rates and prefixes

There is no limit on the number of prefixes in a bucket. With 10 prefixes, reads can scale to 55,000 requests per second. Scaling to a higher rate happens gradually, not instantly. While S3 scales you may see 503 (Slow Down) errors, which stop when scaling completes. The AWS SDKs retry 503 errors automatically.

Spreading across prefixes raises request rate, not per-request latency. For single-digit millisecond latency or higher transfer rates on one connection, AWS points to caching with Amazon CloudFront or Amazon ElastiCache in front of S3. Running compute in the same Region as the bucket also cuts latency and data transfer cost.

Multipart upload and byte-range fetches

Multipart upload splits one object into parts numbered 1 to 10,000. Parts can upload in any order and in parallel. If a part fails, only that part is resent. You can pause and resume, and start before you know the final size. AWS calls it a best practice for objects of 100 MB or larger.

Parts of an unfinished upload are stored and billed until you complete or abort the upload; an upload has no expiry of its own. Add a lifecycle rule with the AbortIncompleteMultipartUpload action to delete abandoned uploads after a set number of days.

For downloads, the Range header fetches part of an object. Several byte-range requests over separate connections give higher total throughput than one whole-object request, and a failed range retries faster.

S3 Transfer Acceleration

Transfer Acceleration routes data through CloudFront's edge locations and then over an optimized network path to S3. AWS lists three cases: customers uploading to a central bucket from around the world, regular transfers of gigabytes to terabytes across continents, and clients that cannot use all their bandwidth over the internet.

You enable it per bucket and use the bucket-name.s3-accelerate.amazonaws.com endpoint. The bucket name must be DNS-compliant and must not contain periods. Only virtual-hosted style requests work. Speeds can take up to 20 minutes to improve after you enable it, and extra data transfer charges may apply.

Where the exam catches people

  • Believing 503 Slow Down on a new busy prefix means a misconfiguration: S3 scales gradually; retry with backoff and the errors stop.
  • Believing more prefixes lower latency: they raise the request rate; cache with CloudFront or ElastiCache for lower latency.
  • Believing abandoned multipart uploads clean themselves up: the parts are billed until an abort, so add an AbortIncompleteMultipartUpload lifecycle rule.
  • Believing Transfer Acceleration works on any bucket name: names with periods cannot use the accelerate endpoint.
  • Believing Transfer Acceleration fixes a failed large upload on its own: pair it with multipart upload so only the failed part is resent.

Written from Amazon Web Services's documentation

Try 3 questions on this

Pick an answer to see why each option is right or wrong.

  1. A media company collects 20 GB video files from studios on four continents into one S3 bucket in us-east-1. Studios report slow uploads that fail near the end and restart from the beginning, and none of them can saturate their own internet connections. The company needs the studios' uploads to reach the us-east-1 bucket quickly and reliably. Which approach will meet these requirements?

    Right. In the game that’s Renown for your Design High-Performing Architectures building.

    Not that one: C is right. In the game your Design High-Performing Architectures building would lose a little Renown, and this question would come back later.

    Show the full explanation and doc quotes

    On distance and unusable bandwidth, Transfer Acceleration "takes advantage of the globally distributed edge locations in Amazon CloudFront. As the data arrives at an edge location, the data is routed to Amazon S3 over an optimized network path", and AWS names these exact cases for it: customers uploading "to a centralized general purpose bucket from all over the world", transfers of "gigabytes to terabytes of data on a regular basis across continents", and "you can't use all of your available bandwidth over the internet when uploading to Amazon S3." On restarting, multipart upload means "if transmission of any part fails, you can retransmit that part without affecting other parts", and AWS calls it a best practice for objects of 100 MB or larger.

    Written from docs.aws.amazon.com/AmazonS3/latest/userguide/transfer-acceleration.html

  2. An application's traffic to a new Amazon S3 prefix rises sharply at the start of a campaign, and the client logs fill with 503 Slow Down responses for several minutes before the errors stop on their own. The team wants to know whether this indicates a misconfiguration. Which solution meets these requirements?

    Right. In the game that’s Renown for your Design High-Performing Architectures building.

    Not that one: A is right. In the game your Design High-Performing Architectures building would lose a little Renown, and this question would come back later.

    Show the full explanation and doc quotes

    "The scaling, in the case of both read and write operations, happens gradually and is not instantaneous, and actual performance will vary based on your specific workload characteristics, usage patterns, and system configuration. While Amazon S3 is scaling to your new higher request rate, you may see some 503 (Slow Down) errors. These errors will dissipate when the scaling is complete." That matches the report exactly - a sharp rise, several minutes of 503s, then silence. The client's job is to retry with backoff so the requests succeed during the ramp.

    Written from docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html

  3. A data platform uploads 30 GB extracts to Amazon S3 with multipart upload. Some uploads are abandoned when a job fails, and the storage bill grows although the objects never appear in the bucket listing. The platform team wants the waste to stop without changing the upload code. Which approach will meet these requirements?

    Right. In the game that’s Renown for your Design High-Performing Architectures building.

    Not that one: D is right. In the game your Design High-Performing Architectures building would lose a little Renown, and this question would come back later.

    Written from docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html