Certopolis

AWS Certified Generative AI Developer – Professional

Bedrock service tiers vs Provisioned Throughput for AIP-C01

On-demand requests run on the Standard tier unless you set service_tier to priority (faster, at a premium), flex (slower, at a discount) or reserved (capacity you reserve per month). Provisioned Throughput buys model units billed every hour until you delete it. Batch inference processes files of prompts from S3 asynchronously.

Service tiers

Set the optional service_tier parameter on a runtime call to reserved, priority, default or flex. Without it, requests go to Standard.

Priority gives the fastest response times at a price premium and needs no reservation. Its requests are served ahead of Standard and Flex. Flex gives a discount for workloads that can wait, such as evaluations and summarization.

Reserved lets you reserve input and output tokens-per-minute capacity for 1 or 3 months, with minimums of 100,000 input and 10,000 output TPM, billed monthly. Usage above the reservation overflows to Standard. Access is through your AWS account team.

Your on-demand quota is shared across priority, default and flex. The reserved capacity is separate. So a Flex job can use up the quota that a Standard or Priority assistant needs, and only a Reserved reservation takes the assistant out of that shared quota.

Provisioned Throughput

Provisioned Throughput buys model units (MUs). Each MU gives a set number of input and output tokens per minute. It is billed hourly whether or not you use it, and billing continues until you delete it.

Commitment can be none, 1 month or 6 months. A longer commitment gives a lower hourly price. With no commitment you can delete it at any time. With a term you can't delete it until the term ends.

Example: an HR assistant gets about 40 requests per business day on a no-commitment Provisioned Throughput. On-demand meets its latency needs, so deleting the Provisioned Throughput and calling the model ID stops paying for idle hours.

Batch inference

Batch inference takes prompts in files in S3, runs them asynchronously and writes the results to S3. It is not supported for provisioned models. Jobs with the Converse invocation type do not support tool calling. Bedrock sends job state changes to Amazon EventBridge, so you do not need to poll.

Where the exam catches people

  • Believing priority on the assistant stops throttling caused by a Flex job: priority, default and flex share one on-demand quota. Reserved capacity is separate.
  • Believing a lightly used Provisioned Throughput costs less when requests are smaller: it is billed hourly regardless of use.
  • Believing a cost-cutting change for an interactive chat can be batch inference: batch is asynchronous, so users would not get answers while they wait.
  • Believing Flex needs a new API or a reservation: it is the service_tier parameter on the same runtime call.
  • Believing a 1-month or 6-month Provisioned Throughput can be deleted early: it can't be deleted until the term is over.

Written from Amazon Web Services's documentation

Try 3 questions on this

Pick an answer to see why each option is right or wrong.

  1. A company bought a no-commitment Provisioned Throughput with one model unit for an Amazon Bedrock base model six months ago. An internal HR assistant is the only workload that uses it. The assistant receives about 40 requests each business day and none at night or on weekends. Tests show that on-demand inference for the same model meets the assistant's latency needs. The company must reduce the cost of the assistant. Employees must still receive each answer while they wait in the chat. Which solution will meet these requirements MOST cost-effectively?

    Right. In the game that’s Renown for your Operational Efficiency and Optimization for GenAI Applications building.

    Not that one: B is right. In the game your Operational Efficiency and Optimization for GenAI Applications building would lose a little Renown, and this question would come back later.

    Written from docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html

  2. A company uses an Anthropic Claude model on Amazon Bedrock through the Converse API. Each night, an AWS Lambda function sends about 40,000 synchronous Converse requests that grade the previous day's chatbot answers against a scoring rubric. The job has a six-hour window and currently finishes in two hours. The company wants to lower the inference cost of this job. The company must keep the same model and the synchronous Converse calls, and must not reserve capacity for a fixed term. Which solution will meet these requirements MOST cost-effectively?

    Right. In the game that’s Renown for your Operational Efficiency and Optimization for GenAI Applications building.

    Not that one: D is right. In the game your Operational Efficiency and Optimization for GenAI Applications building would lose a little Renown, and this question would come back later.

    Written from docs.aws.amazon.com/bedrock/latest/userguide/service-tiers-inference.html

  3. A retail company uses one Anthropic Claude model on Amazon Bedrock in a single AWS account and Region. A customer-facing shopping assistant calls the Converse API without the service_tier parameter. A catalog enrichment job calls the same model all day with service_tier set to flex. Since the enrichment job launched, the assistant has received ThrottlingException errors at peak hours. The enrichment job must keep its current request rate. The assistant must no longer be throttled because of the load from the enrichment job. Which solution will meet these requirements?

    Right. In the game that’s Renown for your Operational Efficiency and Optimization for GenAI Applications building.

    Not that one: C is right. In the game your Operational Efficiency and Optimization for GenAI Applications building would lose a little Renown, and this question would come back later.

    Written from docs.aws.amazon.com/bedrock/latest/userguide/service-tiers-inference.html