Bedrock cross-Region and application inference profiles: AIP-C01

Try 3 questions on this

Pick an answer to see why each option is right or wrong.

1. A software company runs three internal GenAI applications in one AWS account. All three applications call the same Anthropic Claude model through the Amazon Bedrock Converse API. The finance team must see each application's Amazon Bedrock spend in AWS Cost Explorer every month so that it can charge each business unit. The applications must continue to share the same AWS account. Which solution will meet these requirements with the LEAST operational overhead?

   Right. In the game that’s Renown for your Operational Efficiency and Optimization for GenAI Applications building.

   Not that one: C is right. In the game your Operational Efficiency and Optimization for GenAI Applications building would lose a little Renown, and this question would come back later.

   Your answerA.Route each application through a different geographic cross-Region inference profile for the model. Filter Cost Explorer by AWS Region to separate each application's monthly Amazon Bedrock spend.Geographic profiles route across several Regions, so Region is not a per-application boundary in Cost Explorer.Your answerB.Enable model invocation logging to Amazon S3. Use Amazon Athena to total the input and output tokens for each application's IAM role every month, and multiply the totals by the published token prices.This rebuilds billing from token counts every month. It works, but it is the most work of any option and not in Cost Explorer.The answerC.Create an application inference profile for each application from the model, and attach a cost allocation tag to each profile. Activate the tag, and change each application to call its profile ARN.Application inference profiles exist for this: they "let you attribute Amazon Bedrock costs by application, team, or workload for the Amazon Bedrock InvokeModel and Converse APIs". Each profile "carries cost allocation tags that flow to AWS Cost Explorer", and the application uses "the profile ARN in place of the model ID in your API calls".Your answerD.Create an Amazon CloudWatch dashboard that graphs the InputTokenCount and OutputTokenCount metrics for the model. Divide the monthly total between the business units by each unit's number of users.Model-level metrics cannot separate the applications, and splitting by user count is an estimate, not attribution.

   Written from docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-application-inference-profiles.html

2. A European bank runs an internal policy assistant on Amazon Bedrock in the eu-central-1 Region. Each Converse request sends the same 25,000-token policy manual in the system prompt, followed by a short employee question. During peak hours, requests fail with ThrottlingException because the application reaches the model's tokens-per-minute quota. The bank must reduce the throttling without a quota increase and without reserving capacity. Prompts and responses must be processed only in AWS Regions in the EU. Which solutions will meet these requirements? (Select TWO)

   Right. In the game that’s Renown for your Operational Efficiency and Optimization for GenAI Applications building.

   Not that one: D,E is right. In the game your Operational Efficiency and Optimization for GenAI Applications building would lose a little Renown, and this question would come back later.

   Your answerA.Change the model ID in all the requests to the global cross-Region inference profile for the model.Global routing does add headroom and is cheaper, which makes it tempting. It sends requests anywhere in the world, and the bank's data must stay in the EU.Your answerB.Create an application inference profile from the model in eu-central-1, and send the requests through the ARN of that profile.An application profile over one Region is a tag and a tracking handle. The requests still hit the same quota in eu-central-1.Your answerC.Set the service_tier parameter to flex on the requests during peak hours, so that the requests are served at a discounted price with longer processing times.Flex is a discount for work that can wait. It draws on the same on-demand quota, so peak requests are still throttled.Your answerD.Change the model ID in the requests to the EU cross-Region inference profile for the model.Your answerE.Add a cache checkpoint after the policy manual in the system prompt, so that repeated copies of the manual are read from the prompt cache.

   Written from docs.aws.amazon.com/bedrock/latest/userguide/global-cross-region-inference.html

3. A company runs an internal LLM gateway on Amazon ECS. The gateway calls Amazon Bedrock through the Converse API for 15 product teams, and every call uses one IAM role that the gateway assumes. Model invocation logging to Amazon CloudWatch Logs is enabled. Teams are added and removed every week, so the company does not want to create an Amazon Bedrock resource for each team. The finance team must see the monthly Amazon Bedrock cost of each product team in AWS Cost Explorer. The platform team must be able to find which product team sent any individual request in the invocation logs. Which solution will meet these requirements?

   Right. In the game that’s Renown for your Operational Efficiency and Optimization for GenAI Applications building.

   Not that one: D is right. In the game your Operational Efficiency and Optimization for GenAI Applications building would lose a little Renown, and this question would come back later.

   Your answerA.Create an application inference profile for each team from the model, and tag each profile with the team name. Activate the tag as a cost allocation tag. Have the gateway call the profile ARN of each team, and find the profile that served each request in the invocation logs.A profile for each team is a new Amazon Bedrock resource every time a team is added, which the company does not want.Your answerB.Add the team name to the requestMetadata field of each Converse request that the gateway sends to Amazon Bedrock. Activate the team key as a cost allocation tag in the AWS Billing and Cost Management console. Group Cost Explorer by the tag, and find the team of each request in the invocation logs.Request metadata is recorded in invocation logs, not in billing data, so there is no team cost allocation tag for Cost Explorer to group by.Your answerC.Have the gateway assume the role for each team with a fixed session name and a team session tag, and allow sts:TagSession in the role trust policy. Activate the tag as a cost allocation tag. Find the team of each request from the session tag in the invocation logs.Session tags are not written to model invocation logs, and a fixed session name gives every team the same identity.arn, so no log record shows the team.The answerD.Have the gateway assume the role for each team with a fixed session name and a team session tag, and allow sts:TagSession in the role trust policy. Activate the tag as a cost allocation tag. Also add the team name to the requestMetadata field of each Converse request.Each requirement needs a different mechanism. The IAM principal attribution page says session tags are "useful for passing user-specific attributes like email, team, or cost center through a shared role", and that "To make your IAM principal tags appear in AWS Cost Explorer and CUR 2.0, you must activate them as cost allocation tags". The same page warns: "Session tags are not written to your model invocation logs ; logs capture the caller's identity.arn instead." The request metadata page (https://docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-request-metadata.html) says "The tags are recorded with the request in your model invocation logs". Session tags cover Cost Explorer, and requestMetadata covers the logs.

   Written from docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-iam-principal-tracking.html