Generative AI Inference Recommendation for Amazon SageMaker now available in the SageMaker AI Studio

Amazon SageMaker AI Studio: Generative AI Inference Recommendations

Amazon SageMaker AI Studio now offers Generative AI Inference Recommendations, providing a guided, low-code, no-code path to find the best inference configuration for your workload. This feature builds on the API-based launch in April 2026, extending the same benchmarking infrastructure to teams that prefer a visual workflow over programmatic access.

Deploying generative AI models in production requires finding the right combination of instance type, serving container, and optimization strategy. The new experience allows customers to describe their workload and what matters most, whether that's latency, throughput, or cost, and SageMaker AI does the rest. It benchmarks multiple configurations on real GPU infrastructure using NVIDIA AIPerf, applies goal-aligned techniques, and returns ranked, production-ready recommendations with measured performance data. Teams can achieve a validated configuration in hours instead of weeks.

What to do

• In SageMaker AI Studio under Jobs, Inference optimization, select a use-case profile, choose an optimization goal, and pick your model.
• Recommendations are ranked by TTFT, inter-token latency, throughput, and cost, and can be compared visually before deploying to a SageMaker real-time endpoint directly from Studio.

There is no additional cost for generating recommendations. Standard compute costs apply for optimization jobs and endpoints provisioned during benchmarking. This capability is available in multiple regions.

Source: AWS release notes

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.