AWS SageMaker Now Supports Custom Nova Model Inference

Amazon SageMaker now offers general availability of support for custom Nova models, enabling users to deploy and scale full-rank customized Nova models with a production-grade, configurable, and cost-efficient managed inference service. This enhancement builds upon the Amazon Nova customization capabilities introduced at AWS re:Invent 2025 and addresses customer demand for the same level of control and flexibility available when customizing open-weight models.

Reducing Inference Costs and Enhancing Flexibility

With Amazon SageMaker Inference for custom Nova models, users can reduce inference costs through optimized GPU utilization using Amazon Elastic Compute Cloud (Amazon EC2) G5 and G6 instances, as opposed to P5 instances. Auto-scaling, based on 5-minute usage patterns, and configurable inference parameters further contribute to cost efficiency. This feature supports continued pre-training, supervised fine-tuning, or reinforcement fine-tuning for specific use cases, allowing for advanced configurations of context length, concurrency, and batch size to optimize the latency-cost-accuracy tradeoff.

Deploying Custom Nova Models

Custom Nova models can be deployed through the SageMaker Studio interface or the SageMaker AI SDK. Within SageMaker Studio, users can select a trained Nova model and deploy it by choosing the “Deploy” button and configuring endpoint settings. Currently supported instance types for Nova Micro include g5.12xlarge, g5.24xlarge, g5.48xlarge, g6.12xlarge, g6.24xlarge, and g6.48xlarge, as well as p5.48xlarge. Nova Lite supports g5.48xlarge, g6.48xlarge, and p5.48xlarge, while Nova 2 Lite is compatible with p5.48xlarge.

Code Example: Deploying with the SageMaker AI SDK

The following code snippet demonstrates how to create a SageMaker AI model and endpoint configuration using the SageMaker AI SDK:

  # Create a SageMaker AI model model_response = sagemaker.create_model( ModelName="Nova-micro-ml-g5-12xlarge", PrimaryContainer={ 'Image': '708977205387.dkr.ecr.us-east-1.amazonaws.com/nova-inference-repo:v1.0.0', 'ModelDataSource': { 'S3DataSource': { 'S3Uri': 's3://your-bucket-name/path/to/model/artifacts/', 'S3DataType': 'S3Prefix', 'CompressionType': 'None' } }, # Model Parameters 'Environment': { 'CONTEXT_LENGTH': 8000, 'MAX_CONCURRENCY': 16, 'DEFAULT_TEMPERATURE': 0.0, 'DEFAULT_TOP_P': 1.0 } }, ExecutionRoleArn=SAGEMAKER_EXECUTION_ROLE_ARN, EnableNetworkIsolation=True ) print("Model created successfully!") # Create Endpoint Configuration production_variant = { 'VariantName': 'primary', 'ModelName': 'Nova-micro-ml-g5-12xlarge', 'InitialInstanceCount': 1, 'InstanceType': 'ml.g5.12xlarge', } config_response = sagemaker.create_endpoint_config( EndpointConfigName="Nova-micro-ml-g5-12xlarge-Config", ProductionVariants= production_variant ) print("Endpoint configuration created successfully!") # Deploy your Noval model endpoint_response = sagemaker.create_endpoint( EndpointName="Nova-micro-ml-g5-12xlarge-endpoint", EndpointConfigName="Nova-micro-ml-g5-12xlarge-Config" ) print("Endpoint creation initiated successfully!")  

Real-Time Inference and Streaming Support

Amazon SageMaker AI supports synchronous endpoints for real-time inference with both streaming and non-streaming modes, as well as asynchronous endpoints for batch processing. Streaming chat requests can be configured with parameters like max_tokens, temperature, top_p, and reasoning effort. The following example demonstrates a streaming chat request:

  streaming_request = { "messages": [ {"role": "user", "content": "Compare our Q4 2025 actual spend against budget across all departments and highlight variances exceeding 10%"} ], "max_tokens": 512, "stream": True, "temperature": 0.7, "top_p": 0.95, "top_k": 40, "logprobs": True, "top_logprobs": 2, "reasoning_effort": "low", # Options: "low", "high" "stream_options": {"include_usage": True} } invoke_nova_endpoint(streaming_request)  

Regional Availability and Pricing

Amazon SageMaker Inference for custom Nova models is currently available in US East (N. Virginia) and US West (Oregon) AWS Regions. It supports Nova Micro, Nova Lite, and Nova 2 Lite models running on EC2 G5, G6, and P5 instances with auto-scaling. Pricing is based on compute instance usage, with per-hour billing and no minimum commitments. More information on pricing can be found on the Amazon SageMaker AI Pricing page.

For more information and to get started, visit the Amazon SageMaker AI console and provide feedback through AWS re:Post for SageMaker or your AWS Support contacts.