Skip to content
Theme
OnDevice Tools

SoftwareCloud & Services

AWS Lambda GPU support brings serverless inference

AWS has added GPU instances to Lambda Managed Instances, letting event-driven functions run machine learning inference and other accelerated jobs without server management.

Illustration of AWS Lambda GPU capacity provider running serverless inference containers

Key takeaways

  • AWS Lambda now supports G-series GPU instances for Event Functions on Lambda Managed Instances.
  • GPU functions use container images, with Lambda handling provisioning, scaling and event integration.
  • AWS says the feature is aimed at inference, media processing, rendering and scientific computing rather than model training.
  • Standard Amazon EC2 pricing plus a management fee applies, and new accounts may need a G-series quota increase.
In this article
  1. What changed
  2. How GPU functions work
  3. Workloads and limits
  4. Pricing, quotas and setup
  5. Why it matters

What changed

AWS Lambda GPU support is now available for Event Functions on Lambda Managed Instances, according to AWS News Blog. The update, announced on 18 November 2026, adds G-series GPU instances to Lambda. These instances provide graphics processors for accelerated computing.

The change lets a function run on a GPU without the user setting up a cluster, installing drivers or writing scaling logic. AWS says the aim is to make accelerated computing as straightforward as ordinary serverless functions, where the provider runs the underlying servers.

How GPU functions work

Illustration: graphics card, code, Amazon logo, NVIDIA logo and the labels Amazon EC2, Aws Lambda, GPU

GPU-accelerated functions are packaged as container images, a portable format that bundles code and dependencies, using the Open Container Initiative standard. A developer builds an image, links the function to it, and Lambda handles provisioning, scaling and event connections.

AWS says users do not need to manage GPU drivers, CUDA versions or machine images. CUDA is the software toolkit used to run GPU code. Users also do not need to operate device plugins, which connect GPUs to container runtimes, or separate clusters. The functions run on the same Lambda Managed Instances foundation used for other managed compute options.

A capacity provider is a configuration that defines which GPU instance types Lambda may use. AWS says listing several types can help the service choose an available and cost-effective option, with fallback across instance generations when capacity is scarce.

AWS also says GPU functions can be part of larger workflows. Durable functions can coordinate multi-step jobs with checkpoints and recovery. MicroVMs, small virtual machines, can run untrusted or model-generated code in isolated sandboxes, then call a GPU function for inference.

Workloads and limits

AWS positions the feature for inference rather than model training. It is aimed at custom models, not the large foundation models already offered through Amazon Bedrock.

Examples given by AWS include custom decision models, computer vision, speech conversion, recommendation systems, document analysis and small to mid-sized language models. The blog also lists agentic AI, where software completes multi-step tasks, event-driven AI, media transcoding, 3D rendering, generative media, scientific computing and financial modelling.

Functions can use container images as large as 32 GB, with each file capped at 10 GB. Temporary storage can reach 32 GB. Memory ranges from 2,048 MB to 32,768 MB, while accelerator memory, meaning memory on the GPU, ranges from three GB to 48 GB.

Each execution environment, the isolated runtime that runs a function, can handle up to 48 concurrent requests. A capacity provider can draw on as many as 15,000 vCPUs, or virtual CPU units. AWS says longer timeouts, up to 90 minutes, are available for asynchronous calls and event source mappings, which connect functions to event sources. This supports batch jobs and longer media or inference tasks without redesign.

Pricing, quotas and setup

AWS says GPU functions use standard Amazon EC2 pricing plus a management fee. The company also says the design groups functions onto shared instances and tries to keep each GPU active during concurrent requests. AWS says this improves utilisation.

Before creating a GPU function, AWS advises checking the quota for G-series instances. The relevant quota is named Running On-Demand G and VT instances. New accounts can begin with zero allowance, so an increase may be needed.

The example follows four steps: set up a capacity provider, prepare and upload the image, create the function, then publish and invoke it. AWS used a capacity provider named lmi-gpu-cp, built a container image and pushed it to Amazon ECR, the AWS container image registry. The example used a vLLM base image, an inference server, and set the MODEL_ID environment variable to load Qwen/Qwen2.5-1.5B-Instruct from Hugging Face, a model hub. AWS says the base image includes the vLLM inference server, PyTorch, Transformers and the CUDA toolkit. AWS says no inference handler code was needed. The MODEL_ID setting can point to Hugging Face, ModelScope, Amazon S3 or a local path. For private or gated models, AWS says an HF_TOKEN setting can be used.

The example also used AcceleratorConfig to request GPU memory. The minimum accelerator memory setting helps Lambda choose the smallest suitable GPU that has enough memory. AWS says the selected GPU may have more memory than requested. A concurrency setting of four was used, and memory was set to four GiB per virtual CPU, giving the function 16,384 MB and four vCPUs. AWS says the vLLM server batches concurrent requests so they share one model copy. AWS says this leaves room on the 2xlarge instance for management components.

Why it matters

The update targets a common problem in artificial intelligence and media applications: GPUs are costly, but a single small job often leaves much of the hardware idle. By placing multiple functions on shared instances and batching requests, AWS says it can raise utilisation.

For developers, the main benefit is less infrastructure work. Teams can connect GPU inference to events, queues, storage changes or other AWS triggers without building a separate serving platform. AWS also says external serverless GPU services may lack native event sources, identity controls and networking.

The feature does not replace large managed model services. AWS says it is best for custom inference and workloads that need to sit inside an AWS application. Organisations should review quotas, instance availability, container size limits and cost before moving production workloads.

Sources

  1. AWS Lambda now supports GPU instances for Event Functions on Lambda Managed InstancesAWS News Blog

Tags

Spotted a mistake? See our corrections policy. Read how News by OnDevice works in our editorial policy.