国际新闻 · 中文译文与原文对照

星期日 最新 1,000 篇 · 滚动浏览

科技

AWS Lambda现已支持通过48个并发调用在每个GPU上共享一个模型来执行GPU推理任务。

来源:InfoQ 技术发布 上传时间 (多伦多时间)1 家来源 · 1 篇报道

中文译文

AWS已在AWS Lambda中添加了对GPU的支持:在Lambda托管实例上运行的事件函数现在可以在作为容器镜像部署的EC2 G系列实例(发布时包括G4dn、G5、G6、G6f、Gr6、Gr6f和G6e)上运行,并且可以由与其他函数相同的事件源触发。该功能已在同时拥有托管实例和G系列实例的商业区域正式推出。其定价为EC2按需实例的价格加上9.75%的管理费,以及每百万次请求0.20美元的标准费用。

正如InfoQ去年12月所报道的,托管实例允许函数在客户自行选择并付费的EC2资源上运行,而Lambda则负责资源配置、补丁更新和扩展,同时多个调用可以共享同一个执行环境。在最初发布时并未支持GPU功能。AWS高级解决方案架构师Daniel Abib阐述了这一变更对运营方的影响:你构建一个OCI镜像,将函数指向该镜像,让Lambda负责资源调配、扩展以及事件集成。无需处理GPU驱动、CUDA版本或AMI,也无需管理集群或设备插件。

容量提供方会列出允许的实例类型、VPC以及vCPU上限;Lambda会选择有可用容量的最新一代实例,当最新一代实例不足时则回退到较旧世代:

“InstanceRequirements”: {
“AllowedInstanceTypes”: ["g5.2xlarge", “g6.2xlarge”, “g6e.2xlarge”]
},
“CapacityProviderScalingConfig”: { “ScalingMode”: “Auto”, “MaxVCpuCount”: 96 }

该函数通过一个新的AcceleratorConfig块来指定GPU内存,该块与内存、超时时间、存储以及每个环境下的并发数一同定义:

“MemorySize”: 16384,
“Timeout”: 900,
“EphemeralStorage”: { “Size”: 32768 },
“Environment”: { “Variables”: { “MODEL_ID”: “Qwen/Qwen2.5-1.5B-Instruct” } },
“CapacityProviderConfig”: {
“LambdaManagedInstancesCapacityProviderConfig”: {"CapacityProviderArn": "arn:aws:lambda:us-east-2:111122223333:capacity-provider:lmi-gpu-cp",
"ExecutionEnvironmentMemoryGiBPerVCpu": 4,
"PerExecutionEnvironmentMaxConcurrency": 4
}
},
"AcceleratorConfig": { "AcceleratorMemorySize": 16 }

同一环境中的多个并发调用会共享GPU内存中的同一份模型副本,每个环境最多支持48个并发调用,而资源分配则会根据GPU利用率、CPU利用率以及并发数量来进行调整。发布新版本时,系统会先准备好相关环境,下载模型权重,并在第一个事件到达之前预热引擎;通过可寻址的OCI懒加载技术,可将环境启动时间缩短多达70%。

AWS提供了五种基于GPU的镜像:仅支持CUDA的镜像、CuPy镜像、PyTorch镜像、vLLM镜像以及SGLang镜像。这两种推理服务器无需任何处理代码;只需通过MODEL_ID变量指定Hugging Face或ModelScope仓库的路径、S3存储路径或本地路径,即可完成模型切换,而模型切换只需进行配置修改后再发布新版本即可:

aws lambda update-function-configuration \--function-name my-gpu-function \
--environment 'Variables={MODEL_ID=s3://amzn-s3-demo-bucket/models/Qwen2.5-1.5B-Instruct/,VLLM_LOAD_FORMAT=runai_streamer}'

相关限制在启动说明中有所规定:容器镜像大小上限为32 GB,临时存储空间为32 GB,内存容量为2至32 GB,加速器内存容量为3至48 GB,每个服务提供商提供的vCPU数量为15,000个,而异步调用及事件源映射调用的超时时间最长为90分钟。Abib用一句话就凸显了自身的优势:那些隐藏相同处理逻辑的无服务器GPU服务“通常位于AWS环境之外,没有内置的事件源、身份认证机制或网络功能。”首批实际测试报告在一天之内就出现了。Classmethod的Akaike在12月发现没有G型实例被接受后,对一场会议演讲进行了修改,他尝试了us-east-1区域的所有加速型实例类型,发现81个实例中有37个被接受,且均为G系列;P系列、G7、Inferentia、Trainium以及基于Graviton的g5g型号的实例均被拒绝,zip格式的包也同样不被接受。在L4实例上运行PyTorch基准测试的速度比在该实例的CPU上快约66倍。不过,Terraform和云控制提供商目前还不知道AcceleratorConfig的存在,新创建的账户中GPU vCPU的配额为零:“如果使用g6.xlarge(4个vCPU)型号,并搭配默认的三个实例,那么需要12个vCPU的配额。”由名为sasidhar-aws的用户在11月16日提交的Terraform提供商问题及相关pull request,以及该用户提交的另一份pull request至今仍未解决。另一项测试由AWS社区构建者xbill进行,他通过vLLM镜像在单个g6.2xlarge实例上运行Gemma 4 E2B模型,该镜像包含了谷歌的4位QAT权重以及内置在9GB镜像中的推测解码功能。一个环境能够处理全部48个同时发起的请求,417次调用全部成功,单个请求的处理速度约为每秒130个标记,48个请求的同时处理速度则在3,400到3,600个标记/秒之间。性能瓶颈在于处理流程而非GPU本身:从代码发布到生成可用版本需要484秒,Docker镜像索引会被Lambda拒绝,直到以单平台格式推送为止,此外处理程序会返回完整回复,因此无法测量生成第一个标记所需的时间。他的结论仅限于此范围:

对于那些需要通过事件、队列或其他AWS服务来调用的模型,使用Lambda GPU函数更为简单:一个镜像、一个函数,Lambda负责保持服务器运行并路由请求。若需同时与少量用户进行对话,可保持推测解码功能启用;而对于大规模批量处理任务,则无需该功能即可发布模型。当需要流式回复或对服务器进行完全控制时,他写道:“请自己在EC2 G6上运行vLLM。”

其定价模式与托管实例所采用的相同,GPU的收费低于CPU实例类型所收取的15%费用。节省计划和预留实例也适用于此类实例,且没有时长费用,只要最小环境数量大于零,就会持续计费。颜翠在12月对这一模式的分析中,当时该模式仍标注着“不支持GPU实例”,这一分析对于GPU场景同样适用:

常规Lambda函数可以通过利用大量共享的EC2实例池实现快速扩展。而使用托管实例时,启动新的EC2实例需要数十秒时间。因此,当流量出现大规模且难以预测的峰值时,这种方案并不合适。托管实例仍然仅支持事件函数,因此GPU路径尚未延伸至Web函数——即今年Lambda新增的基于HTTP服务器的版本,而AWS也尚未表明是否会实现这一功能。

原文

AWS has added GPU support to AWS Lambda: Event Functions running on Lambda Managed Instances can now run on EC2 G-series instances (G4dn, G5, G6, G6f, Gr6, Gr6f and G6e at launch), deployed as container images, and triggered by the same event sources as any other function. The feature is generally available in the commercial regions where both Managed Instances and G-series instances exist. It is priced at the EC2 On-Demand rate plus a 9.75 percent management fee and the standard $0.20 per million requests.

As InfoQ reported last December, Managed Instances let a function run on EC2 capacity the customer selects and pays for, with Lambda handling provisioning, patching and scaling and several invocations sharing one execution environment. GPUs were missing at that launch. Daniel Abib, senior specialist solutions architect at AWS, describes the operator's side of the change:

You build an OCI image, point the function at it, and let Lambda handle provisioning, scaling, and event integration. There are no GPU drivers, CUDA versions, or AMIs to reconcile, and no clusters or device plugins to operate.

A capacity provider lists the allowed instance types, the VPC and a vCPU ceiling; Lambda picks the newest generation with capacity and falls back to older generations when newer ones are short:

"InstanceRequirements": {
"AllowedInstanceTypes": ["g5.2xlarge", "g6.2xlarge", "g6e.2xlarge"]
},
"CapacityProviderScalingConfig": { "ScalingMode": "Auto", "MaxVCpuCount": 96 }

The function declares its GPU memory through a new AcceleratorConfig block, next to memory, timeout, storage, and a per-environment concurrency:

"MemorySize": 16384,
"Timeout": 900,
"EphemeralStorage": { "Size": 32768 },
"Environment": { "Variables": { "MODEL_ID": "Qwen/Qwen2.5-1.5B-Instruct" } },
"CapacityProviderConfig": {
"LambdaManagedInstancesCapacityProviderConfig": {
"CapacityProviderArn": "arn:aws:lambda:us-east-2:111122223333:capacity-provider:lmi-gpu-cp",
"ExecutionEnvironmentMemoryGiBPerVCpu": 4,
"PerExecutionEnvironmentMaxConcurrency": 4
}
},
"AcceleratorConfig": { "AcceleratorMemorySize": 16 }

Concurrent invocations in one environment share a single copy of the model in GPU memory, up to 48 per environment, and scaling follows GPU utilization, CPU utilization, and concurrency. Publishing a version provisions the environments, downloads the weights, and warms the engine before the first event arrives; Seekable OCI lazy loading can cut environment start time by up to 70 percent.

AWS ships five GPU base images: CUDA only, CuPy, PyTorch, vLLM, and SGLang. The two inference servers need no handler code; a MODEL_ID variable points to a Hugging Face or ModelScope repository, an S3 prefix, or a local path, and a model swap is a configuration change followed by a new version:

aws lambda update-function-configuration \
--function-name my-gpu-function \
--environment 'Variables={MODEL_ID=s3://amzn-s3-demo-bucket/models/Qwen2.5-1.5B-Instruct/,VLLM_LOAD_FORMAT=runai_streamer}'

The limits are in the launch post: container images up to 32 GB, 32 GB of ephemeral storage, 2 to 32 GB of memory, 3 to 48 GB of accelerator memory, 15,000 vCPUs per capacity provider, and a timeout of up to 90 minutes for asynchronous and event source mapping invocations. Abib draws the line against the competition in one sentence: serverless GPU services that hide the same work "often sit outside your AWS environment, with no native event sources, identity, or networking."

The first hands-on reports came within a day. Classmethod's Akaike, who had corrected a conference talk in December after finding that no G-type was accepted, tried every accelerated instance type in us-east-1 and found 37 of 81 accepted, all G-family; P-family, G7, Inferentia, Trainium, and the Graviton-based g5g were refused, as were zip packages. A PyTorch benchmark ran about 66 times faster on the L4 than on the instance's CPU. Still, neither Terraform nor the Cloud Control provider knew AcceleratorConfig yet, and the GPU vCPU quota in a fresh account is zero: "if you use g6.xlarge (4 vCPU) with the default three instances, you need a quota of 12 vCPUs." A Terraform provider issue and pull request, opened on 16 November by a user named sasidhar-aws, and a pull request from the same user remain open.

A second test, by an AWS Community Builder writing as xbill, put Gemma 4 E2B on a single g6.2xlarge through the vLLM image, with Google's 4-bit QAT weights and a speculative-decoding drafter baked into a 9 GB image. One environment served all 48 simultaneous requests and every one of 417 calls succeeded, at roughly 130 tokens a second for a single request and 3,400 to 3,600 at 48. The friction was in the pipeline rather than the GPU: 484 seconds from publish to an active version, a Docker image index that Lambda rejected until it was pushed as a single-platform manifest, and a handler that returns complete replies, so time to first token could not be measured. His verdict keeps to that scope:

For a model behind events, queues or other AWS services, the Lambda GPU function is the simpler road: one image, one function, and Lambda keeps the server alive and routes the requests. Keep the drafter on for a chat with a few users at a time, and publish without it for heavy batch load.

When streamed replies or full control of the server are needed, he writes, "run vLLM on an EC2 G6 yourself."

The pricing model is the one Managed Instances introduced, with a lower fee for GPUs than the 15 percent charged on CPU instance types. Savings Plans and Reserved Instances apply to the instance, and there is no duration charge, so an instance bills for as long as the minimum environment count is above zero. Yan Cui's December analysis of the model, written when it still carried the line "GPU instances are NOT supported," applies to the GPU case unchanged:

Regular Lambda functions can scale rapidly by tapping into a large, shared pool of EC2 instances. With managed instances, it takes tens of seconds to launch new EC2 instances. As such, it's not a good fit when you have large, unpredictable spikes in traffic.

Managed Instances still support only Event Functions, so the GPU path does not yet reach Web Functions, the HTTP-server shape Lambda gained this year, and AWS has not said whether it will.