InfoQ

AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications

6.9内容质量
AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications

TL;DR · AI 摘要

AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications - InfoQ InfoQ Homepage News AWS Lau...

核心要点

  • 主题聚焦:AWS Launches TorchNeuron to Run PyTorch Models o
  • 来源:InfoQ,建议结合原文判断细节。
  • AI 分析暂不可用,本条为保底评分与摘要。
#AI#编程#后端#安全#产品
打开原文

AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications - InfoQ

InfoQ Homepage News AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications

AI, ML & Data Engineering

Scaling OpenTelemetry: Managing Collectors, Configuration, and Cost (Webinar Nov 19th)

AWS Launches TorchNeuron to Run PyTorch Models on AWS Trainium Without Modifications

Nov 09, 2026 2 min read

by

  • Sergio De Simone

#### Follow us on

Youtube

232K Followers

Linkedin

26K Followers

Instagram

New

RSS

19K Readers

X

57.1k Followers

Facebook

21K Likes

Bluesky

Listen to this article -

0:00

Audio ready to play

Your browser does not support the audio element.

Normal

1.25x

1.5x

Like

new dropdown rading list

  • Reading list

At the recent PyTorch Conference North America 2026 , Amazon introduced TorchNeuron , a native PyTorch backend that enables developers to run existing PyTorch code on AWS Trainium AI chips with virtually no code modifications.

Beyond reducing engineering effort, TorchNeuron allows enterprises to share hardware capacity between training and inference workloads, reallocating resources from inference to training during off-peak hours.

Amazon says that a single configuration change is sufficient to switch hardware , eliminating the need to rewrite and revalidate models when moving to Trainium.

Today, 85% of AI researchers and developers write their code in PyTorch. Until now, the hardware that code could run on was far more limited than the code itself, and expanding to new chips meant rewriting and revalidating work that took months to perfect.

According to the TorchNeuron documentations , when leveraging PyTorch's automatic device detection through torch.accelerator , the only required change is replacing .to('cuda') to .to('neuron') . The backend also natively supports distributed training and PyTorch parallelism APIs, including DDP, FSDP2, DTensor, and others. In addition, TorchNeuron provides native support for Torch streams and events through torch_neuronx.Stream and torch_neuronx.Event .

Amazon Annapurna Labs vice president Colin Brace, who leads the team behind TorchNeuron, emphasized that his team validated the 100,000 most popular models and more than 800 ML architectures to ensure a smooth developer experience when adopting the new backend.

Additionally, TorchNeuron enables developers and researchers to use the Neuron Kernel Interface to develop, optimize, and execute custom operators:

Developers can obtain finer performance control beyond Adaptive Eager and torch.compile by calling performance critical NKI kernels directly in model code to replace sequences of standard PyTorch operations (ATen ops).

Commenting on Amazon's announcement on LinkedId, Liner Notes Consulting CEO Yasuo Maeda noted that:

The real test will be whether developers can achieve competitive performance and total cost of ownership in production. The AI infrastructure race is not just about who builds the best chips. It is also about who makes them easiest to use at scale.

Similarly, user Pratham pratham hightlighted that, while reducing migration friction is important to avoiding accelerator lock-in, the real test lies in workload-specific performance, cost per token, and reproducibility .

Amazon also emphasizes that TorchNeuron has already been adopted by several organizations, including Cartesia, which is using it for voice AI inference, and the Lieber Institute for Brain Development, which explores applications in drug target identification and drug discovery.

AWS TorchNeuron announcement comes on the heels of Google's launch of TorchTPU , its native PyTorch backend for Google's TPU hardware.

main wrapper for authors section

About the Author

section title

main wrapper for each author

#### Sergio De Simone

Show more

Show less

#### This content is in the AI, ML & Data Engineering topic

##### Related Topics:

  • Development
  • AI, ML & Data Engineering
  • AWS
  • PyTorch
  • Machine Learning
  • Large language models
  • Related Editorial
  • Related Sponsors AI Security Playbook: Move Faster with AI, Without Compromising Security
  • Related Sponsor Equal Experts helps organizations harness AI, modernize technology, and build better software through expert engineering, collaboration, and innovative delivery practices. Learn more .

The InfoQ Newsletter

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

We protect your privacy.