Amazon Software Development Engineer AI/ML Inference Serving AWS Neuron 

United States, California, Cupertino 

724603681

Key job responsibilities
* Architect and lead the design of distributed ML serving systems optimized for generative AI workloads
* Drive technical excellence in performance optimization and system reliability across the Neuron ecosystem
* Design and implement scalable solutions for both offline and online inference workloads
* Lead integration efforts with frameworks such as vLLM, SGLang, Torch XLA, TensorRT, and Triton
* Develop and optimize system components for tensor/data parallelism and disaggregated serving
* Implement and optimize custom PyTorch operators and NKI kernels
* Mentor team members and provide technical leadership across multiple work streams
* Drive architectural decisions that impact the entire Neuron serving stack* Author technical documentation, design proposals, and architectural guidelinesA day in the life
You'll lead critical technical initiatives while mentoring team members. You'll collaborate with cross-functional teams of applied scientists, system engineers, and product managers to architect and deliver state-of-the-art inference capabilities. Your day might involve:* Leading design reviews and architectural discussions
* Rapidly prototyping software to show customer value
* Debugging complex performance issues across the stack
* Mentoring junior engineers on system design and optimization
* Collaborating with research teams on new ML serving capabilities
* Driving technical decisions that shape the future of Neuron's inference stack
Recent shares:

- 5+ years of programming using a modern programming language such as Java, C++, or C#, including object-oriented design experience
- 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- 5+ years of non-internship professional software development experience
- Experience as a mentor, tech lead or leading an engineering team

- Master's degree in computer science or equivalent
- Deep expertise in ML Frameworks/Libraries such as JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, TensorRT.