Amazon Senior Software Development Engineer AI/ML AWS Neuron Model Inference 

United States, California, Cupertino 

34248779

Key job responsibilities
This role will help lead the efforts in building distributed inference support for Pytorch in the Neuron SDK. This role will tune these models to ensure highest performance and maximize the efficiency of them running on the customer AWS Trainium and Inferentia silicon and servers. Strong software development using Python, System level programming and ML knowledge are both critical to this role. Our engineers collaborate across compiler, runtime, framework, and hardware teams to optimize machine learning workloads for our global customer base. Working at the intersection of software, hardware, and machine learning systems, you'll bring expertise in low-level optimization, system architecture, and ML model acceleration. In this role, you will:* Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators.
* Participate in all stages of the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.
* Build infrastructure to systematically analyze and onboard multiple models with diverse architecture.
* Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models
* Analyze and optimize system-level performance across multiple generations of Neuron hardware
* Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks
* Implement optimizations such as fusion, sharding, tiling, and scheduling
* Conduct comprehensive testing, including unit and end-to-end model testing with continuous deployment and releases through pipelines.
* Work directly with customers to enable and optimize their ML models on AWS accelerators
* Collaborate across teams to develop innovative optimization techniquesA day in the life
You will also build high-impact solutions to deliver to our large customer base and participate in design discussions, code review, and communicate with internal and external stakeholders. You will work cross-functionally to help drive business decisions with your technical input. You will work in a startup-like development environment, where you’re always working on the most important initiative.

- Bachelor's degree in computer science or equivalent
- 5+ years of non-internship professional software development experience
- 5+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Fundamentals of Machine learning and LLMs, their architecture, training and inference lifecycles along with work experience on some optimizations for improving the model execution.
- Software development experience in C++, Python (experience in at least one language is required).
- Strong understanding of system performance, memory management, and parallel computing principles.
- Proficiency in debugging, profiling, and implementing best software engineering practices in large-scale systems.