Amazon Software Development Manager Open Data Analytics Engines 

United States, Washington, Redmond 

132417864

As a manager in this space, you will own speeding customer workloads by shipping query performance optimizations in the AWS Runtime for Apache Spark. Your team will apply and extend our automated statistical analysis tools to ensure that these optimizations can be delivered to all AWS Spark customers and allow them to benefit from them out-of-the-box, with no configuration required. This team sped the AWS Runtime for Spark’s 3 TB TPC-DS performance by 1.7x in 2023 through techniques including modifying Spark’s internal data structure implementations, enhancing CodeGen, and enhancing statistics/building advanced heuristics in Spark Catalyst rules to improve query plans both before and during runtime through Adaptive Query Execution. You will drive the team to identify new optimization opportunities, prioritize them, and deliver them to demonstrate speedups vs. Athena/EMR Spark customer workloads and industry benchmarks.You will take requirements from the savviest customers in the business, work with our Product Management group to identify and pursue market-differentiating opportunities to enhance our Spark customers’ experience, and design and deliver these opportunities, solving deep problems with a strong group of Senior and Principal Engineers.* AWS Athena: https://aws.amazon.com/athena
* AWS EMR: http://aws.amazon.com/emr
* AWS Big Data Blog: https://aws.amazon.com/blogs/big-data/
Utility Computing (UC)About AWS
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.Work/Life BalanceMentorship & Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Key job responsibilities
* Prioritize the efforts of a strong engineering team researching and delivering new/enhanced query operator implementation and query plan optimizations
* Ensure data consistency and durability while obtaining breakthrough performance and scalability.
* Drive team design review processes, ensuring both rapid delivery and a high quality bar.
* Lead technical/product discussions with other Software and Product managers.
* Interface with a diverse customer base to understand requirements, priorities, and processes.* Hire, coach, and mentor query optimization SDEs.
* Interact and partner with the open source community.
* Propose and implement new projects or recommend engine improvements.
* Own all operational metrics and support.
* Help develop long-term development and business technology strategies.

- 3+ years of engineering team management experience
- 7+ years of working directly within engineering teams experience
- 3+ years of designing or architecting (design patterns, reliability and scaling) of new and existing systems experience
- Knowledge of engineering practices and patterns for the full software/hardware/networks development life cycle, including coding standards, code reviews, source control management, build processes, testing, certification, and livesite operations
- Experience partnering with product or program management teams

- Experience in communicating with users, other technical teams, and senior leadership to collect requirements, describe software product features, technical designs, and product strategy
- Experience in recruiting, hiring, mentoring/coaching and managing teams of Software Engineers to improve their skills, and make them more effective, product software engineers
- Masters degree in Computer Science
- 9+ years of experience in Software development
- 5+ years as a Development Manager managing teams of 10 engineers
- Deep understanding of big data processing
- Experience with query optimizations (plans, execution, resource utilization)
- Experience with SQL and No SQL data stores
- Experience in development of distributed/scalable systems and high-volume transaction applications
- Proven track record of building and managing high performing engineering teams
- Strong business and technical vision
- Ability to handle multiple competing priorities in a fast-paced environment
- Experience with Linux/Unix systems