JPMorgan PySpark / AWS Databricks Lead Software Engineer 

United States, Ohio 

560900519

Job responsibilities

• Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems
• Develops secure high-quality production code, and reviews and debugs code written by others
• Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems
• Leads evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability for use within existing systems and information architecture
• Leads communities of practice across Software Engineering to drive awareness and use of new and leading-edge technologies
• Work with business stakeholders to understand requirements and design appropriate solutions, producing architecture and design artifacts for complex applications
• Implement robust monitoring and alerting systems to proactively identify and address data ingestion issues, optimizing performance and throughput
• Implement data quality checks and validation processes to ensure accuracy and reliability of data
• Design and implement scalable data frameworks to manage end-to-end data pipelines for workforce data analytics
• Share and develop best practices with Platform and Architecture teams to improve data pipeline framework and modernize the workforce data analytics platform
• Gather, analyze, and synthesize large, diverse data sets to continuously improve capabilities and user experiences, leveraging data-driven insights

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Hands-on practical experience delivering system design, application development, testing, and operational stability
• Proficiency in automation and continuous delivery methods
• Proficient in all aspects of the Software Development Life Cycle
• Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security
• Demonstrated proficiency in software applications and technical processes within a technical discipline (e.g., cloud, artificial intelligence, machine learning, mobile, etc.)
• In-depth knowledge of the financial services industry and their IT systems
• Advanced proficiency in data processing frameworks and tools (Parquet, Iceberg, PySpark, Glue, Lambda, AWS, EMR, Athena, and Redshift)
• Proficiency in programming languages (7+ years) such as Python and Scala for data processing and application development
• Experience with scheduling tools (Autosys or Airflow) to automate and manage job scheduling for efficient workflow execution

• Proficiency in Relational Databases (Oracle or SQL Server)
• Skilled in writing Oracle SQL queries utilizing (DML, DDL and PL/SQL)
• AWS Certification
• Databricks knowledge