JPMorgan Lead Software Engineer - Databricks PySpark AWS 

United States, New Jersey, Jersey City 

224740116

Job responsibilities:
- Designs and delivers trusted data collection, storage, access and data platform solutions in a secure, stable, and scalable way.
- Provides recommendations and insight on data management, governance procedures, and intricacies applicable to the acquisition, maintenance, validation, and utilization of data.
- Develops and implements data federation strategies using APIs, Data share, messaging services
- Utilizes ANSI SQL to enable efficient data aggregation and manipulation across multiple data sources, enhancing analytical capabilities and decision-making processes.
- Designs and maintains robust software solutions for authorization and key management, ensuring secure connectivity to both private and public cloud environments.
- Builds and manages container images for custom services and applications, as well as third-party open-source applications, to support scalable and reliable deployment in cloud environments.
- Evaluates and reports on access control processes to determine effectiveness of data asset security with minimal supervision.- Develop solution in well architected framework addressing scalability, resiliency, cost, operational excellence.

Required qualifications, capabilities, and skills

Formal training or certification on data and software engineering concepts and 5+ years applied experience Strong experience with both relational ( Aurora) and NoSQL databases​.
Experience and proficiency across the data lifecycle.
Experience in building and deploying applications within containerized environments utilizing tools such as Kubernetes , Docker and ECS
Strong proficiency in programming languages such as Python, JAVA and SQL.
In-depth knowledge of OLTP , data warehousing solutions and ETL processes.
Ability to work collaboratively with cross-functional teams, including data scientists, analysts, and business stakeholders.
Excellent communication skills, with the ability to convey complex technical concepts to non-technical audiences.
Experience with version control systems like Git and CI/CD pipelines for data engineering workflows.
Solid understanding of the Parquet, ORC and AVRO file formats.
Experience in ETL tools ( Glue , data bricks)

Preferred qualifications, capabilities, and skills

Proficiency in Big Data technologies, with a strong focus on Performance Optimization using best practices.
Solid understanding of open table formats particularly iceberg.
Experience in building, maintaining and optimizing iceberg tables.
Proficiency in building and maintaining efficient container images,
Experience with orchestration tools such as Airflow, AWS step , Event bridge