JPMorgan Lead Data Engineer Consumption Access SD - Chase UK 

United Kingdom, England, London 

420559575

Job responsibilities

• Architect and develop scalable data pipelines on cloud infrastructure.
• Collaborate in an agile, customer-facing environment to deliver high-quality solutions.
• Utilize your expertise in Python and SQL to develop and optimize data processes.
• Design and manage relational databases and complex data structures.
• Deploy and manage containerized applications using Docker and Kubernetes.
• Develop Infrastructure as Code (IaC) with Terraform and Terragrunt.
• Engage in all stages of the software development lifecycle, from design to support.
• Utilize scheduling systems, with a preference for Airflow, to manage workflows.
• Automate deployment, releases, and testing in CI/CD pipelines.
• Write and automate unit, component, integration, and end-to-end tests.

Required Qualifications, Capabilities, and Skills:

• Formal training or certification on Python and SQL concepts and proficient advanced experience
• Extensive hands-on experience in Python and SQL (any dialect).
• Experience with distributed systems and cloud technologies (AWS, GCP, Azure, etc.); EMR.
• Hands-on experience in relational databases, data structures, caching concepts, race conditions, and complex analytical queries.
• Experience with Infrastructure as Code (IaC) using Terraform and Terragrunt.
• Experience with scheduling systems, with Airflow.
• Basic understanding of data streaming and messaging frameworks (Kafka, Spark Structured Streaming, Flink, etc.).
• Good knowledge of the Spark framework and its deployment with cloud services.

Preferred qualifications, capabilities and skills

• Academic qualification in computer science or a STEM-related field, or foreign equivalent.
• At least 3 years of hands-on experience as a back-end software engineer or data engineer.
• Familiarity with table formats such as Iceberg.
• Experience with EMR and Spark is a plus.
• Understanding of RESTful services.