Expoint - all jobs in one place

מציאת משרת הייטק בחברות הטובות ביותר מעולם לא הייתה קלה יותר

Limitless High-tech career opportunities - Expoint

JPMorgan Lead Data Engineer 
India, Karnataka, Bengaluru 
421096810

15.04.2025

As a Software Engineer III in Data Engineering, you will be at the forefront of architecting, building, deploying, and maintaining a cloud-native, web-scale data platform. Collaborate with a dynamic team to deliver robust, scalable, and sustainable data pipelines, ensuring seamless integration and optimal performance.

Key Responsibilities:

  • Architect and develop scalable data pipelines on cloud infrastructure.
  • Collaborate in an agile, customer-facing environment to deliver high-quality solutions.
  • Utilize your expertise in Python and SQL to develop and optimize data processes.
  • Design and manage relational databases and complex data structures.
  • Deploy and manage containerized applications using Docker and Kubernetes.
  • Develop Infrastructure as Code (IaC) with Terraform and Terragrunt.
  • Engage in all stages of the software development lifecycle, from design to support.
  • Utilize scheduling systems, with a preference for Airflow, to manage workflows.
  • Automate deployment, releases, and testing in CI/CD pipelines.
  • Write and automate unit, component, integration, and end-to-end tests.

Required Qualifications, Capabilities, and Skills:

  • Extensive hands-on experience in Python and SQL (any dialect).
  • Experience with distributed systems and cloud technologies (AWS, GCP, Azure, etc.); EMR is a plus.
  • Hands-on experience in relational databases, data structures, caching concepts, race conditions, and complex analytical queries.
  • Experience with Infrastructure as Code (IaC) using Terraform and Terragrunt.
  • Experience with scheduling systems, with Airflow preferred.
  • Basic understanding of data streaming and messaging frameworks (Kafka, Spark Structured Streaming, Flink, etc.).
  • Good knowledge of the Spark framework and its deployment with cloud services.

Preferred Qualifications, Capabilities and Skills:

  • Academic qualification in computer science or a STEM-related field, or foreign equivalent.
  • At least 3 years of hands-on experience as a back-end software engineer or data engineer.
  • Familiarity with table formats such as Iceberg.
  • Experience with EMR and Spark is a plus.
  • Understanding of RESTful services.