MSD Manager Databricks Tech 

India, Telangana, Hyderabad 

558453600

What will you do in this role:

• Develop and implement data engineering strategies and data mesh architectures on Databricks

• Design, build, and maintain ETL/ELT pipelines usingPySparkand other data engineering tools.

• Collaborate with cross-functional teams to ensure data governance, security, and compliance.

• Stay up to date with the latest Databricks features and advancements, implementing them into our data engineering practices.

• Leverage the toolsetutilizedbyvarious typesof Data Engineers, ensuring seamless collaboration with cross-functional data teams across the organization.

• Drive and manage the team of Data engineers to ensure the best performance during the project life cycle.

• Design, develop, andmaintaindata pipelines to extract data from various sources and populate a data lake and data warehouse.

• identifyand transform data for ingestion, exploration, and modeling.

• maintaindata catalogs.

• Use data orchestration, logging, and monitoring tools to build resilient pipelines.

• Use test-driven developmentmethodologywhen building ELT/ETL pipelines

• Understand and apply concepts like a data lake, data warehouse, lake-house, data mesh, and data fabric where relevant

• Develop data models for cloud data warehouses

• Develop pipelines to ingest data into cloud data warehouses

• Lead in an agile engineering environment, providing guidance on complex data and unplanned data challenges.

• Collaborate with peers in the "Data Integration" and "Central Data and Analytics" teams to provide feedback about the toolset and help define product strategies.

• objectivesand value delivery strategies.

• Participate in negotiations with vendors to secure favorable terms and conditions.

• Retain and attract diverse, exceptional talent, fostering an inclusive environment.

What Should you have:

• Minimum of 8-10 years in senior developer role of developing data pipelines & data infrastructure, ideally within a drug development or life sciences context.

• Working on large data sets and handling raw SQL

• Experience using technologies such asPyspark/AWS/Databricks

• Experience creating ETL Pipeline

• Experience leading teams and managing projects.

• Strong understanding of data governance, compliance, and security best practices.

• Excellent communication and problem-solving skills, with the ability to manage multiple priorities

• expertisein delivering large-scale information management technology solutions encompassing data integration and self-service analytics enablement.

• Expert in software/data engineering practices (including versioning, release management, deployment of datasets, agile & related software tools).

• Strong working knowledge of at least one large-scale data processing technology (e.g., High-performance computing, distributed computing), databases, and underlyingtechnology (cloud or on-prem environments, containerization, distributed storage & databases)

• Cloud-native, ideally AWS certified.

• Strong interpersonal and communication skills (verbal and written), effectively bridging scientific and business needs; experience working in a matrix environment.

• Proven record of delivering high-quality results

• Product and customer-centric approach.

• Innovative thinking, experimental mindset.