We’re looking for Senior – Data Engineering Databricks Pyspark, ETL development, Data pipeline creation and Big data processing, RDBMS management.
Primary skills and key responsibilities
- Mandatory: Hands on experience in Azure Data factory, Azure Databricks, PySpark, ETL, SQL
- Strong programming, unit testing & debugging skill in SQL, Python/Scala
- Work with the business team to gather requirements. Identify source data systems, develop pipelines to process data from source to target.
- Design and build reusable components, frameworks, and libraries at scale to support analytics products.
- Design and implement product features in collaboration with business and technology stakeholders.
- Identify and solve issues concerning data management to improve data quality.
- Clean, prepare and optimize data for ingestion and consumption.
- Collaborate on the implementation of new data management projects and re-structure of the current data architecture.
- Implement automated workflows and routines using scheduling tools.
- Build continuous integration, test driven development and product deployment frameworks.
- Troubleshoot the data issues and perform root cause analysis to proactively resolve operational issues.
- Stay abreast on technology advancements and ensure keeps updated on skills.
Nice to have skills
- 4+ years in a data engineer role using service and hosting solutions.
- such as private/public cloud PaaS platforms.
- Good experience in Azure Databricks, Data factory with development expertise on batch and real-time data integration.
- Implementation, and operations of OLTP, OLAP, DW technologies such as Azure SQL, Azure SQL DW, , Azure Data Lake Store , Azure Data Factory and understanding of Microsoft Azure PaaS features.
- Design and configuration of data movement, streaming and transformation (ETL) technologies such as Azure Data Factory, Azure Data Bricks
To qualify for the role, you must have
- BE/BTech/MCA/MBA
- Minimum 3 years hand-on experience in one or more relevant areas.
- Total of 6-8 years industry experience
Ideally, you’ll also have
- Experience on Banking and Capital Markets domains
Skills and attributes for success
- Use an issue-based approach to deliver growth, market and portfolio strategy engagements for corporates
- Strong communication, presentation and team building skills and experience in producing high quality reports, papers, and presentations.
- Experience in executing and managing research and analysis of companies and markets, preferably from a commercial due diligence standpoint.
What we look for
- A Team of people with commercial acumen, technical experience and enthusiasm to learn new things in this fast-moving environment
- An opportunity to be a part of market-leading, multi-disciplinary team of 1400 + professionals, in the only integrated global transaction business worldwide.
- Opportunities to work with EY Advisory practices globally with leading businesses across a range of industries
At EY, we’re dedicated to helping our clients, from start–ups to Fortune 500 companies — and the work we do with them is as varied as they are.
You get to work with inspiring and meaningful projects. Our focus is education and coaching alongside practical experience to ensure your personal development. We value our employees and you will be able to control your own development with an individual progression plan. You will quickly grow into a responsible role with challenging and stimulating assignments. Moreover, you will be part of an interdisciplinary environment that emphasizes high quality and knowledge exchange. Plus, we offer:
- Support, coaching and feedback from some of the most engaging colleagues around
- Opportunities to develop new skills and progress your career
- The freedom and flexibility to handle your role in a way that’s right for you
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.