EY EY - GDS Consulting AI DATA Pyspark Senior 

India, Kerala, Kochi 

924415321

Responsibilities

• Create Spark jobs for data transformation and aggregation
• Produce unit tests for Spark transformations and helper methods
• Design data processing pipelines
• 3-4 years of exposure to big data stack

Skills

• Py (with a focus on the functional programming paradigm)
• Apache Spark 2.x
• Apache Spark RDD API
• Apache Spark SQL DataFrame API
• Apache Spark GraphX API
• Apache Spark Streaming API
• Spark query tuning and performance optimization
• Bachelor’s degree in computer science, Information Technology, or related field.
• Minimum 4-6 years of experience in designing, implementing, and managing Data engineering practices that includes all aspect of data warehousing , data quality , master data management.
• Minimum 3 years’ experience in Spark with either pyspark or scala and building data pipelines using any orchestration tools.
• Expertise in writing automations frameworks using any scripting language, preferably python/shell.
• Strong knowledge on SQL , PL/SQL is must.
• Optimizing spark sql code and job performance using spark configurations is strongly desirable/
• Understanding & knowledge of Data warehouse concepts and infrastructures.
• Hands-on experience with cloud platforms such as AWS, Azure.
• In-depth knowledge of CI/CD process
• Knowledge in Data acquisition, real time systems is desirable.

You get to work with inspiring and meaningful projects. Our focus is education and coaching alongside practical experience to ensure your personal development. We value our employees and you will be able to control your own development with an individual progression plan. You will quickly grow into a responsible role with challenging and stimulating assignments. Moreover, you will be part of an interdisciplinary environment that emphasizes high quality and knowledge exchange. Plus, we offer:

• Support, coaching and feedback from some of the most engaging colleagues around
• Opportunities to develop new skills and progress your career
• The freedom and flexibility to handle your role in a way that’s right for you

EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.