Job responsibilities
- Integrate data from various firm sources into big data warehouse
- Investigate data issues, provide support on data issues.
- Develop automation for data extraction.
- Design and tune schema for data landed on platform
- Partner with information modelling teams on firm wide logical data models.
- Serve as the primary subject matter expert (SME) for data in the analytics platform
- Develop data quality rules and controls for data
- Analyze and solve query performance bottlenecks in Cloud based warehouses like Redshift and AWS Glue
Required qualifications, capabilities, and skills
- Formal training or certification on software engineering concepts and 5+ years applied experience
- Experience in big data technologies - Apache Spark, Hadoop and analytics.
- Hands on coding experience in Java/Python
- Experience in designing & developing using Redshift
- Strong CS fundamentals, data structures, algorithms with good understanding of big data
- Experience with AWS application development including services such as Lambda, Glue, ECS/EKS
- Excellent communication skills are a must for this position
- Experience with Unix/Linux and shell scripting, Redshift, Hive.
Preferred qualifications, capabilities, and skills
- Good understanding of data modelling challenges with big data
- Good understanding of Financial data especially in front office investment banking is a major plus
- Ability to code in Apache Spark using Scala is an added advantage