Role and Responsibilities
- Implement, maintain and evolve cloud-based products and services
- Responsible for designing, implementing and maintaining ETL processes for Analytics and Business Intelligence Systems
- Perform extensive research and analysis to make optimal architecture and design decisions
- Analyze data warehouses and suggest optimizations to SQL Queries used for Business Intelligence reporting
- Modify structured and unstructured data to provide meaningful reports to business
- Write large amounts of code, perform code reviews, write unit tests
- Write documentation
- Interface with other groups including Product Management, QA and Operations
- Create quick proof-of-concept prototypes
- Participate in an agile team
Skills and Qualifications
- BS or MS in Computer Science or equivalent experience
- Working experience in software development using Java, Scala, or Python is a must
- Extensive experience in implementing scalable and reliable ETL pipelines on Cloud Platforms using Big Data Technologies such as Apache Spark, Hadoop, AWS Glue
- Experience transforming data in various formats (JSON, CSV, PARQUET, YAML, Avro, Orc)
- Experience managing and scheduling workflows (Apache Airflow experience is preferred)
- Experience with working on Cloud Data Warehouses such as Amazon Redshift, Google BigQuery, and Data Lakes/Delta Lakes
- Experience with stream processing technology like Kafka, and Spark Streaming
- Ability to analyze and simplify complex code and algorithms in Java, Scala or Python
- Ability to adapt and learn different technologies that best solves the problem regardless of whether it is written in Python, Java, Scala, etc.
- Ability to perform ad-hoc complex SQL queries and query optimization
- Team player with strong communication skills
- Desire to learn fast and pick up latest technologies and tools
- Experience with data visualization software (Tableau preferred) is a plus
- Experience with Kubernetes and Machine Learning is a plus