Job summary
You should be expert at implementing and operating stable, scalable data flow solutions from production systems into end-user facing applications/reports. These solutions will be fault tolerant, self-healing and adaptive. You will be working on developing solutions that provide some of the unique challenges of space, size and speed. You will implement data analytics using cutting edge analytics patterns and technologies that are inclusive of but not limited to various AWS Offerings - EMR, Lambda, Kinesis, and Spectrum. You will extract huge volumes of structured and unstructured data from various sources (Relational /Non-relational/No-SQL database) and message streams and construct complex analyses. You will write scalable code and tune performance running over billion of rows of data. You will implement data flow solutions that process data on Spark, Redshift and store in both Redshift and File based storage (S3) for reporting and adhoc analysis.You should be detail-oriented and must have an aptitude for solving unstructured problems. You should work in a self-directed environment, own tasks and drive them to completion.

• Bachelor’s degree in Computer Science, Mathematics, Statistics, Finance or related technical field.
• 3+ years of industry experience in software development, data engineering, business intelligence, data science, or related field with a track record of manipulating, processing, and extracting value from large datasets
• Experience with data modeling, data warehousing, and building ETL pipelines
• Experience and advanced knowledge of SQL
• Experience using big data technologies (Hadoop, Hive, Hbase, Spark, EMR, etc.)
• Experience with coding languages like Python/Java/Scala
• Exposure to large databases, BI applications, data quality and performance tuning

• Experience partnering with business owners directly to understand their requirements and provide data which can help them observe patterns and spot anomalies
• Experience with web technology to develop dashboards
• Practical Knowledge of Linux or Unix shell scripting
• Experience in processing large volume of data
• Experience providing technical leadership and mentoring other engineers for best practices on data engineering
• Knowledge of software engineering best practices across the development life cycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations