JPMorgan Lead Software Engineer - Big Data 

United States, Texas, Plano 

978561640

Job responsibilities:

• Promote to become data domain expert and drive to know everything about the data on the platform
• Create Functional and Technical Specifications, Epics and User Stories, Process Flows, Data Analysis, Mapping Documents, Implementation Plan, Agile artifacts
• Migrate from Hadoop to AWS using Pipelines, EMR
• Develop, enhance and test new/existing interfaces. The candidate will be part of existing agile team and will work on developing, enhancing ETL pipelines, design solutions,
• Identify, analyze, and interpret trends or patterns in complex data sets and transforming existing ETL logic into Hadoop Platform
• Innovate new ways of managing, transforming and validating data
• Establish and enforce guidelines to ensure consistency, quality and completeness of data assets Apply quality assurance best practices to all work products
• Experience of working in a development teams, using agile techniques and Object-Oriented development and scripting languages, is preferred.
• Adds to team culture of diversity, equity, inclusion, and respect

Required qualifications, capabilities, and skills:

• Formal training or certification on software engineering concepts and 5+ years of applied experience
• Knowledge of application, data, and infrastructure architecture disciplines
• Strong experience with documentation and structuring information in tools like Confluence and Jira
• Experience in SparkSQL,Imapala and Bigdata technologies
• Familiar with Data Science concepts and applying them to analyze large volumes of data
  Comfortable with data concepts; Oracle, Java, Python, Spark, Kafka, HDFS, AirFlow, Elastic Search.
• Working proficiency in SDLC CI/CD Execution (GitHub, Jenkins, SNOR, Spinnaker, AIM etc.)
• Minimum 2+ Experience with AWS Services like Lambda EC2
• Experience in real time streaming data
• Strong Experience with UNIX shell scripting is must
• Experience with relational database environment (Oracle, Teradata, SQL Server, etc.) leveraging databases, tables/views, stored procedures, agent jobs, etc. with strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy
• Handle Dev Ops effort in terms of CICD, Scanning, Code, Performance testing and Test overaged

• Experience in Athena, EMR, Redshift, Glue, Kinesis, Aurora,RDS,S3
• Knowledge in one or more modern programming languages like Java or Python is a plus
• AWS Cloud Practitioner certification a plus for applicant and expected upon joining team
• Experience working on AdTech/MarTech platforms is a plus