Finding the best job has never been easier
Share
US Consulting - AI & Data - Data Architect, Industrials & Energy Sector – Senior
In this role, you will create, maintain, and support the data platform and infrastructure that enables the analytics front-end; this includes the testing, maintenance, construction, and development of architectures such as high-volume, large-scale data processing and databases with proper verification and validation processes.
Your key responsibilities
Design, develop, optimize, and maintain data architecture and pipelines that adheres to ETL principles and business goals
Develop and maintain scalable data pipelines, build out new integrations using AWS native technologies to support continuing increases in data source, volume, and complexity
Define data requirements, gather and mine large scale of structured and unstructured data, and validate data by running various data tools in the Big Data Environment
Support standardization, customization and ad hoc data analysis and develop the mechanisms to ingest, analyse, validate, normalize, and clean data
Write unit/integration/performance test scripts and perform data analysis required to troubleshoot data related issues and assist in the resolution of data issues
Implement processes and systems to drive data reconciliation and monitor data quality, ensuring production data is always accurate and available for key stakeholders, downstream systems, and business processes
Lead the evaluation, implementation and deployment of emerging tools and processes for analytic data engineering to improve productivity
Develop and deliver communication and education plans on analytic data engineering capabilities, standards, and processes
Learn about machine learning, data science, computer vision, artificial intelligence, statistics, and/or applied mathematics
Solve complex data problems to deliver insights that help achieve business objectives
Implement statistical data quality procedures on new data sources by applying rigorous iterative data analytics
Strong understanding & familiarity with all Hadoop Ecosystem components and Hadoop Administrative Fundamentals
Strong understanding of underlying Hadoop Architectural concepts and distributed computing paradigms
Skills and attributes for success
Experience in the development of Hadoop APIs and MapReduce jobs for large scale data processing
Hands-on programming experience in Apache Spark using SparkSQL and Spark Streaming or Apache Storm
Hands on experience with major components like Hive, Spark, and MapReduce
Experience working with NoSQL in at least one of the data stores - HBase, Cassandra, MongoDB
Experienced in Hadoop clustering and Auto scaling
Good knowledge in apache Kafka & Apache Flume
Knowledge of Spark and Kafka integration with multiple Spark jobs to consume messages from multiple Kafka partitions
Advanced experience and understanding of data/Big Data, data integration, data modelling, AWS, and cloud technologies
Strong business acumen with knowledge of the Industrial Products sector is preferred, but not required
Ability to build processes that support data transformation, workload management, data structures, dependency, and metadata
Ability to build and optimize queries (SQL), data sets, 'Big Data' pipelines, and architectures for structured and unstructured data
Experience with or knowledge of Agile Software Development methodologies.
Demonstrated understanding and experience using:
Data Engineering Programming Languages (i.e., Python)
Distributed Data Technologies (e.g., Pyspark)
Cloud platform deployment and tools (e.g., Kubernetes)
Relational SQL databases
DevOps and continuous integration
AWS cloud services and technologies (i.e., Lambda, S3, DMS, Step Functions, Event Bridge, Cloud Watch, RDS)
Databricks/ETL
IICS/DMS
GitHub
Event Bridge, Tidal
To qualify for the role you must have
Flexible and proactive/self-motivated working style with strong personal ownership of problem resolution
Excellent communicator (written and verbal formal and informal)
Ability to multi-task under pressure and work independently with minimal supervision.
Partner with Business Analytics and Solution Architects to develop technical architectures for strategic enterprise projects and initiatives
Coordinate with Data Scientists to understand data requirements, and design solutions that enable advanced analytics, machine learning, and predictive modelling
Support Data Scientists in data sourcing and preparation to visualize data and synthesize insights of commercial value
Collaborate with AI/ML engineers to create data products for analytics and data scientist team members to improve productivity
Advise, consult, mentor and coach other data and analytic professionals on data standards and practices, promoting the values of learning and growth
Foster a culture of sharing, re-use, design for scale stability, and operational efficiency of data and analytical solutions
Ideally, you’ll also have
Experience in leading and influencing teams, with a focus on mentorship and professional development
A passion for innovation and the strategic application of emerging technologies to solve real-world challenges
The ability to foster an inclusive environment that values diverse perspectives and empowers team members
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.
These jobs might be a good fit