Responsibilities:
- Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
- Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
- Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
- Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
- Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
- Ensure essential procedures are followed and help define operating standards and processes
- Serve as advisor or coach to new or lower level analysts
- Has the ability to operate with a limited level of direct supervision.
- Can exercise independence of judgement and autonomy.
- Acts as SME to senior stakeholders and /or other team members.
- Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
Qualifications:
- 5-8 years of relevant experience
- Experience in systems analysis and programming of software applications
- Experience in managing and implementing successful projects
- Working knowledge of consulting/project management techniques/methods
- Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
Education:
- Bachelor’s degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
ETL Developer will be responsible for designing, implementing, and optimizing distributed data processing jobs to handle large-scale data in Hadoop Distributed File System(HDFS) using Apache Spark and Python. This role required deep understanding of data engineering principles, proficiency in Python and hands-on experience with Spark and Hadoop ecosystems. Developer will collaborate with data engineers, analysts, and business stakeholders to process, transform and drive insights and data driven decisions.
Key Responsibilities:
Data Processing and Transformation:
Design and Implement of Spark applications to process and transform large datasets in HDFS.
Develop ETL Pipelines in Spark using Python for data Ingestion, cleaning, aggregation, and transformations.
Performance Optimization:
Optimize Spark jobs for efficiency, reducing run time and resource usage.
Finetune memory management, caching, and partitioning strategies for Optimal performance
Data Engineering with Hadoop and Spark:
Load data from different sources into HDFS, ensuring data accuracy and integrity.
Integrate Spark Applications with Hadoop frameworks like Hive, Sqoop etc.
Testing and debugging:
Full timeIrving Texas United States$107,120.00 - $160,680.00
Anticipated Posting Close Date:
Nov 20, 2024View the " " poster. View the .
View the .
View the