About This Role
This ETL Developer role focuses on designing, developing, and maintaining ETL processes to move and transform data from various sources into target systems. The position involves optimizing data integration workflows and building, scheduling, and monitoring data pipelines using key technologies.
Responsibilities
- Design, develop, and maintain ETL processes to extract, transform, and load data from various sources into target systems
- Develop and optimize data integration workflows using Ab Initio, Python, and Apache Spark
- Build, schedule, and monitor data pipelines using Apache Airflow
- Perform data transformation, cleansing, validation, and reconciliation to ensure data accuracy and consistency
- Optimize ETL workflows and data processing performance to handle large-scale datasets efficiently
- Troubleshoot data pipeline issues and implement solutions to improve reliability and performance
- Collaborate with data engineers, business analysts, and other stakeholders to support data integration and reporting requirements
- Maintain technical documentation for ETL processes, data pipelines, and workflows
Requirements
- Bachelor's degree in Computer Science, Information Technology, Information Systems, or a related field
- 4–6 years of experience as an ETL Developer, Data Engineer, or in a similar role
- Hands-on experience with Ab Initio for ETL development and data integration
- Proficiency in Python for data processing, scripting, and automation
- Experience with Apache Spark for distributed data processing and large-scale data transformation
- Experience with Apache Airflow for workflow orchestration, scheduling, and monitoring
- Strong understanding of ETL concepts, data warehousing, data integration, and data quality management
- Experience working with SQL and relational databases
- Strong analytical, problem-solving, and troubleshooting skills
- Good communication skills and the ability to collaborate effectively within a team
