About the role
Role- Big data engineer (Spark, Scala, AWS)
We are looking for a Data Engineer with strong experience in designing, developing, and maintaining scalable, secure, and high-performance ETL/data processing pipelines. The ideal candidate should have hands-on expertise in Apache Spark, Scala, AWS EMR, and Amazon S3, along with a strong understanding of big data technologies and cloud-based data platforms.
Key Responsibilities
- Design, develop, and maintain scalable batch and big data processing pipelines using Apache Spark and Scala.
- Build, deploy, and manage large-scale data processing solutions on AWS EMR.
- Utilize Amazon S3 as the primary data lake and storage layer for ingesting, storing, and processing structured and unstructured data.
- Optimize Spark applications for performance, scalability, reliability, and cost efficiency.
- Develop reusable frameworks and components for data ingestion, transformation, validation, and processing.
- Collaborate with data architects, business analysts, application teams, and stakeholders to understand data requirements and deliver robust data solutions.
- Ensure data quality, integrity, security, and governance across all data pipelines.
- Monitor, troubleshoot, and resolve production issues related to data workflows and processing jobs.
- Support deployment, release management, and ongoing maintenance of data platforms.
- Follow best practices related to coding standards, testing, documentation, version control, and CI/CD processes.
Required Skills & Qualifications
- Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline.
- Strong hands-on experience in Scala programming.
- Extensive experience with Apache Spark for large-scale distributed data processing.
- Proven experience working with AWS EMR for big data workloads.
- Strong knowledge of Amazon S3 and modern data lake architectures.
- Solid understanding of ETL/ELT design principles and data pipeline development.
- Strong SQL skills and experience working with large and complex datasets.
- Good understanding of cloud-based architecture, performance tuning, job monitoring, and operational support.
- Experience with version control systems such as Git/GitHub.
- Excellent analytical, problem-solving, communication, and stakeholder management skills.
Millions of jobs, with real people getting hired every day
Questions, answered
Click "Apply with JobAssist" – we tailor your resume and application to this role and submit it for your approval.
Yes. This role at Coforge was screened before publishing – we confirmed the employer before listing it.
The employer didn't disclose a salary range for this listing. JobAssist shows pay whenever it's available.
This position can be done from anywhere, with no in-office requirement.
Yes – every application is tailored from your profile and this job's requirements, and you can review and edit before it's sent.