About the role
We are looking for an experienced Python Data Engineer with strong expertise in Python, PySpark, SQL, ETL/ELT, and Cloud Data Platforms. The ideal candidate will design, build, and optimize scalable data pipelines, support modern data architectures, and deliver enterprise-grade data solutions across cloud environments.
The role requires strong hands-on data engineering experience, excellent problem-solving skills, and the ability to work closely with business and technology stakeholders in a customer-facing environment.
Key Responsibilities
- Design, develop, and optimize scalable data pipelines using Python and PySpark.
- Build ETL/ELT frameworks for ingesting, transforming, and processing large datasets.
- Develop data ingestion solutions from multiple structured and semi-structured data sources.
- Implement data transformation, cleansing, validation, and quality checks.
- Work with cloud-based data platforms to build modern data engineering solutions.
- Collaborate with technology, architecture, product, and design teams to deliver scalable data solutions.
- Design and maintain data models supporting analytics and reporting requirements.
- Optimize data pipeline performance, scalability, and reliability.
- Ensure data quality and governance standards are maintained.
- Support Data Warehousing and enterprise analytics initiatives.
- Participate in Agile ceremonies and contribute to continuous improvement initiatives.
- Work closely with customers and stakeholders to understand business requirements and translate them into technical solutions.
Required Skills
-
Strong hands-on experience with Python for data processing and transformation.
-
Expertise in PySpark and distributed data processing frameworks.
-
Strong SQL skills, including:
-
Complex joins
-
Aggregations
-
Query optimization
-
Data transformation
-
Experience building ETL/ELT pipelines and data ingestion frameworks.
-
Hands-on experience with cloud platforms:
-
Azure
-
AWS
-
GCP
-
Strong understanding of:
-
Data Warehousing
-
Dimensional Modeling
-
Data Modeling Concepts
-
Experience handling structured and semi-structured data.
Preferred Skills
- Azure Data Factory (ADF).
- Databricks.
- Azure Synapse Analytics.
- Kafka.
- Azure Event Hubs.
- CI/CD implementation for Data Engineering workflows.
- Agile methodologies and JIRA.
- BFSI, Insurance, or Investment domain experience.
- LOMA certification or similar industry certifications.
Desired Candidate Profile
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management abilities.
- Strong ownership mindset and accountability.
- Ability to work effectively in customer-facing environments.
- Experience working in Agile and cross-functional teams.
- Passion for building scalable and reliable data platforms.
Millions of jobs, with real people getting hired every day
Questions, answered
Click "Apply with JobAssist" – we tailor your resume and application to this role and submit it for your approval.
Yes. This role at Tata Consultancy Services was screened before publishing – we confirmed the employer before listing it.
The employer didn't disclose a salary range for this listing. JobAssist shows pay whenever it's available.
This position can be done from anywhere, with no in-office requirement.
Yes – every application is tailored from your profile and this job's requirements, and you can review and edit before it's sent.