About This Role
This ETL Tester role focuses on validating end-to-end data pipelines and developing automated test frameworks within a Databricks and cloud environment. The position is suitable for a professional with strong data testing and automation skills, ensuring data consistency and integrity.
Responsibilities
- Validate end-to-end data ingestion, transformation, and consumption pipelines across bronze, silver, and gold layers
- Perform source-to-target data validation for structured data
- Validate business rules and data transformations
- Ensure data consistency, completeness, uniqueness, and referential integrity
- Design and develop automated test frameworks for data validation and regression testing
- Automate ETL and data quality testing using Python, SQL, PySpark, or testing frameworks
- Build reusable validation scripts for schema validation, reconciliation, and data drift detection
- Integrate automated test suites into CI/CD pipelines
- Validate Databricks notebooks, workflows, Delta Tables, and job orchestration
- Test Delta Lake functionalities including ACID compliance, schema evolution, and time travel
- Validate performance and scalability of Spark-based transformations
Requirements
- SQL
- Python
- PySpark
- Databricks
- Batch Pipelines
- Incremental Loads
- CDC pipelines
- Knowledge of Medallion Architecture concepts
- 4-6 years of experience in QA/Data Testing
- 2+ years of experience in Databricks/Data Lakehouse projects
- Experience working in Agile/Scrum delivery models
Nice to have:
- Certification in Databricks, Cloud, or a Testing tool
Important Details
- Work format: remote (India), with flexibility to work from an EXL office whenever business requires; not a permanent work-from-home position
