About This Role
This Data Engineer role involves designing, building, and optimizing scalable data pipelines and data platforms. The position suits a highly skilled engineer with strong expertise in data modeling, cloud-based data architectures, and modern data engineering tools across Azure, Snowflake, and Databricks environments.
Responsibilities
- Design, develop, and maintain robust ETL/ELT pipelines using Databricks, PySpark, and Azure Data Factory (ADF)
- Build scalable and efficient data ingestion frameworks for structured and unstructured data
- Optimize pipeline performance through performance tuning and orchestration best practices
- Develop and maintain data models using modern tools (DBT preferred)
- Implement Master Data Management (MDM) solutions to ensure data consistency and integrity
- Design scalable and efficient Snowflake schemas (star/snowflake schema, dimensional modeling)
- Write and optimize advanced SQL queries across Snowflake, Azure SQL, and Synapse
- Develop and manage stored procedures and database objects
- Ensure efficient data retrieval through indexing, partitioning, and query optimization
- Work with Azure data services including Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL Database
- Integrate and maintain Snowflake with Azure ecosystem
- Develop data transformation and automation scripts using Python libraries: pandas, pyodbc, SQLAlchemy
- Build reusable components for data processing and validation
- Implement data validation rules, quality checks, and anomaly detection frameworks
- Perform root cause analysis for data inconsistencies
- Develop dashboards or tools for data quality monitoring
- Use GitHub for version control, branching strategies, and code reviews
- Manage workload scheduling and dependency management for pipelines
- Collaborate with cross-functional teams including data analysts, data scientists, and business stakeholders
Requirements
- Bachelor of Engineering, Bachelor Of Technology, Master Of Technology, or Master Of Engineering
- Bachelors or Masters degree in Computer Science, Information Systems, or related field
- Strong experience in data engineering and data platform development
- Expertise in DBT (preferred) for data modeling
- Strong SQL skills with hands-on experience in Snowflake, Azure SQL, and stored procedures
- Proficiency in Python for data engineering workflows
- Hands-on experience with Databricks PySpark and Azure Data Services (ADF, ADLS, Synapse)
- Strong knowledge of Snowflake architecture and schema design
