About This Role
This engineering role focuses on building and managing secure, cost-efficient, and compliant cloud-based data platforms on AWS for a financial services environment. It involves extensive use of Infrastructure as Code, Linux administration, CI/CD pipeline engineering, and comprehensive monitoring to ensure high availability and reliability of data services.
Responsibilities
- Build and manage cloud-based data platforms on AWS (e.g., S3, Lambda, Glue, Redshift, EMR, IAM)
- Architect solutions that are secure, cost-efficient, and compliant with JPMorgan Chase cloud and data protection policies
- Provision, manage, and version all cloud infrastructure using Terraform
- Ensure infrastructure deployments are repeatable, auditable, and consistent across development, staging, and production environments
- Manage and maintain Linux-based environments that host data platforms and services
- Develop and maintain Shell Scripts (Bash/Ksh) for task automation, job scheduling, log management, file processing, and system administration
- Troubleshoot and optimize Linux-based systems to ensure stability, security, and performance across all environments
- Design and maintain end-to-end CI/CD pipelines for automated build, test, and deployment of data applications and infrastructure
- Utilize tools such as Jenkins, Spinnaker, GitHub Actions, or equivalent firmwide-approved platforms to enable continuous integration and delivery with minimal manual intervention
- Implement comprehensive monitoring, alerting, and observability solutions for data pipelines and infrastructure
- Leverage tools such as Splunk, Grafana, CloudWatch, Datadog, or equivalent to proactively detect, diagnose, and resolve issues, ensuring high availability and reliability of data services
Requirements
- Strong proficiency in Python for scripting, automation, and building production-grade applications
- Experience with Python libraries such as Pandas, Boto3, and pytest
- Solid experience with AWS services relevant to data engineering (S3, Glue, Lambda, IAM, etc.)
- Understanding of cloud security best practices
- Proficiency in writing, managing, and versioning Terraform modules for cloud infrastructure provisioning and management
- Experience building and maintaining CI/CD pipelines using Jenkins, Spinnaker, GitHub Actions, or similar tools
- Strong understanding of automated testing and deployment strategies
- Strong working knowledge of Linux operating systems (RHEL, CentOS, Ubuntu), including file systems, process management, networking, permissions, cron jobs, and system-level troubleshooting
- Proficiency in Bash/Ksh scripting for automating repetitive tasks, orchestrating data workflows, managing log rotation, file transfers, environment setup, and integrating with CI/CD and scheduling tools
