Engineering Manager - Streaming

About This Role

This engineering leadership role focuses on spearheading Spark™ Structured Streaming development initiatives, covering both open-source and Databricks-specific components. The primary mission is to advance Spark™ Structured Streaming into a state-of-the-art stream processing engine by adding advanced features like sophisticated state management and new operators, while also enhancing engine performance in terms of latency and throughput through architectural reimagination. The role involves leading a talented engineering team, overseeing recruitment, and implementing robust processes to align with organizational goals.

Responsibilities

• Lead a talented engineering team in Spark™ Structured Streaming, developing and promoting the engine in OSS and the Databricks Data Intelligence Platform
• Oversee sustained recruitment of top-tier talent, and upskill talent on the team
• Implement robust processes to efficiently execute product vision, strategy, and roadmap in alignment with organizational goals and priorities
• Build software that is not just high quality but easy to operate
• Make company-wide impact by driving Stream Processing adoption across the Databricks product portfolio
• Manage technical debt, including long-term technical architecture decisions and balance product roadmap

Requirements

• 5+ years experience working in a related system, streaming, query processing, query optimization, including big-data ecosystem, Apache Spark™ or database internal
• A passion for database systems, storage systems, distributed systems, language design, or performance optimization
• Can ensure the team builds high quality and reliable infrastructure services
• Experience being responsible for testing, quality, and SLAs of a product
• Previous experience building and leading teams in a complex technical domain, such as on distributed data systems or database internals
• Ability to attract, hire, and coach engineers who meet the Databricks hiring standards
• Can uplevel existing team via hiring top-notch senior talent, growing leaders and helping struggling members
• Can gain trust of the team and guide their careers
• Experience managing distributed teams preferred
• Comfortable working cross-functionally with product management and directly with customers
• Ability to deeply understand product and customer personas

Benefits

• Eligibility for annual performance bonus
• Equity

Important Details

• Local Pay Range: $181,000–$253,750 USD

About the Company

Databricks is the Data and AI company, with more than 20,000 organizations worldwide, including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and 70% of the Fortune 500, relying on its Data + AI Platform to build and scale data and AI apps, analytics and agents. Headquartered in San Francisco with 30+ offices globally, Databricks offers a unified platform that includes Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog.