Senior Software Engineer (Streaming and Durability)

Every transaction, every listing view, every agent interaction at Compass touches a database, a cache, or a streaming pipeline owned by the Streaming & Durability team

This team is responsible for the foundational data layer: the systems that have to be fast, correct, and isolated across thousands of tenants, every single time

We don’t just keep the lights on. We’re solving hard, unsolved problems: how do you enforce tenant-level data isolation across a fleet of shared PostgreSQL clusters and Redis/Valkey caches without sacrificing latency? How do you upgrade 50+ production databases with zero downtime while application teams ship features on top of them? How do you build SDKs that make the safe path the easy path for hundreds of engineers who never think about infrastructure?

As a Senior Engineer, you’ll own answers to those questions

You’ll design systems that quietly protect millions of records, ship tooling that engineers across the company reach for by default, and operate infrastructure where the cost of getting it wrong is measured in trust, not just latency

Design and operate the database and caching fleet (RDS/Aurora PostgreSQL, ElastiCache Redis/ValKey, DynamoDB) that underpins Compass’s core products, making decisions about replication, failover, capacity, and engine strategy that affect every team in the company

Build multi-tenant data isolation into Compass’s data layer from the ground up, shipping SDKs in Go, Java, and Python that enforce tenant-safe access patterns so product engineers can move fast without risking cross-tenant data exposure

Own production database upgrades at fleet scale, coordinating engine version bumps and Terraform state mutations across hundreds of clusters with zero customer-facing impact

Build and scale Kafka/MSK streaming infrastructure, the connective tissue between Compass services, including topic architecture, consumer patterns, and blue/green deployment strategies

Develop deep observability into the data layer, including SLI dashboards, tenant-dimensioned telemetry, and real-time performance insights that let the team (and the company) understand system behavior before customers do

Partner directly with product and platform teams to assess database readiness for scaling events, migrate workloads onto golden-path infrastructure, and resolve the hardest production incidents in the stack

You think in trade-offs, not absolutes. You can articulate why you’d choose a soft application-enforced boundary over a hard infrastructure lock and when that calculus changes

You’ve led cross-team operational programs, including change management, migration coordination, and rollback planning, where the blast radius of a mistake extends well beyond your own team

5+ years of experience building and operating distributed systems, with depth in at least one major managed database (PostgreSQL or similar) at production scale in AWS

You’ve built shared tooling (SDKs, client libraries, platform APIs), and you understand what it takes to make infrastructure accessible and safe for engineers who aren’t infrastructure specialists

You write Go (or can ramp quickly) and you’re comfortable in Terraform, Kubernetes, and IAM-heavy AWS environments

You’ve operated Kafka/MSK at scale and dealt with the hard parts: consumer lag, partition rebalancing, cross-account IAM auth

You’ve worked on multi-tenant isolation problems such as key prefixing, row-level security, and RBAC/ACL models and have opinions about where application-layer enforcement breaks down

You’ve built or contributed to observability platforms, specifically for database and cache monitoring