Software Engineer (Big Data) - Recommendation Ecosystem Infrastructure

About This Role

This engineering role focuses on designing and implementing storage solutions for offline data within a recommendation system that serves over a billion users. The position involves ensuring system reliability, uninterrupted service, and seamless performance, creating a scalable infrastructure for diverse data sources and storage needs.

Responsibilities

• Design and implement real-time (streaming computing) data systems for large-scale recommendation systems
• Create flexible, scalable, stable, and high-performance storage systems and computing models
• Troubleshoot production system failures, design and implement necessary mechanisms and tools to ensure overall stability of the production systems
• Construct industry-leading streaming computing frameworks and other distributed systems to provide reliable infrastructure for massive data and large-scale business systems
• Research, design, and develop computer and network software or specialised utility programs
• Analyse user needs and develop software solutions, applying principles and techniques of computer science, engineering, and mathematical analysis
• Update software, enhance existing software capabilities, and develop and direct software testing and validation procedures
• Work with computer hardware engineers to integrate hardware and software systems and develop specifications and performance requirements

Requirements

• Proficient in programming languages like Java, C++, Scala, Python
• Strong coding and troubleshooting skills
• At least 5 years of relevant experience
• Deep understanding of streaming computing systems, with formal production experience in developing TB-level Flink real-time computing systems
• Proficient in modules like FlinkDataStream, FlinkSQL, FlinkCheckpoint, FlinkState, and preferably with experience in reading Flink source code
• Familiar with at least one data lake technology such as Hudi, Iceberg, DeltaLake, and preferably with experience in reading their source code
• Willingness to tackle problems without clear answers, with a strong passion for learning new technologies

Nice to have:

• Experience in handling PB-level data
• Familiarity with other big data systems, including YARN, K8S, Spark, SparkSQL, Kudu, and others
• Experience in storage systems such as Hbase, Cassandra, RocksDB
• Experience in data lake development

About the Company

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Lemon8, CapCut and Pico, as well as platforms specific to the China market like Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content. ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives.