This engineering role focuses on building the next generation of AI infrastructure for TikTok's For You recommendation system and Large Language Models (LLMs). It involves developing core training and serving infrastructure for one of the world's largest recommendation systems, as well as next-generation infrastructure for foundation models and LLMs. The position is ideal for engineers passionate about AI systems, distributed computing, GPU optimization, and building high-performance infrastructure at massive scale.
Responsibilities
Design, develop, and optimize large-scale AI training and online inference infrastructure for recommendation models and LLMs
Drive the architecture and implementation of distributed training and serving systems with high scalability, reliability, and efficiency
Optimize end-to-end model performance across GPU computation, communication, memory, networking, and runtime systems
Develop and optimize LLM training and serving infrastructure, including model parallelism, KV Cache, Continuous Batching, and efficient inference
Work closely with researchers and algorithm engineers to productionize new model architectures and algorithms
Identify and resolve performance bottlenecks across the full AI stack, from model and framework to GPU kernels and distributed runtime
Improve system latency, throughput, GPU utilization, scalability, and infrastructure cost efficiency
Drive technical design, implementation, performance benchmarking, and production rollout of critical infrastructure components
Mentor junior engineers and contribute to the team's technical direction and engineering standards
Requirements
Bachelor's, Master's, or Ph.D. degree in Computer Science, Computer Engineering, or a related technical field, or equivalent practical experience
3+ years of software engineering, ML systems, distributed systems, or related industry experience
Strong programming skills in C++ or Python and experience building production-quality software
Strong understanding of data structures, algorithms, operating systems, distributed systems, or computer architecture
Hands-on experience with PyTorch, TensorFlow, or other machine learning frameworks
Experience designing, developing, or optimizing large-scale production systems
Strong problem-solving skills and the ability to independently drive complex technical projects
Nice to have:
Experience building infrastructure for large-scale recommendation systems, LLMs, or foundation models
Strong understanding of distributed training and model parallelism, including DP, TP, PP, FSDP, ZeRO, or related technologies
Hands-on experience with GPU programming and optimization, including CUDA, Triton, GPU kernels, or similar technologies
Millions of jobs, with real people getting hired every day
200,000+
New jobs added daily
7,000,000+
Verified job listings
1,000,000+
Tailored applications submitted
Experience with LLM inference and serving, including KV Cache, Continuous Batching, FlashAttention, CUDA Graph, speculative decoding, or related techniques
Experience optimizing GPU utilization, memory efficiency, communication performance, latency, and throughput at scale
Experience with distributed systems, high-performance computing, networking, or ML infrastructure
Experience designing and owning large-scale production systems from architecture through deployment and operation
Contributions to open-source projects, research publications, or technical projects in machine learning systems, distributed systems, GPU computing, or LLM infrastructure
Experience mentoring engineers or leading technical projects
Benefits
Additional discretionary bonuses/incentives
Restricted stock units
Day one access to medical, dental, and vision insurance
401(k) savings plan with company match
Paid parental leave
Short-term and long-term disability coverage
Life insurance
Wellbeing benefits
10 paid holidays per year
10 paid sick days per year
17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure)
About the Company
TikTok is the leading destination for short-form mobile video, with a mission to inspire creativity and bring joy. Its global headquarters are in Los Angeles and Singapore, with offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.
