Machine Learning Engineer

About This Role

This role is for an experienced Machine Learning Engineer to join Virtu's Research Technology team, focusing on building infrastructure for quantitative researchers. It offers a unique opportunity to work at the intersection of machine learning and systematic trading, developing tools that directly impact research speed and GPU cluster efficiency. The engineer will be responsible for the ML research platform, managing data, compute, and experiment tracking to optimize the research workflow.

Responsibilities

• Design and build experiment tracking, job orchestration, and reproducibility infrastructure
• Create tools for all stages of the simulation lifecycle, including historical back-tests and production monitoring
• Add new features to simulators
• Own visibility into GPU cluster utilization, tracking allocation, surfacing bottlenecks, and ensuring effective compute investment
• Diagnose and resolve performance issues across training pipelines: data loading throughput, storage I/O, GPU utilization, and inter-node communication in distributed training runs
• Build and maintain data pipelines that move financial data from storage into training workflows efficiently, with strong guarantees on correctness and versioning
• Develop feature storage and retrieval patterns that support fast, reproducible access to training data at scale
• Work directly with researchers to understand friction in their workflows and build solutions
• Collaborate with existing infrastructure engineers on capacity planning, cloud/on-prem tradeoffs, and tooling decisions
• Stay current with developments in ML infrastructure tooling and integrate relevant ideas and tools into the stack

Requirements

• 5+ years of experience in ML engineering, research infrastructure, or HPC environments
• Strong Python engineering skills, producing clean, maintainable, well-tested code
• Exposure to C++ in a performance-sensitive context is a plus
• Experience building or operating distributed training infrastructure, with working knowledge of collective communication libraries (NCCL, Horovod, or similar) at scale
• Practical experience with experiment tracking systems and strong opinions about good research infrastructure
• Comfort working across the Linux systems stack (storage, networking, job scheduling)
• Excellent communication skills and the ability to work closely with researchers and engineers across disciplines
• Intellectually curious and self-driven, proactively identifying problems worth solving

Nice to have:

• Experience with on-prem compute environments and job orchestration tools such as Slurm
• Familiarity with GPU profiling tools (NSight Systems, PyTorch Profiler) and hands-on experience optimizing GPU memory or compute utilization