Machine Learning Infrastructure Engineer

About the Role

Join an AI team developing large foundation models for physical systems, with weather as a core application. You will build the infrastructure that enables scalable model training and inference, helping advance AI for science and physical AI.

What You'll Do

• Design, deploy, and maintain distributed machine learning training and inference clusters.

• Build scalable end-to-end pipelines for large datasets and model training across the machine learning lifecycle.

• Test training approaches, including parallelization methods and numerical precision trade-offs across model scales.

• Profile and debug low-level GPU operations to improve performance.

• Follow relevant research and apply useful ideas to infrastructure and training workflows.

What We're Looking For

• Two to ten years of experience building large-scale machine learning infrastructure for foundation models, including compute clusters, data infrastructure, inference, and distributed training.

• Experience optimizing large-scale training, data, or inference workloads, with expertise in distributed training frameworks such as FSDP or DeepSpeed.

• Proficiency with Python and C++, and experience with GPU performance optimization, CUDA, or JAX.

• Familiarity with cloud platforms such as GCP, AWS, or Azure, plus Docker, Kubernetes, and scalable model serving or task management systems.

• Background in physical AI or AI for science, such as robotics, biology, or other science-focused applications.

• Experience across the machine learning lifecycle, including analyzing training experiments and ablation studies, and working in fast-paced environments.

Compensation & Benefits

Base salary range: $200,000 to $400,000 USD annually. Visa sponsorship is not available.

Location

On-site in San Francisco, California, United States, five days per week. Relocation support is available.