(Senior/Junior AI Inference Reliability Engineer, Compute Infra

At Together AI, the AI Native Cloud company, the engineering team focuses on building and maintaining a Kubernetes-native control plane that manages a GPU inference fleet. This role centers on ensuring the reliability and efficiency of the production infrastructure that provisions and runs AI model deployments. The team develops APIs and controllers that automate cluster and model deployment management, aiming to keep the system stable and performant under real-world conditions.

The engineer will be responsible for operating and improving the provisioning state machine, diagnosing complex issues such as InfiniBand/RDMA networking faults and GPU driver problems, and extending automated self-healing mechanisms. They will lead incident response, manage service-level objectives, and collaborate closely with ML platform and research teams to integrate new AI technologies into production. The role demands hands-on troubleshooting and software engineering skills focused on large-scale distributed GPU-serving systems.

Candidates should have experience with production reliability for distributed systems, Kubernetes controller patterns, and GPU infrastructure as software. Familiarity with multi-tenant model serving, event-driven architectures, and automation to reduce operational toil is essential. The position is based in San Francisco and involves working cross-functionally to improve system observability, performance, and stability, ensuring the AI inference platform meets high standards of reliability and efficiency.