(Staff / Senior Staff) AI Inference Reliability Engineer, Compute Infra

At Together AI, the AI Native Cloud company, the engineering team focuses on building and maintaining a Kubernetes-native control plane that manages a GPU inference fleet. This role is centered on ensuring the reliability and efficiency of the production environment where AI models are deployed and served. The team works closely with machine learning researchers and product managers to integrate cutting-edge AI technologies into a scalable infrastructure.

The AI Inference Reliability Engineer will be responsible for operating and improving the provisioning pipeline that handles GPU lifecycle events such as discovery, driver validation, and decommissioning. They will troubleshoot complex issues involving InfiniBand/RDMA networking and GPU serving faults, extending automated self-healing systems to prevent recurring failures. The role involves owning service-level objectives, leading incident responses, and collaborating cross-functionally to enhance system stability and performance.

Candidates should have extensive experience with large-scale distributed systems, Kubernetes controller patterns, and GPU/ML-serving infrastructure. Strong skills in diagnosing network and hardware faults, automating operational tasks, and designing event-driven architectures are essential. The position is based in San Francisco and demands a software engineering mindset focused on building reliable, automated infrastructure for AI inference workloads.