Software Engineer - Inference Platform

At Baseten, the engineering team focuses on building a scalable inference platform that supports large-scale AI model deployment. The role involves developing infrastructure for distributed systems, managing model APIs, and optimizing inference performance to ensure reliability and speed. The team works across the stack, from user-facing deployment tools to low-level orchestration on Kubernetes, aiming to make AI model deployment seamless and efficient.

Day-to-day responsibilities include designing and maintaining systems for routing, autoscaling, and runtime management of large language models, as well as implementing features like structured outputs and multimodal serving. The engineer will also focus on observability, benchmarking, and troubleshooting complex production environments, collaborating closely with performance engineers and other teams to enhance system capabilities.

Candidates should have over three years of experience in building reliable distributed systems, with strong skills in backend infrastructure, APIs, and performance optimization. Familiarity with Kubernetes, GPU workloads, and inference systems is advantageous. The role offers a hybrid work environment, with opportunities to work remotely or on-site in major North American cities, and involves ownership of projects from architecture to deployment, emphasizing operational excellence and developer experience.