About the role
At Sarvam AI, the engineering team is focused on developing and maintaining a sophisticated API platform that supports various AI models including large language models, speech recognition, text-to-speech, vision, and translation. This platform enables developers and enterprises to integrate foundation models into their products through OpenAI-compatible APIs. The team is currently transitioning their backend from Python to Go to enhance performance and reliability, emphasizing a design-first and documentation-driven engineering culture.
The Software Development Engineer 2 role involves owning critical components of the API platform, ensuring high reliability, performance, and scalability. Daily responsibilities include building and evolving the API stack, managing streaming infrastructure for chat completions, handling routing and failover mechanisms, and maintaining commercial features like metering and billing. The engineer will collaborate closely with inference and MLOps teams to prepare new model launches for production, while also focusing on observability and integration testing to maintain robust service delivery.
Candidates should have at least three years of experience building backend systems, strong Python skills with frameworks like FastAPI, and familiarity or willingness to learn Go. Experience with streaming protocols such as SSE or WebSockets, PostgreSQL and Redis for data management, and running services on Kubernetes is essential. Bonus qualifications include prior work with LLM APIs, inference servers, and usage-based pricing systems. The role is onsite in Bengaluru, offering an opportunity to contribute to AI solutions with significant societal impact in India.