Research Intern, Frontier Speculative Decoding (Winter 2027)

At Together AI, the Speculative Decoding team focuses on developing innovative models and algorithms that enable large language models to generate outputs more quickly without compromising quality. Their work integrates model training, inference algorithms, and serving systems to reduce generation costs and latency. The team operates on cutting-edge open models and aims to improve efficiency in AI generation processes.

The research intern will engage in exploring new architectures and verification methods for speculative decoding, working on training and fine-tuning draft models using GPU clusters. They will test their approaches against realistic workloads, including long-context and agentic traffic, contributing to projects that enhance generation speed and adaptability of models under varying conditions. The role involves designing experiments, implementing optimized serving engines, and communicating findings through technical documentation and presentations.

Candidates should be pursuing advanced degrees in computer science or related fields, with strong foundations in machine learning and experience in distributed training of large models. Familiarity with modern architectures like Mixture-of-Experts and hybrid attention is important. Preferred qualifications include prior research in efficient inference, publications in top conferences, and experience with knowledge distillation and LLM serving engines. The internship is a 12 to 14-week program based in San Francisco, offering competitive compensation and opportunities to collaborate with leading AI researchers.