Research Intern, Frontier Speculative Decoding (Summer 2027)

At Together AI, the Speculative Decoding team focuses on developing draft models and algorithms that enable large language models to generate outputs faster without compromising quality. Their work integrates model training, inference algorithms, and serving systems to reduce generation costs and latency. The team operates on frontier open models and aims to innovate in speculative decoding techniques that adapt to changing workloads and real-world traffic patterns.

The research intern will engage in exploring new architectures and verification methods for speculative decoding, training and fine-tuning draft models on GPU clusters, and evaluating their performance under realistic conditions. The role involves designing experiments, implementing optimized serving engines, and communicating findings through technical documentation and collaboration with the broader team. The internship spans 12 to 14 weeks during the summer, offering hands-on experience with cutting-edge AI research and production systems.

Candidates should be pursuing a Master's or Ph.D. in a relevant field and possess strong machine learning fundamentals, experience with distributed training frameworks, and familiarity with modern model architectures like Mixture-of-Experts. Preferred qualifications include prior research in efficient inference, publications in top conferences, and knowledge of LLM serving engines. The position is based in San Francisco and offers competitive compensation and benefits, emphasizing diversity and equal opportunity.