Sasha Rush on Building Cursor Composer and the Future of Agentic Coding

Vision and Core Principles of Composer

• 💡 The primary vision for Cursor Composer was to create a smart, fast, and agentic coding model, building on the success of Cursor Tab's snappy autocomplete feature.
• ⚡ Speed was a critical design decision, as a fast model allows developers to stay in the flow of coding and iterate quickly on solutions.
• ✅ The goal was to develop a model that was not only fast but also smart enough to be trusted for writing code, addressing limitations of previous fast but less capable models.

Architectural Innovations

• 🧠 Composer utilizes Reinforcement Learning (RL) to specialize the model for coding tasks, moving beyond general-purpose abilities to excel in a specific domain.
• 🧩 The model incorporates a Mixture of Experts (MoE) architecture, which is an enhancement to the core transformer design where a set of distributed neural networks are used, and only a subset is activated for each computation.
• 🚀 MoE allows for computational efficiency and distributed computation by sharding experts onto different GPUs, making training on many GPUs more effective.

Agentic Capabilities and Tool Use

• 🛠️ Composer is designed as an agentic system, differing from simple LLMs by its ability to persistently call numerous tools (e.g., 150+) to find answers.
• 🔍 These tools include capabilities like searching large codebases and running terminal commands, enabling the model to perform tasks that humans might find tedious or time-consuming.
• 🎯 The agent's power lies in its persistence in trying to find the correct answer, making it highly effective for complex coding challenges.

Scaling Training and Infrastructure

• ☁️ Training Composer at scale required running a full version of Cursor within virtual machines, launching hundreds of thousands of these to simulate real user experiences.
• 📊 Distributed computing was essential, with Ray being used extensively across five different areas, including as the standard for the RL controller to manage rollouts with varying completion times.
• 📈 Ray Data was also employed for processing and analyzing large volumes of rollout data, allowing for efficient identification of what was working or not in the training process.

Practical Application and Future Outlook

• 💡 A recommended way to use Composer is to have a slower model draft a strategic plan for large changes, then use Composer to quickly implement that plan, allowing for rapid corrections and iterations.
• 🚀 Cursor 2.0 offers features like running multiple agents in parallel and cloud agents for offline or long-term changes, providing flexibility in how developers interact with the system.
• 🌱 The future of AI models in coding is expected to involve continued improvements in intelligence and speed, with a trend towards more specialized models and the increasing importance of open-source frameworks like PyTorch and VLM.