Microsoft’s VS Code Team Redefines Product Management with AI Agents
The way software is built is undergoing a significant shift, with product managers (PMs) at Microsoft’s VS Code team now leveraging artificial intelligence (AI) agents to define, prototype, and evaluate user-facing features—a process traditionally handled by engineers. This move, spearheaded by Pierce Boggan, Product Lead for VS Code and GitHub Copilot, is reshaping the product development lifecycle and challenging conventional roles within engineering teams.
AI-Powered Workflows: A Day in the Life
Boggan emphasizes a core principle: using VS Code to build VS Code. This extends to AI integration, with PMs utilizing VS Code and GitHub Copilot for daily tasks. His morning routine begins with a prompt file aggregating calendar, email, and Teams data via Work IQ, coupled with GitHub MCP to provide updates on product and engineering progress. Pierce Boggan relies on this summary to stay informed about changes within VS Code.
AI-driven workflows extend to analyzing feedback repositories, social media sentiment, and maintaining documentation and release notes. Engineers are likewise benefiting, with custom agents and slash commands automating tasks like summarizing commits, grooming issues, and deduplicating tasks directly within VS Code. Copilot Code Review is now a mandatory first step in every pull request (PR).
The team has even developed a custom agent called ‘demonstrate’ that validates changes by launching VS Code, navigating to the feature, capturing screenshots, and assessing functionality—essentially, self-validating code.
Model Selection and Continuous Evolution
The VS Code team employs a benchmark, “vsc-bench,” to evaluate the performance of different AI models within their environment. Faster models are used for commit summarization, while more capable models handle code generation and review. In some cases, multiple models are run in parallel, with each grading the others’ work.
Boggan acknowledges the rapid evolution of AI tooling, stating that the tools in use today are likely to differ from those used a month from now—a deliberate aspect of their experimentation.
From Specs to Prototypes: The PR as the New Specification
A fundamental shift has occurred: the traditional specification or Product Requirements Document (PRD) is now a working prototype—a PR. Instead of documenting desired experiences, PMs are directly building them within VS Code’s plan mode. The specification refines itself through usage, not just documentation.
For example, a recent PR for forking conversations in Copilot Chat was authored by Boggan, refined with engineer feedback, and merged into VS Code. This approach is particularly effective for UI and interaction-level changes, where experiential evaluation is key. However, engineers retain accountability for code quality and architecture.
Quantifying the Impact: Metrics and Results
The integration of AI has yielded measurable results. After a decade of monthly releases, VS Code now ships updates weekly. Automation and agent assistance have reduced the overhead associated with release cycles, allowing for a faster cadence without compromising quality. Pierce Boggan’s team has seen a significant increase in commit velocity, regularly exceeding 100 commits per day, and a compression of PR cycle times.
The team’s agent-based triage pipeline now handles initial sorting, duplicate detection, and owner assignment, tasks previously requiring a dedicated human rotation. Regression rates are closely monitored to ensure velocity doesn’t negatively impact quality, and current data indicates fewer regressions are reaching stable releases.
The Future of Roles: Blurring Boundaries and the Agent-Ready Codebase
The line between product thinking and product building is becoming increasingly blurred. Everyone is evolving into a more full-stack contributor. As agents handle more mechanical tasks, skills like taste, judgment, and the ability to evaluate user delight become more valuable.
Boggan emphasizes that the key to future success lies in building an “agent-ready codebase”—one that is well-documented, well-tested, and has clear ownership boundaries to facilitate safe and effective AI-assisted contributions. Investing in this meta-skill is crucial for engineering teams looking to maximize the benefits of AI.
Further Learning
For those interested in learning more about building iOS apps with GitHub Copilot, Microsoft Learn offers a tutorial with Pierce Boggan. You can also view the tutorial on YouTube.