Smarter Code Search in Visual Studio: From BM25 to Semantic Search 

In our latest 17.14.11 release, we’ve made a significant leap forward in how we explore your code to retrieve meaningful context. Our new Remote Semantic Search integration helps you find exactly what you need faster and with greater precision than ever before. 

By embedding Remote Semantic Search directly into the Visual Studio Copilot code search experience, we’ve combined the power of traditional keyword search (BM25) with the deep contextual understanding of cutting-edge AI models. This means your searches go beyond just matching words, they grasp the concepts and intent behind your search queries. 

Let’s dive into what’s new, why it’s a game-changer, and how you can start using it today. 

The Legacy: BM25 + Reranking 

Before this update, Visual Studio’s code search relied solely on a BM25-based search engine. BM25 is a well-established ranking algorithm that evaluates how closely each document, such as code files or symbols, matches your query. It does this by analyzing term frequency and document length, balancing how often a term appears within a document against how rare it is across the entire codebase. Simply put, the more a term shows up in a file (to a certain extent) and the less common it is overall, the more relevant that file is considered. 

On top of that, we used reranking logic to fine-tune results. This step gave extra weight to certain matches, like hits in file names or symbols within active projects, so that the most likely relevant results would surface right at the top. 

The limitations: 

• BM25 is purely keyword-based. It doesn’t understand synonyms, concepts or context by itself. 

• Searching for “get user authentication token” would only match files containing those exact words (authentication, token, etc), missing matches like RetrieveOAuthCredential or JWT.

The Upgrade: Semantic Search 

Unlike traditional search that focuses on matching exact words, semantic search matches meaning. Powered by advanced vector embeddings, it transforms both your query and every piece of code into points in a high-dimensional space. That’s where their semantic similarity can be measured. 

This means it understands that phrases like “fetch user credentials” and “get authentication token” are closely related, even if they don’t share any exact words. Semantic search captures the purpose of functions, the intent behind variables, and even the context in code comments to deliver results that truly match what you’re looking for. 

Using Semantic Code Search in Visual Studio 

Thanks to our remote indexing, Semantic Code Search is available for solutions hosted in Azure DevOps and GitHub repositories that have been indexed. To learn more about the GitHub integration check out About GitHub Code Search – GitHub Docs.  

To try it out: 

• Open your Copilot Chat window  

• Use our #solution feature to ask questions like: 

• • “#solution Where are the API requests?”  

• • “Where is Authentication Handled? #solution”  

Feel free to use natural language, our semantic engine understands full sentences. For example: “Where do we generate the JWT token for API requests?” 

Examples in Action 

Let’s explore how this works using the Roslyn repository. We’ll run search queries in Copilot and compare the results side-by-side. In the table below, you’ll see results returned by the traditional BM25 algorithm on the left, and the new Semantic Search on the right. These examples highlight how much smarter and more relevant your code search can be. 

Comparison table
BM25 method	Semantic Search
With semantic search, the results may be fewer, but they are significantly more accurate. The search engine was able to understand that “deleting not synchronous modifiers” and “removing Async modifiers” convey the same intent, even though the wording is different, demonstrating its deeper understanding of code meaning.
BM25 tends to return files that contain matching terms, even if they are less relevant or potentially unrelated, such as corrupted or test files. For example, with BM25, an extra file named SmartIdenterEnterOnTokenTests was included in the results, which was not directly relevant to the query.

Wrapping Up 

Remote Semantic Search into Visual Studio Copilot marks a major step forward in how Copilot understands developer’s codebase. By combining the precision of traditional keyword search with the deep, context-aware insights of AI-driven semantic search, finding the right code has never been easier. 

Whether you’re digging into complex legacy projects or exploring unfamiliar repositories, this powerful new search experience helps get straight to the heart of what matters, saving time and reducing frustration. 

Give it a try today and experience a smarter way to search your code. 

Happy coding!