Embeddings and vector search in Microsoft Agent Framework for Go

A user asks, “How do I sign in without keeping a password in my application?” Your documentation answers that question in an article about Microsoft Entra ID authentication. A keyword search might miss the connection, but an embedding model lets your application find passages with a similar meaning to the query, even when the words don’t match.

The model converts the query and each passage into vectors: lists of numbers that can be compared mathematically. Texts with similar meanings tend to produce similar vectors. Microsoft’s guide to embeddings for retrieval-augmented generation illustrates how this comparison works.

Microsoft Agent Framework for Go now includes embedding generation and vector-store APIs for building that retrieval loop. These are building blocks for retrieval-augmented generation (RAG), where retrieved documents provide context for a model’s response. You can start with an in-memory collection, move to Azure AI Search or Azure Cosmos DB, and use the results in an agent without writing a different retrieval pipeline for every backend.

These features are available in Microsoft Agent Framework for Go v0.2.0. The framework remains in public preview.

Generate embeddings and search records

There are two building blocks:

• embedding generates vectors from text. Its Generator interface accepts batches of inputs and returns embeddings in the same order.
• vectordata stores records in collections and finds records whose vectors are closest to a query vector. Collections have typed keys and records, with schemas described through struct tags or explicit definitions. Map-based records are available when a schema is known only at runtime.

You can use either package on its own: call embedding.GenerateVector to embed a single input, or use vectordata with precomputed vectors to find records whose vectors are closest to a query vector.

To let vectordata generate embeddings for you, configure a generator on the store. The collection can then accept text in vector fields during Upsert and text queries during Search, calling the generator in both cases.

Semantic search with a Go struct

Here’s a complete example using Microsoft Foundry embeddings and in-memory storage. It requires Go 1.26 or later:

go mod init example.com/semantic-search
go get github.com/microsoft/agent-framework-go@v0.2.0

Use a Microsoft Foundry resource with a text-embedding-3-large deployment. If you named the deployment differently, change Model in the example to that deployment name. Set FOUNDRY_PROJECT_ENDPOINT to your project endpoint, for example https://<resource-name>.services.ai.azure.com/api/projects/<project-name>.

For local development, sign in with az login using an identity with permission to invoke the deployment. The example uses DefaultAzureCredential, which can use those Azure CLI credentials to authenticate the embedding requests.

Save the following as main.go. Storage is in-process, but embedding requests go to Foundry and incur model usage charges.

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
    "github.com/microsoft/agent-framework-go/embedding"
    "github.com/microsoft/agent-framework-go/provider/foundryprovider"
    "github.com/microsoft/agent-framework-go/vectordata"
    "github.com/microsoft/agent-framework-go/vectordata/inmemory"
)

type Document struct {
    ID        string `vectordata:"key"`
    Category  string `vectordata:"data,indexed"`
    Text      string `vectordata:"data"`
    Embedding string `vectordata:"vector,dimensions=3072"`
}

func main() {
    ctx := context.Background()
    endpoint := os.Getenv("FOUNDRY_PROJECT_ENDPOINT")
    if endpoint == "" {
        log.Fatal("FOUNDRY_PROJECT_ENDPOINT must be set")
    }
    credential, err := azidentity.NewDefaultAzureCredential(nil)
    if err != nil {
        log.Fatal(err)
    }
    generator := foundryprovider.NewEmbeddingGenerator(
        endpoint, credential,
        foundryprovider.EmbeddingGeneratorConfig{
            Model: "text-embedding-3-large",
        },
    )
    store := inmemory.New(inmemory.Config{
        EmbeddingGenerator: embedding.Bind(generator),
    })
    docs, err := vectordata.GetCollection[string, *Document](
        store.Store, "documentation", nil,
    )
    if err != nil {
        log.Fatal(err)
    }
    if err := docs.EnsureExists(ctx); err != nil {
        log.Fatal(err)
    }

    records := []*Document{
        {ID: "auth", Category: "guides",
            Text: "Use Microsoft Entra ID to authenticate without storing application passwords."},
        {ID: "streaming", Category: "guides",
            Text: "Stream agent responses to display text as it arrives."},
        {ID: "internal", Category: "notes",
            Text: "Internal notes about application passwords."},
    }
    // Text holds the passage to show the user. Embedding supplies text to the generator.
    // Here, embed the full passage.
    for _, record := range records {
        record.Embedding = record.Text
    }
    if err := docs.Upsert(ctx, records...); err != nil {
        log.Fatal(err)
    }

    filter := vectordata.Filter{
        Operator: vectordata.Equal,
        Property: "Category", Value: "guides",
    }
    const maxResults = 1
    for result, err := range docs.Search(
        ctx, "How can I sign in without keeping a password?", maxResults,
        &vectordata.SearchOptions{Filter: &filter},
    ) {
        if err != nil {
            log.Fatal(err)
        }
        fmt.Println(result.Record.Text)
    }
}

With the file saved, resolve its dependencies and run the example:

go mod tidy
go run .

The example copies each passage into two fields for different purposes:

• Text uses the data tag to keep the original passage for search results.
• Embedding uses the vector tag to supply text to the embedding model. The store uses the generated vector to find similar passages.

Azure AI Search and Cosmos DB store the generated vector, not the text supplied through Embedding. Keeping that text separately in Text lets the application display the original passage when it finds a match.

In this in-memory example, docs.Upsert passes the three Embedding input strings to the generator as one batch, leaving the strings unchanged. The connector stores the vectors separately and returns the original records from docs.Search. Calling docs.Upsert again regenerates their embeddings, even if the text has not changed.

A string vector field is treated as source text, not as a serialized vector. To supply vectors you have already computed, declare that field as []float32 instead.

Searching with a string generates the query vector through the same configured generator. The category filter limits the search to records in "guides". maxResults caps the number of matches: set it to 1 for the closest match, or increase it to retrieve more results.

Choose the storage that fits

The v0.2.0 release includes three storage connectors:

The connectors share methods for inserting, updating, reading, deleting, and searching records, but their capabilities differ:

• Azure AI Search and Cosmos DB support hybrid search, which combines keyword matching with vector similarity. The in-memory connector supports vector search only.
• Cosmos DB requires equality filters for every partition-key component on ranked or ordered queries because the Go client does not yet support ranking or ordering across partitions.
• Azure AI Search accepts token credentials. The Cosmos DB connector uses an azcosmos.Client that you configure and pass in.

From search results to agent context

You can use these packages directly in a search endpoint, CLI, or background worker, or connect retrieval to an agent.

For agents, agent/textsearch accepts a search function returning passages and optional source names and links. It can retrieve context before a run or expose search as a tool the model calls on demand. This supports RAG by providing retrieved passages as context for the model’s response, with source information for citations.

agent/memory uses vector retrieval to store messages and recall related conversation history across sessions. You can scope that history by application, agent, user, and session.

Tips for using your own data

The example uses three short passages. For a larger collection, consider the following during development:

• Test with representative questions. Inspect the returned passages to check whether they answer the question. Even the closest vector match might not be relevant enough to use.
• Use compatible embeddings. Use the same model for documents and queries, and match the schema’s dimensions to its output. Regenerate stored embeddings when changing models, even if the dimensions stay the same. If different collections need different models, override the store’s generator per collection or vector property.
• Split large documents into passages. Choose those boundaries before indexing, and retain source names and links so users can find the original document.
• Track embedding usage. The optional otelprovider.WrapEmbeddingGenerator wrapper provides OpenTelemetry spans and metrics for generation latency and reported input-token usage.

Try it

The framework’s basic text RAG sample is a companion you can clone and run. It uses Foundry embeddings and an in-memory vector store, then supplies the retrieved passages to an agent. Clone the full repository at v0.2.0 to include its go.mod and shared example helpers.

For other backends and conversation memory, explore the Azure AI Search sample, Cosmos DB sample, and chat-history memory sample.

We welcome feedback and contributions. Tell us which retrieval workloads, embedding providers, and vector databases you want to build with Go.