{"id":12397,"date":"2026-06-02T12:15:48","date_gmt":"2026-06-02T19:15:48","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=12397"},"modified":"2026-06-02T20:08:08","modified_gmt":"2026-06-03T03:08:08","slug":"introducing-omnivec-an-open-source-embedding-platform-for-ai-apps-on-azure","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/introducing-omnivec-an-open-source-embedding-platform-for-ai-apps-on-azure\/","title":{"rendered":"Introducing OmniVec: An Open-Source Embedding Platform for AI Apps on Azure"},"content":{"rendered":"<p>Today we are open-sourcing <a href=\"https:\/\/github.com\/AzureCosmosDB\/OmniVec\">OmniVec<\/a>, a platform for building and operating the embedding pipelines that keep the vector representation of your operational data in sync as it changes. You register data sources, embedding model(s), vector stores (destination), and OmniVec does the rest: initial backfill, change tracking, model invocation to geenrate, and writing them back to your vector store. We are shipping this with support for <a href=\"https:\/\/learn.microsoft.com\/azure\/cosmos-db\/\">Azure Cosmos DB<\/a>, PostgreSQL, SQL Server (source and destination), and <a href=\"https:\/\/learn.microsoft.com\/azure\/storage\/blobs\/\">Azure Blob Storage<\/a> (destination). You deploy OmniVec in your own Azure subscription, and use the web UI, CLI, or the\u00a0 REST API to manage it.<\/p>\n<p>Most AI applications end up building the same plumbing to keep that vector representation in sync: a change-capture process on the source, a consumer pool that generates embeddings, retry and backfill logic, dead-letter handling, and the operational work to keep it all running. OmniVec collapses that stack into four configurable components: a source, a model, a destination, and a pipeline that wires the first three together.<\/p>\n<p dir=\"auto\" data-line=\"4\"><iframe src=\"\/\/www.youtube.com\/embed\/GnAI60YO75Q\" width=\"560\" height=\"314\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p>&nbsp;<\/p>\n<h2>Deep Dive<\/h2>\n<p>Deploying OmniVec into your Azure subscription provisions multiple Azure, including an <a href=\"https:\/\/learn.microsoft.com\/azure\/aks\/\">Azure Kubernetes Service<\/a> (AKS) cluster that hosts the OmniVec services (API, controller, and workers), an Azure Cosmos DB account for pipeline metadata, job state, and progress metrics, and an <a href=\"https:\/\/learn.microsoft.com\/azure\/container-registry\/\">Azure Container Registry<\/a> that stores the service images AKS pulls from, and more. You control the choices that affect cost and capacity, such as the embedding model (hosted Azure OpenAI or a self-hosted GPU model), the AKS node size and count, and whether to provision a GPU pool, etc.<\/p>\n<h3>Key concepts<\/h3>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/components-scaled.png\"><img decoding=\"async\" class=\"aligncenter wp-image-12402\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/components-300x191.png\" alt=\"OmniVec components\" width=\"691\" height=\"440\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/components-300x191.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/components-1024x653.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/components-768x490.png 768w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/components-1536x979.png 1536w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/components-2048x1306.png 2048w\" sizes=\"(max-width: 691px) 100vw, 691px\" \/><\/a><\/p>\n<p>OmniVec exposes the following core primitives:<\/p>\n<ul>\n<li><strong>Source<\/strong>: A system OmniVec reads records from. The source definition tells OmniVec how to connect to it and how to track changes on it.<\/li>\n<li><strong>Model<\/strong>: The embedding model OmniVec calls to turn record content into a vector. A model registration captures the provider, model name, endpoint, and credentials, and the runtime calls it with batching and retries.<\/li>\n<li><strong>Destination<\/strong>: A vector store where the generated embeddings are written, alongside (or as an update to) the originating record.<\/li>\n<li><strong>Pipeline<\/strong>: The binding that ties a source, a model, and a destination together. It also declares which source fields to embed and which destination field receives the vector.<\/li>\n<\/ul>\n<h3>Architecture at a glance<\/h3>\n<p>OmniVec runs as a set of services on Azure Kubernetes Service:<\/p>\n<ul>\n<li><strong>REST API<\/strong>: A FastAPI control plane that holds the configuration of sources, models, destinations, and pipelines, and exposes it over REST. The CLI and the web UI use this API behind the scenes.<\/li>\n<li><strong>Ingestion<\/strong>: Watches each source for changes and queues up the records that need embedding. How it tracks changes depends on the source: the change feed for Azure Cosmos DB, blob events for Azure Blob Storage, and CDC for SQL Server and PostgreSQL.<\/li>\n<li><strong>Workers<\/strong>: A horizontally scalable pool on compute that pulls records off the queue, calls the embedding model, and writes the resulting embedding to the destination.<\/li>\n<li><strong>Model routing layer<\/strong>: A unified entry point in front of the embedding models. It lets a pipeline call either an external provider like <a href=\"https:\/\/learn.microsoft.com\/azure\/ai-services\/openai\/\">Azure OpenAI<\/a> or a self-hosted GPU model.<\/li>\n<li><strong>Metadata store<\/strong>: An Azure Cosmos DB account that the deployment provisions for you. It holds pipeline definitions, job state, progress metrics, etc.<\/li>\n<\/ul>\n<h2>OmniVec in action<\/h2>\n<p>Let\u2019s put OmniVec to work end to end. In this walkthrough, you will:<\/p>\n<ul>\n<li>deploy it into your subscription<\/li>\n<li>point it at an Azure Cosmos DB container<\/li>\n<li>wire up a pipeline to automatically generate vector embeddings<\/li>\n<li>then run vector search over this data<\/li>\n<\/ul>\n<p>We use Azure Cosmos DB as both the source and the destination here as an example, but this applies to any supported combination of sources, models, and destinations.<\/p>\n<h3>Prerequisites<\/h3>\n<p>Before you start, make sure you have:<\/p>\n<ul>\n<li><strong>An Azure subscription<\/strong> with permissions to create resource groups, an AKS cluster, an ACR, and an Azure Cosmos DB account.<\/li>\n<li><strong>An embedding model.<\/strong> This walkthrough uses an <a href=\"https:\/\/learn.microsoft.com\/azure\/foundry-classic\/foundry-models\/concepts\/models-sold-directly-by-azure?tabs=americas%2Caz-global-standard%2Cglobal-standard&amp;pivots=azure-openai#embeddings\">Azure OpenAI embedding model deployed in Microsoft Foundry<\/a>. <a href=\"https:\/\/learn.microsoft.com\/azure\/foundry-classic\/openai\/how-to\/create-resource?pivots=cli#deploy-a-model\">Deploy one in your subscription<\/a> and note the endpoint, model name, and API key. You\u2019ll configure them in OmniVec later.<\/li>\n<li><strong>CLI tools:<\/strong>\n<ul>\n<li><a href=\"https:\/\/learn.microsoft.com\/azure\/developer\/azure-developer-cli\/install-azd\">Azure Developer CLI (azd)<\/a> for deployment.<\/li>\n<li><a href=\"https:\/\/learn.microsoft.com\/cli\/azure\/install-azure-cli\">Azure CLI (az)<\/a> to setup Azure Cosmos DB (but you can also do it directly via the Azure portal).<\/li>\n<li><a href=\"https:\/\/kubernetes.io\/docs\/tasks\/tools\/\">kubectl<\/a> and <a href=\"https:\/\/helm.sh\/docs\/intro\/install\/\">Helm<\/a><\/li>\n<li><strong>OmniVec CLI<\/strong> from the <a href=\"https:\/\/github.com\/AzureCosmosDB\/OmniVec\/releases\">OmniVec releases page<\/a> to manage sources, destinations, pipeline, and config.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Then clone the quickstart repo, which has the sample data and seed script used later:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">git clone https:\/\/github.com\/AzureCosmosDB\/omnivec-cosmosdb-quickstart\r\n\r\ncd omnivec-cosmosdb-quickstart<\/code><\/pre>\n<h3>Deploy and configure OmniVec<\/h3>\n<p>Create a new <code>azd<\/code> environment for the deployment. Pick an <code>ENV_NAME<\/code> (it becomes part of resource names), and choose a <code>LOCATION<\/code> that supports AKS and Azure Cosmos DB in your subscription:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">export ENV_NAME=\"my-omnivec\"\r\nexport LOCATION=\"westus2\"\r\nexport SUBSCRIPTION_ID=\"&lt;your-subscription-id&gt;\"\r\n\r\nazd env new \"$ENV_NAME\" --location \"$LOCATION\" --subscription \"$SUBSCRIPTION_ID\"<\/code><\/pre>\n<p>Configure the deployment:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">azd env set OMNIVEC_METADATA_STORE \"cosmosdb-provisioned\"\r\nazd env set OMNIVEC_ENABLE_BLOB_SOURCE \"false\"\r\nazd env set OMNIVEC_SYSTEM_NODE_VM_SIZE \"Standard_D2ds_v5\"\r\nazd env set OMNIVEC_SYSTEM_NODE_COUNT 2\r\nazd env set OMNIVEC_GPU_NODE_VM_SIZE \"\"\r\nazd env set OMNIVEC_GPU_NODE_COUNT 0\r\nazd env set OMNIVEC_BUILD_MODE \"acr\"\r\nazd env set OMNIVEC_BUILD true<\/code><\/pre>\n<ul>\n<li><code>OMNIVEC_METADATA_STORE<\/code> provisions a dedicated Azure Cosmos DB account for OmniVec\u2019s own pipeline metadata, job state, and progress metrics.<\/li>\n<li><code>OMNIVEC_ENABLE_BLOB_SOURCE<\/code> is off because this walkthrough only uses Azure Cosmos DB as a source; turn it on if you need Azure Blob Storage.<\/li>\n<li>Size the AKS system pool with <code>OMNIVEC_SYSTEM_NODE_VM_SIZE<\/code> and <code>OMNIVEC_SYSTEM_NODE_COUNT<\/code> based on your expected workload.<\/li>\n<li>The GPU pool (<code>OMNIVEC_GPU_NODE_VM_SIZE<\/code>, <code>OMNIVEC_GPU_NODE_COUNT<\/code>) is empty here because we\u2019re calling a hosted Azure OpenAI model; set them if you want to run self-hosted embedding models inside the cluster.<\/li>\n<li><code>OMNIVEC_BUILD_MODE=acr<\/code> builds the service images in the provisioned ACR, and <code>OMNIVEC_BUILD=true<\/code> triggers that build as part of this deployment.<\/li>\n<\/ul>\n<p>You only need one command for deployment:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">azd up<\/code><\/pre>\n<p>This takes a few minutes. It provisions the AKS cluster, Azure Cosmos DB account, and ACR, builds the OmniVec service images in ACR, and installs the Helm chart onto the cluster. When it finishes, you\u2019ll see a summary similar to the following (your values will differ):<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">\/\/..... redacted\r\nInstance ID: my-omnivec-dhmzlhlv4lk7s\r\nEnvironment: my-omnivec\r\nAKS Cluster: omnivec-aks-dhmzlhlv4lk7s\r\nACR Registry: omnivecacrdhmzlhlv4lk7s.azurecr.io\r\nCosmosDB: https:\/\/omnivec-cosmos-dhmzlhlv4lk7s.documents.azure.com:443\/\r\nAdmin Token: &lt;admin-token&gt;\r\n\r\nOmniVec FQDN: http:\/\/my-omnivec-dhmzlhlv4lk7s.westus2.cloudapp.azure.com\/ui\r\nOmniVec IP: http:\/\/&lt;public-ip&gt;\/ui\r\nHealth Check: http:\/\/my-omnivec-dhmzlhlv4lk7s.westus2.cloudapp.azure.com\/health<\/code><\/pre>\n<p><div class=\"alert alert-info\">Save the admin token and the OmniVec FQDN. You\u2019ll use both in the next step to point the CLI at this deployment. You can also open the FQDN in a browser to confirm the web UI is up.<\/div><\/p>\n<p>The rest of this walkthrough uses the CLI (the web UI exposes the same operations if you prefer to click through it). Point it at your deployment and authenticate:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">export SERVER_URL=&lt;e.g. http:\/\/my-omnivec-dhmzlhlv4lk7s.westus2.cloudapp.azure.com\/&gt;\r\nexport ADMIN_TOKEN=&lt;admin-token&gt;\r\n\r\nomnivec config set server $SERVER_URL\r\nomnivec auth login --token $ADMIN_TOKEN<\/code><\/pre>\n<p><div class=\"alert alert-primary\">Use the bare FQDN, without the \/ui suffix.<\/div><\/p>\n<p>Confirm the CLI is wired up:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">omnivec config view<\/code><\/pre>\n<p>Finally, register your Azure OpenAI embedding model with OmniVec. Pipelines reference models by the name you give them here:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">export FOUNDRY_ENDPOINT=&lt;your-foundry-endpoint&gt;\r\nexport FOUNDRY_MODEL_NAME=&lt;e.g. text-embedding-3-small&gt;\r\nexport FOUNDRY_API_KEY=&lt;your-foundry-api-key&gt;\r\n\r\nomnivec model add \\\r\n--name demo-foundry-oai-model \\\r\n--type azure-openai \\\r\n--model $FOUNDRY_MODEL_NAME \\\r\n--endpoint $FOUNDRY_ENDPOINT \\\r\n--api-key $FOUNDRY_API_KEY<\/code><\/pre>\n<h3>Set up Azure Cosmos DB<\/h3>\n<p>You need an Azure Cosmos DB account, database, and container that OmniVec will read from and write embeddings to. This walkthrough uses the same container as both source and destination. If you don\u2019t already have an Azure Cosmos DB account, <a href=\"https:\/\/learn.microsoft.com\/azure\/cosmos-db\/nosql\/quickstart-portal#create-account\">create one<\/a> before continuing.<\/p>\n<p>Sign in to the Azure CLI and select the subscription that holds the account:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">az login\r\naz account set --subscription \"$SUBSCRIPTION_ID\"<\/code><\/pre>\n<p>Your signed-in user needs permission to create a database and container on the account. The built-in <code>Cosmos DB Operator<\/code> role (or any role granting <code>Microsoft.DocumentDB\/databaseAccounts\/sqlDatabases\/*<\/code> and .<code>..\/containers\/*<\/code> write actions) is sufficient. Account <code>Contributor<\/code> or <code>Owner<\/code> also works.<\/p>\n<p>Set the names you\u2019ll use throughout the rest of the walkthrough:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">export RG=&lt;your-resource-group&gt;\r\nexport COSMOS_ACCOUNT=&lt;your-cosmos-account&gt;\r\nexport COSMOS_DB=omnivec-demodb\r\nexport COSMOS_CONTAINER=demo-container<\/code><\/pre>\n<p>Create the database:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">az cosmosdb sql database create \\\r\n--account-name $COSMOS_ACCOUNT \\\r\n--resource-group $RG \\\r\n--name $COSMOS_DB<\/code><\/pre>\n<p>Create the container with a vector embedding policy and a DiskANN vector index on <code>\/embedding<\/code>. The <code>1536<\/code> dimension matches <code>text-embedding-3-small<\/code>; change it if you registered a different model:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">az cosmosdb sql container create \\\r\n--account-name $COSMOS_ACCOUNT \\\r\n--resource-group $RG \\\r\n--database-name $COSMOS_DB \\\r\n--name $COSMOS_CONTAINER \\\r\n--partition-key-path \/id \\\r\n--vector-embeddings '{\"vectorEmbeddings\":[{\"path\":\"\/embedding\",\"dataType\":\"float32\",\"dimensions\":1536,\"distanceFunction\":\"cosine\"}]}' \\\r\n--idx '{\"indexingMode\":\"consistent\",\"automatic\":true,\"includedPaths\":[{\"path\":\"\/*\"}],\"excludedPaths\":[{\"path\":\"\/\\\"_etag\\\"\/?\"},{\"path\":\"\/embedding\/*\"}],\"vectorIndexes\":[{\"path\":\"\/embedding\",\"type\":\"diskANN\"}]}'<\/code><\/pre>\n<p>Now seed it with sample data. The seed script uses <code>DefaultAzureCredential<\/code>, so first grant your signed-in user the <code>Cosmos DB Built-in<\/code><code>Data Contributor<\/code> role on the Azure Cosmos DB account:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">export USER_PRINCIPAL_ID=$(az ad signed-in-user show --query id -o tsv)\r\n\r\naz cosmosdb sql role assignment create \\\r\n--account-name \"$COSMOS_ACCOUNT\" \\\r\n--resource-group \"$RG\" \\\r\n--role-definition-id \"00000000-0000-0000-0000-000000000002\" \\\r\n--principal-id \"$USER_PRINCIPAL_ID\" \\\r\n--scope \"\/\"<\/code><\/pre>\n<p>Then install the sample app\u2019s dependencies and run the seed script (it reads <code>COSMOS_ACCOUNT<\/code>, <code>COSMOS_DB<\/code>, and <code>COSMOS_CONTAINER<\/code> from the environment):<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">cd sample-app\r\npython3 -m venv .venv &amp;&amp; source .venv\/bin\/activate\r\n\r\npip install -r requirements.txt\r\npython insert_sample_data.py<\/code><\/pre>\n<p>You should now have 100 product documents in the container, each with name and description fields.<\/p>\n<h3>Grant OmniVec access to Azure Cosmos DB<\/h3>\n<p>OmniVec pods authenticate to Azure Cosmos DB using a managed identity, not a connection string. Before you create a source or destination, grant that identity two roles on the Azure Cosmos DB account:<\/p>\n<ul>\n<li><strong>Data plane<\/strong> (<code>Cosmos DB Built-in Data Contributor<\/code>): read source documents and write embeddings.<\/li>\n<li><strong>Control plane<\/strong> (<code>Cosmos DB Account Reader Role<\/code>): read the account-level metadata and vector policies that the change-feed processor inspects at startup. Skipping it causes pipeline failures with a readMetadata error.<\/li>\n<\/ul>\n<p>First, find the principal ID of the OmniVec workload identity. It lives in the OmniVec deployment\u2019s resource group (not the Azure Cosmos DB account\u2019s <code>$RG<\/code>). That resource group has several managed identities, so filter by the client ID that <code>azd<\/code> exported rather than picking the first one:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">export OMNIVEC_RG=$(azd env get-value AZURE_RESOURCE_GROUP)\r\n\r\nexport OMNIVEC_PRINCIPAL_ID=$(az identity list -g \"$OMNIVEC_RG\" \\\r\n--query \"[?clientId=='$(azd env get-value AZURE_IDENTITY_CLIENT_ID)'].principalId\" -o tsv)<\/code><\/pre>\n<p>Grant the data-plane role:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">az cosmosdb sql role assignment create \\\r\n--account-name \"$COSMOS_ACCOUNT\" \\\r\n--resource-group \"$RG\" \\\r\n--role-definition-id \"00000000-0000-0000-0000-000000000002\" \\\r\n--principal-id \"$OMNIVEC_PRINCIPAL_ID\" \\\r\n--scope \"\/\"<\/code><\/pre>\n<p>Grant the control-plane role:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">az role assignment create \\\r\n--assignee \"$OMNIVEC_PRINCIPAL_ID\" \\\r\n--role \"Cosmos DB Account Reader Role\" \\\r\n--scope \"$(az cosmosdb show -n \"$COSMOS_ACCOUNT\" -g \"$RG\" --query id -o tsv)\"<\/code><\/pre>\n<p>Wait 1\u20132 minutes for the assignments to propagate before moving on.<\/p>\n<h3>Wire up the pipeline<\/h3>\n<p>With Azure Cosmos DB ready and OmniVec authorized, the rest of the setup is three OmniVec resources: a source, a destination, and a pipeline that binds them to the model you registered earlier.<\/p>\n<p>Register the Azure Cosmos DB container as a source. OmniVec will watch its change feed:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">omnivec source create --name demo-cosmosdb-source \\\r\n--type cosmosdb \\\r\n--config \"{\\\"endpoint\\\":\\\"https:\/\/$COSMOS_ACCOUNT.documents.azure.com:443\/\\\",\\\"database\\\":\\\"$COSMOS_DB\\\",\\\"container\\\":\\\"$COSMOS_CONTAINER\\\",\\\"auth_type\\\":\\\"managed-identity\\\"}\"<\/code><\/pre>\n<p>Register the same Azure Cosmos DB container as a destination. OmniVec will write embeddings back into it:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">omnivec destination create --name demo-cosmosdb-destination \\\r\n--type cosmosdb-vector \\\r\n--config \"{\\\"endpoint\\\":\\\"https:\/\/$COSMOS_ACCOUNT.documents.azure.com:443\/\\\",\\\"database\\\":\\\"$COSMOS_DB\\\",\\\"container\\\":\\\"$COSMOS_CONTAINER\\\",\\\"auth_type\\\":\\\"managed-identity\\\"}\"<\/code><\/pre>\n<p>The pipeline needs three IDs: source, destination, and model. Grab the source and destination IDs from <code>omnivec source list<\/code> and <code>omnivec destination list<\/code> (they look like <code>src-\u2026<\/code> and <code>dst-\u2026<\/code>). For the model, the registered name (demo-foundry-oai-model) isn\u2019t the ID; the internal <code>mdl-ext-*<\/code> ID is what the pipeline expects:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">export SOURCE_ID=&lt;src-id-from-source-list&gt;\r\nexport DESTINATION_ID=&lt;dst-id-from-destination-list&gt;\r\nexport MODEL_ID=$(omnivec model test demo-foundry-oai-model | sed 's\/^OK: Model test returned: \/\/' | jq -r '.id')<\/code><\/pre>\n<p>Create the pipeline. <code>--content-fields<\/code> tells OmniVec which fields on each source document to embed (concatenated), <code>--embedding-field<\/code> is where the resulting vector lands on the destination document, and <code>--vector-index-path<\/code> is the path the destination container\u2019s vector index looks at (it should match the path you set in the <code>az cosmosdb sql container create<\/code> step):<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">omnivec pipeline create --name demo-cosmosdb-pipeline \\\r\n--source $SOURCE_ID --destination $DESTINATION_ID --model $MODEL_ID \\\r\n--content-fields name,description --embedding-field embedding \\\r\n--vector-index-path \/embedding --processing-mode inline<\/code><\/pre>\n<p>You should see something like:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">OK: Pipeline created: pip-94348d5e (status: paused)<\/code><\/pre>\n<p>New pipelines start paused. Grab the pipeline ID from the output (or from <code>omnivec pipeline list<\/code>) and resume it:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">export PIPELINE_ID=&lt;pip-id-from-create-output&gt;\r\nomnivec pipeline resume $PIPELINE_ID<\/code><\/pre>\n<p>Check the pipeline\u2019s progress:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">omnivec pipeline show $PIPELINE_ID<\/code><\/pre>\n<p>Once the workers have processed the seed data, you\u2019ll see output similar to the following (truncated for brevity):<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">Pipeline\r\nID: pip-94348d5e\r\nName: demo-cosmosdb-pipeline\r\nStatus: active\r\n...\r\n\r\nStats\r\nDocuments Embedded: 100\r\nSource Docs: 100\r\nCompletion: 100.0%\r\nFailed: 0\r\nPending: 0\r\n...<\/code><\/pre>\n<p>Completion: 100.0% means every seed document now has an embedding written back to its embedding field. From this point on, any insert or update to the source container flows through the change feed and gets re-embedded automatically.<\/p>\n<h3>Run vector search<\/h3>\n<p>The embeddings are in place, so you can run vector search against the destination. Use the OmniVec CLI\u2019s search command with a natural-language query:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">omnivec search \"warm waterproof hiking pants\" --index $DESTINATION_ID --top-k 3<\/code><\/pre>\n<p>Try another query to confirm it generalizes beyond the obvious matches:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">omnivec search \"lightweight running shoes\" --index $DESTINATION_ID --top-k 3<\/code><\/pre>\n<p><div class=\"alert alert-success\">The web UI ships with a vector search playground that calls the same API<\/div><\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/vector-search-playground.png\"><img decoding=\"async\" class=\"aligncenter wp-image-12399\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/vector-search-playground-300x205.png\" alt=\"Screenshot of the OmniVec vector search playground interface showing an AI-powered semantic search experience. The UI includes a search input area for natural language queries, embedding and vector search controls, and a results panel displaying matched documents with similarity scores. The interface uses a modern dark theme with purple and blue accents, demonstrating how developers can experiment with vector embeddings and semantic retrieval for AI applications using Azure Cosmos DB and OmniVec.\" width=\"574\" height=\"393\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/vector-search-playground-300x205.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/vector-search-playground-768x524.png 768w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/vector-search-playground-1536x1048.png 1536w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2026\/05\/vector-search-playground-2048x1397.png 2048w\" sizes=\"(max-width: 574px) 100vw, 574px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<h2>Wrapping up<\/h2>\n<p>The walkthrough above used Azure Cosmos DB as both source and destination, with a hosted Azure OpenAI embedding model, but you can mix and match sources, destinations, and models. It supports a wide range of scenarios \u2013 a common use case is pointing OmniVec at an Azure Blob Storage container full of PDFs and other documents and writing the embeddings into Azure Cosmos DB.<\/p>\n<p>OmniVec is on GitHub at <a href=\"https:\/\/github.com\/AzureCosmosDB\/OmniVec\">AzureCosmosDB\/OmniVec<\/a>. It is a <strong>public preview<\/strong>, so expect changes as it matures. Clone it, deploy it into your own subscription, and try it on one of your own datasets. Bug reports, feedback, and requests are all welcome through GitHub issues. And if you build something interesting on top of it, we\u2019d love to hear about that too!<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Today we are open-sourcing OmniVec, a platform for building and operating the embedding pipelines that keep the vector representation of your operational data in sync as it changes. You register data sources, embedding model(s), vector stores (destination), and OmniVec does the rest: initial backfill, change tracking, model invocation to geenrate, and writing them back to [&hellip;]<\/p>\n","protected":false},"author":181737,"featured_media":12459,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1610,1980,1862,644,2023],"tags":[],"class_list":["post-12397","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-azure-cosmos-db","category-azure-openai","category-change-feed","category-rag-ai"],"acf":[],"blog_post_summary":"<p>Today we are open-sourcing OmniVec, a platform for building and operating the embedding pipelines that keep the vector representation of your operational data in sync as it changes. You register data sources, embedding model(s), vector stores (destination), and OmniVec does the rest: initial backfill, change tracking, model invocation to geenrate, and writing them back to [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/12397","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/users\/181737"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/comments?post=12397"}],"version-history":[{"count":3,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/12397\/revisions"}],"predecessor-version":[{"id":12531,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/12397\/revisions\/12531"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media\/12459"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media?parent=12397"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/categories?post=12397"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/tags?post=12397"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}