{"id":24354,"date":"2025-06-04T23:36:49","date_gmt":"2025-06-05T06:36:49","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/?p=24354"},"modified":"2025-06-04T23:36:49","modified_gmt":"2025-06-05T06:36:49","slug":"dev-proxy-v0-28-with-llm-usage-and-costs-tracking","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/dev-proxy-v0-28-with-llm-usage-and-costs-tracking\/","title":{"rendered":"Dev Proxy v0.28 with LLM usage and costs tracking"},"content":{"rendered":"<p>We\u2019re thrilled to announce the release of Dev Proxy v0.28. This version introduces a new ability to help you understand language models\u2019 usage and costs in your applications, alongside many improvements to mocking, TypeSpec generation, and plugin flexibility.<\/p>\n<p>In this version:<\/p>\n<ul>\n<li>New OpenAITelemetryPlugin to track language models\u2019 usage and costs<\/li>\n<li>Expanded OpenAI payload support<\/li>\n<li>New Dev Proxy extensions for .NET Aspire<\/li>\n<li>Improved generating PATCH operations for TypeSpec<\/li>\n<li>Support for JSONC in mock files<\/li>\n<li>Improved logging<\/li>\n<li>CRUD plugin CORS and JSON response fixes<\/li>\n<li>&#8230;and more!<\/li>\n<\/ul>\n<h2>Track OpenAI usage with the new OpenAITelemetryPlugin<\/h2>\n<p>LLMs are powerful. When integrated thoughtfully with applications, they can support new scenarios and significantly improve existing ones. Using LLMs typically incurs costs. But how much does it cost exactly? The new Dev Proxy <strong>OpenAITelemetryPlugin<\/strong> gives you visibility into how your apps interact with OpenAI or Azure OpenAI endpoints.<\/p>\n<p>This plugin intercepts LLM requests from your application, and for each request tracks:<\/p>\n<ul>\n<li>Model used<\/li>\n<li>Token count (prompt, completion, total)<\/li>\n<li>Cost estimation per request<\/li>\n<li>Grouped summaries per model<\/li>\n<\/ul>\n<p>You can visualize the usage patterns and costs using specialized solutions such as <a href=\"https:\/\/openlit.io\/\">OpenLIT<\/a>:<\/p>\n<p style=\"text-align: center;\"><img decoding=\"async\" class=\"alignnone wp-image-24355\" src=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1.png\" alt=\"OpenLIT visualizing language model usage data emitted by Dev Proxy\" width=\"2542\" height=\"2592\" srcset=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1.png 2542w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1-294x300.png 294w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1-1004x1024.png 1004w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1-768x783.png 768w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1-1506x1536.png 1506w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1-2008x2048.png 2008w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1-24x24.png 24w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-1-48x48.png 48w\" sizes=\"(max-width: 2542px) 100vw, 2542px\" \/><em>OpenLIT visualizing language model usage data emitted by Dev Proxy<\/em><\/p>\n<p>Or generic Open Telemetry dashboards such as .NET Aspire:<\/p>\n<p style=\"text-align: center;\" data-wp-editing=\"1\"><img decoding=\"async\" class=\"alignnone wp-image-24356\" src=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-2.png\" alt=\"The .NET Aspire dashboard visualizing language model usage information emitted by Dev Proxy\" width=\"2546\" height=\"2012\" srcset=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-2.png 2546w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-2-300x237.png 300w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-2-1024x809.png 1024w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-2-768x607.png 768w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-2-1536x1214.png 1536w, https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-content\/uploads\/sites\/73\/2025\/06\/word-image-24354-2-2048x1618.png 2048w\" sizes=\"(max-width: 2546px) 100vw, 2546px\" \/><em>The .NET Aspire dashboard visualizing language model usage information emitted by Dev Proxy<\/em><\/p>\n<p>Use Dev Proxy and the new OpenAITelemetryPlugin and <a href=\"https:\/\/learn.microsoft.com\/microsoft-cloud\/dev\/dev-proxy\/how-to\/understand-language-model-usage?tabs=aspire\">understand how your application is using language models<\/a>, before you go live.<\/p>\n<h2>Try Dev Proxy with Foundry Local<\/h2>\n<p>At the recent Build conference, Microsoft announced <a href=\"https:\/\/devblogs.microsoft.com\/foundry\/unlock-instant-on-device-ai-with-foundry-local\/\">Foundry Local<\/a> &#8211; the high-performance local AI runtime stack that brings Azure AI Foundry\u2019s power to client devices. We\u2019re excited to share that you can configure Dev Proxy to use Foundry Local as the local language model provider. By enabling the integration, you can improve <a href=\"https:\/\/learn.microsoft.com\/microsoft-cloud\/dev\/dev-proxy\/how-to\/generate-openapi-spec\">generating OpenAPI specs<\/a> and <a href=\"https:\/\/learn.microsoft.com\/microsoft-cloud\/dev\/dev-proxy\/how-to\/generate-typespec-file\">TypeSpec files<\/a>, and <a href=\"https:\/\/learn.microsoft.com\/microsoft-cloud\/dev\/dev-proxy\/how-to\/simulate-azure-openai\">redirect cloud LLM requests to your local machine<\/a> to save costs and enable offline development.<\/p>\n<p>To use Dev Proxy with Foundry Local, configure Dev Proxy to use Foundry Local:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/waldekmastykarz\/aed048d055fe086f635534123da43c64.js\"><\/script>\nOur initial tests show significant improvements using Phi-4 mini on Foundry Local compared to other models we\u2019ve used in the past. We\u2019re planning to integrate with Foundry Local by default, in the future versions of Dev Proxy. Meanwhile, try it today!<\/p>\n<h2>Dev Proxy .NET Aspire extensions<\/h2>\n<p>To simplify integrating Dev Proxy with .NET Aspire applications, we released a preview version of Dev Proxy extensions for .NET Aspire. Using the extensions, you can integrate Dev Proxy with your .NET Aspire application with just a few lines of code. The extensions allow you to integrate the locally installed Dev Proxy instance or run it from a Docker container for easier portability across your team.<\/p>\n<p><a href=\"https:\/\/learn.microsoft.com\/microsoft-cloud\/dev\/dev-proxy\/how-to\/use-dev-proxy-with-dotnet-aspire\">Learn more about using the Dev Proxy .NET Aspire extensions<\/a> and let us know how we could make it better.<\/p>\n<h2>Bug fixes and improvements<\/h2>\n<p>In this version, we include several bug fixes and improvements.<\/p>\n<h3>Expanded OpenAI payload support<\/h3>\n<p>Continuing our work related to AI, we\u2019ve expanded our support for OpenAI payloads. Previously, when redirecting OpenAI-compatible requests, we only supported text completions. In this version, we introduce support for other types of completions as well.<\/p>\n<h3>Improved generating PATCH operations for TypeSpec<\/h3>\n<p>In the previous version, we introduced support for <a href=\"https:\/\/devblogs.microsoft.com\/microsoft365dev\/dev-proxy-v0-27-with-generating-typespec-files-and-configuring-using-natural-language\/#generate-typespec-definitions-from-requests\">generating TypeSpec files<\/a>: a new language for describing APIs. In this version, we improved generating PATCH operations, aligning with TypeSpec v1.0 features, so that merge patch operations are designated with MergePatchUpdate.<\/p>\n<h3>Support for JSONC in all Dev Proxy files<\/h3>\n<p>In this version, we introduce support for JSONC in all Dev Proxy files. With support for JSONC, you can annotate Dev Proxy\u2019s config files with additional information. Including comments right in Dev Proxy files is highly convenient for sharing configuration with others in the team and picking it up after a while.<\/p>\n<h3>Concurrency fixes for mocking and logging<\/h3>\n<p>We resolved several issues related to logging, when Dev Proxy was processing parallel requests. Thanks to these fixes, logs are displayed properly showing all information related to the specific request together, making it easier for you to understand how Dev Proxy processed each request.<\/p>\n<h3>CRUD API plugin enhancements<\/h3>\n<p>We\u2019ve improved how the CrudApiPlugin supports CORS and returns JSON responses. With these changes, you can reliably use it in client-side applications that issue cross-domain API calls.<\/p>\n<h2>Breaking changes<\/h2>\n<p>In this version, we\u2019re also introducing two breaking changes.<\/p>\n<p>First, we removed the GraphConnectorNotificationPlugin. As deploying Graph connectors using Microsoft Teams apps has been deprecated, we removed the plugin which is no longer necessary. We still offer <a href=\"https:\/\/learn.microsoft.com\/microsoft-cloud\/dev\/dev-proxy\/technical-reference\/graphconnectorguidanceplugin\">guidance to help you build Graph connectors<\/a>.<\/p>\n<p>The second change is related to the `devproxy jwt create` command which you can use to create mock JWT tokens. In the command, we renamed the `&#8211;audience` option to `&#8211;audiences` (plural) to more accurately reflect the fact that it supports multiple audiences. Using the short `-a` alias to specify audiences is unaffected by this change.<\/p>\n<h2>There\u2019s more<\/h2>\n<p>This release also includes several bug fixes and improvements. Check out the <a href=\"https:\/\/github.com\/dotnet\/dev-proxy\/releases\/tag\/v0.28.0\">release notes<\/a> for the complete list of changes <a id=\"post-24354-_Int_aWwTP5i2\"><\/a>in this version.<\/p>\n<h2>New version of Dev Proxy Toolkit<\/h2>\n<p>Dev Proxy Toolkit is a Visual Studio Code extension that makes it easy to create and update configuration files. Alongside the new release of Dev Proxy, we\u2019ve also released a new version of the toolkit, v0.24.0.<\/p>\n<p>In this version, we\u2019ve included:<\/p>\n<ul>\n<li>New command\n<ul>\n<li>Discover URLs to watch starts Dev Proxy in discovery mode<\/li>\n<\/ul>\n<\/li>\n<li>New snippets\n<ul>\n<li>OpenAITelemetryPlugin instance<\/li>\n<li>OpenAITelemetryPlugin config section<\/li>\n<li>OpenAITelemetryPlugin prices file<\/li>\n<li>OpenAITelemetryPlugin model price<\/li>\n<\/ul>\n<\/li>\n<li>Updated all snippets to use latest v0.28.0 schemas<\/li>\n<li>Improved support for Dev Proxy Beta<\/li>\n<li>Improved Dev Proxy process detection<\/li>\n<\/ul>\n<p>Check out the <a href=\"https:\/\/marketplace.visualstudio.com\/items\/garrytrinder.dev-proxy-toolkit\/changelog\">changelog<\/a> for more information on changes and bug fixes.<\/p>\n<h2>Try it now<\/h2>\n<p>Download <strong>Dev Proxy v0.28<\/strong> and start building resilient apps and AI solutions. Thanks to <a href=\"https:\/\/github.com\/bartizan\">Artem Azaraev<\/a> for contributing to this release.<\/p>\n<p>Got feedback or ideas? <a href=\"https:\/\/github.com\/dotnet\/dev-proxy\/discussions\">Join us<\/a> and be part of the conversation.<\/p>\n<p><em>Follow us on <\/em><a href=\"http:\/\/twitter.com\/microsoft365dev\"><em>X,<\/em><\/a><a href=\"http:\/\/linkedin.com\/showcase\/microsoft365dev\"><em> LinkedIn<\/em><\/a><em>, and subscribe to our <\/em><a href=\"https:\/\/www.youtube.com\/microsoft365developer\"><em>YouTube<\/em><\/a> channel <em>to stay up to date on the latest developer news and announcements.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The latest version of Dev Proxy introduces a new ability to help you understand language models\u2019 usage and costs in your applications, alongside many improvements to mocking, TypeSpec generation, and plugin flexibility.<\/p>\n","protected":false},"author":74222,"featured_media":24358,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[289,384,326],"class_list":["post-24354","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-microsoft-365-developer","tag-dev-proxy","tag-dev-proxy-toolkit","tag-openai"],"acf":[],"blog_post_summary":"<p>The latest version of Dev Proxy introduces a new ability to help you understand language models\u2019 usage and costs in your applications, alongside many improvements to mocking, TypeSpec generation, and plugin flexibility.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts\/24354","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/users\/74222"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/comments?post=24354"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/posts\/24354\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/media\/24358"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/media?parent=24354"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/categories?post=24354"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/microsoft365dev\/wp-json\/wp\/v2\/tags?post=24354"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}