{"id":3142,"date":"2026-11-05T06:00:38","date_gmt":"2026-11-05T14:00:38","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/foundry\/?p=3142"},"modified":"2026-11-05T12:02:50","modified_gmt":"2026-11-05T20:02:50","slug":"foundry-local-agentic-coding-qwen3-8-27b","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/foundry\/foundry-local-agentic-coding-qwen3-8-27b\/","title":{"rendered":"Bring Agentic Coding On-Device with Foundry Local"},"content":{"rendered":"<p>Coding agents can dramatically accelerate development\u2014but cloud-only AI assistants can rack up costs, send sensitive context over the wire, and stall the moment your connection drops. Local inference flips that trade-off: your code and your context never leave the machine in front of you.<\/p>\n<p><strong><a href=\"https:\/\/www.foundrylocal.ai\/\">Foundry Local<\/a> brings agentic coding directly to developer hardware.<\/strong> Run Qwen3.8-27B locally, connect it to GitHub Copilot, and get a responsive coding experience with predictable inference costs, greater privacy, and the freedom to keep working offline.<\/p>\n<p><div style=\"width: 640px;\" class=\"wp-video\"><video class=\"wp-video-shortcode\" id=\"video-3142-1\" width=\"640\" height=\"360\" preload=\"metadata\" controls=\"controls\"><source type=\"video\/mp4\" src=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL_demo.mp4?_=1\" \/><a href=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL_demo.mp4\">https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL_demo.mp4<\/a><\/video><\/div><\/p>\n<p><!-- TODO: Embed demo video showing Foundry Local + GitHub Copilot --><\/p>\n<h2>Foundry Local with Qwen3.8-27B for local coding<\/h2>\n<p>Qwen3.8-27B is our primary optimization target because it pairs strong coding capabilities with a size suited to high-performance local deployment. <a href=\"https:\/\/huggingface.co\/Qwen\/Qwen3.8-27B#text-performance\">Published benchmarks<\/a> show broad capability across agentic terminal tasks, software engineering, and repository-level code generation.<\/p>\n<table>\n<thead>\n<tr>\n<th>Model \/ Benchmark<\/th>\n<th>Agentic terminal coding &#8211; Terminal Bench 2.1<\/th>\n<th>Agentic coding &#8211; SWE-bench Pro<\/th>\n<th>Repo-level code generation &#8211; NL2Repo-Bench<\/th>\n<th>Agentic coding &#8211; DeepSWE 1.1<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><strong>Qwen3.8-27B<\/strong><\/td>\n<td>73.0<\/td>\n<td>61.7<\/td>\n<td>42.3<\/td>\n<td>42.2<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<h3>Optimized end to end for developer flow<\/h3>\n<p>Powered by <a href=\"https:\/\/github.com\/microsoft\/onnxruntime-genai\">ONNX Runtime<\/a>, Foundry Local optimizes the full Qwen3.8-27B inference pipeline on Nvidia GPUs\u2014from prefill and decoding to memory management, long-context processing, and concurrent requests\u2014so local coding feels fast and responsive.<\/p>\n<p>The hardware-aware optimization stack combines a comprehensive set of inference techniques to reduce compute and memory bottlenecks, maximizing performance across diverse hardware configurations:<\/p>\n<ul>\n<li><strong>Faster inference speed:<\/strong> It leverages DFlash2 speculative decoding\u2014a technique that uses a lightweight &#8220;drafter&#8221; model to predict multiple tokens at once, breaking through the memory-bandwidth ceiling and dramatically increasing decode throughput. It also applies CUDA graph fusions and kernel optimizations tuned specifically for the latest NVIDIA SM121 GPU architecture.<\/li>\n<li><strong>Lower memory pressure:<\/strong> To fit within the memory limits of consumer GPUs, the solution combines efficient INT4 weight handling, KV-cache quantization, embedding memory sharing and CPU offloading. It also supports configurable, memory-aware inference to balance memory usage and performance across different hardware configurations. With 4-bit quantization, the model size is reduced to approximately 18 GB, making it runnable on consumer GPUs with 24 GB of VRAM, supporting context windows of up to 64K tokens. GPUs with 32 GB or more of VRAM can support longer context windows of up to the model&#8217;s maximum supported length of 262K tokens.<\/li>\n<li><strong>Long context and parallel work:<\/strong> Modern coding workflows require processing large, repository-wide contexts across multiple simultaneous requests. The serving infrastructure uses prefix caching\u2014reusing previously computed context so repeated prompts don&#8217;t start from scratch\u2014and continuous batching to process concurrent workloads efficiently.<\/li>\n<\/ul>\n<p>Together, these optimizations keep your assistant responsive as you explore a codebase, generate changes, run tools, and iterate across tasks. For model optimization instruction, see the <a href=\"https:\/\/github.com\/microsoft\/olive-recipes\/tree\/main\/Qwen-Qwen3.8-27B\/cuda\">Qwen3.8-27B optimization recipe<\/a>.<\/p>\n<h3>Performance for coding workloads<\/h3>\n<p>We compared Foundry Local with llama.cpp using the 4-bit Qwen3.8-27B model on two systems: an NVIDIA GeForce RTX 5090 with 32 GB of VRAM and an NVIDIA DGX Spark with 128 GB of unified memory. The evaluation used a dataset of real GitHub issues. Foundry Local delivered higher generation speed at every tested prompt length, with the advantage widening beyond 131K tokens\u2014exactly where repository-scale coding workflows need sustained performance.<\/p>\n<p><strong>On an NVIDIA GeForce RTX 5090 (32 GB VRAM)<\/strong><\/p>\n<p><img decoding=\"async\" class=\"aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/rtx-5090-generation-speed.webp\" alt=\"On an NVIDIA GeForce RTX 5090 (32 GB VRAM)\" width=\"1010\" height=\"627\" \/><\/p>\n<p><strong>On NVIDIA DGX Spark (128 GB unified memory)<\/strong><\/p>\n<p><img decoding=\"async\" class=\"aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/dgx-spark-generation-speed.webp\" alt=\"On NVIDIA DGX Spark (128 GB unified memory)\" width=\"982\" height=\"607\" \/><\/p>\n<h2>Start coding locally with Foundry Local in GitHub Copilot<\/h2>\n<p><span data-olk-copy-source=\"MailCompose\">Have an NVIDIA GPU with at least 24 GB of VRAM? Follow the steps below to connect Foundry Local to GitHub Copilot using Bring Your Own Key (BYOK) and start coding locally. The example below uses BYOK in the GitHub Copilot app. Instructions for using GitHub Copilot BYOK with other clients are available <a href=\"https:\/\/docs.github.com\/en\/copilot\/concepts\/models\/bring-your-own-key\">here<\/a>.<\/span><\/p>\n<p>This experience requires Foundry Local CLI 0.13.0 or later, which includes the optimizations and capabilities described above. For NVIDIA GPU execution, install the required CUDA and compatible cuDNN runtimes separately, as Foundry Local does not download these dependencies. Ensure the runtime library paths are configured correctly (add the DLL directories to <code>PATH<\/code> on Windows). Refer to the <a href=\"https:\/\/github.com\/microsoft\/foundry-local\/releases\/tag\/cli-preview-0.13.0?utm_source=chatgpt.com\">release notes<\/a> for platform-specific runtime requirements and setup instructions.<\/p>\n<ol>\n<li>Install the latest Foundry Local CLI with WinGet, then check the installed version.\n<pre><code class=\"language-batch\">winget install Microsoft.FoundryLocal\r\nfoundry --version<\/code><\/pre>\n<p>The version command should report a version of <code>0.13.0<\/code>\u00a0or higher. If Foundry Local was already running when you updated <code>PATH<\/code>, run <code>foundry server restart<\/code> from the new terminal before continuing.<\/li>\n<li>Load Qwen3.8-27B in Foundry Local and retrieve the API endpoint. <code>foundry model load<\/code> will trigger a model download if the model hasn&#8217;t been downloaded yet.\n<pre><code class=\"language-batch\">foundry model load qwen3.8-27b\r\nfoundry server status<\/code><\/pre>\n<p>Use the API endpoint returned by <code>foundry server status<\/code> on your machine, rather than copying the port shown in the screenshot.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-API-endpoint.webp\"><img decoding=\"async\" class=\"alignnone wp-image-3177\" src=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-API-endpoint-300x89.webp\" alt=\"FL API endpoint image\" width=\"549\" height=\"163\" srcset=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-API-endpoint-300x89.webp 300w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-API-endpoint-768x228.webp 768w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-API-endpoint.webp 848w\" sizes=\"(max-width: 549px) 100vw, 549px\" \/><\/a><\/li>\n<li>Connect GitHub Copilot to Foundry Local model.\n<ul>\n<li>Open Settings in GitHub Copilot app, select Model providers, choose Add provider, and select Foundry Local. <a href=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider.webp\"><img decoding=\"async\" class=\"alignnone wp-image-3179\" src=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-300x206.webp\" alt=\"FL model provider image\" width=\"1032\" height=\"709\" srcset=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-300x206.webp 300w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-1024x704.webp 1024w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-768x528.webp 768w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-1536x1057.webp 1536w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-2048x1409.webp 2048w\" sizes=\"(max-width: 1032px) 100vw, 1032px\" \/><\/a><\/li>\n<li>Enter the API endpoint from the previous step in the Base URL field.<a href=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-URL.webp\"><img decoding=\"async\" class=\"alignnone wp-image-3186\" src=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-URL-300x124.webp\" alt=\"FL model provider URL image\" width=\"688\" height=\"284\" srcset=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-URL-300x124.webp 300w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-URL-1024x422.webp 1024w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-URL-768x317.webp 768w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-provider-URL.webp 1407w\" sizes=\"(max-width: 688px) 100vw, 688px\" \/><\/a><\/li>\n<li>Search for Qwen3.8-27B in Browse Catalog and add it to complete the setup.<a href=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model.webp\"><img decoding=\"async\" class=\"alignnone wp-image-3187\" src=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-300x124.webp\" alt=\"FL model image\" width=\"680\" height=\"281\" srcset=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-300x124.webp 300w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-1024x424.webp 1024w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model-768x318.webp 768w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/FL-model.webp 1402w\" sizes=\"(max-width: 680px) 100vw, 680px\" \/><\/a><\/li>\n<\/ul>\n<\/li>\n<li>Switch the model provider to Foundry Local and select Qwen3.8-27B in the GitHub Copilot model picker. You&#8217;re now ready to start coding locally on your device.<\/li>\n<\/ol>\n<p style=\"padding-left: 40px;\"><a href=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/GHCP-model-picker.webp\"><img decoding=\"async\" class=\"alignnone wp-image-3190\" src=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/GHCP-model-picker-300x66.webp\" alt=\"GHCP 8211 model picker image\" width=\"604\" height=\"133\" srcset=\"https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/GHCP-model-picker-300x66.webp 300w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/GHCP-model-picker-1024x227.webp 1024w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/GHCP-model-picker-768x170.webp 768w, https:\/\/devblogs.microsoft.com\/foundry\/wp-content\/uploads\/sites\/89\/2026\/11\/GHCP-model-picker.webp 1252w\" sizes=\"(max-width: 604px) 100vw, 604px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>It\u2019s worth noting that local models may have limited reasoning capabilities compared with advanced cloud models due to their smaller size. For complex, high-reasoning tasks, cloud models remain a strong choice, while local models can efficiently handle everyday coding and targeted subtasks\u2014enabling a flexible hybrid workflow.<\/p>\n<h2>What\u2019s next for local coding<\/h2>\n<p>Next, we plan to expand Qwen3.8-27B support to more devices, including Macs and AMD GPUs, grow our lineup of coding-optimized local models, and deepen Foundry Local integration with GitHub Copilot for a more seamless local coding experience.<\/p>\n<p><strong>Ready to try local coding?<\/strong> Install Foundry Local, connect your endpoint to GitHub Copilot, and put Qwen3.8-27B to work on your own hardware. You get a private, cost-conscious, and responsive option for the everyday developer loop\u2014right alongside the cloud models you already use.<\/p>\n<p><strong>Resources<\/strong><\/p>\n<ul>\n<li><a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/foundry-local\/get-started?tabs=windows&amp;pivots=programming-language-javascript\">Foundry Local documentation<\/a> \u2014 full reference documentation and samples for SDK and CLI usage<\/li>\n<li><a href=\"https:\/\/github.com\/microsoft\/foundry-local\/releases#release-cli-preview-0.13.0\">Install Foundry Local CLI<\/a> \u2014 set up the CLI and run your first model in minutes<\/li>\n<li><a href=\"https:\/\/docs.github.com\/en\/copilot\/concepts\/models\/bring-your-own-key\">GitHub Copilot BYOK setup guide<\/a> \u2014 step-by-step instructions for connecting a local endpoint in GitHub Copilot<\/li>\n<li><a href=\"https:\/\/huggingface.co\/Qwen\/Qwen3.8-27B#text-performance\">Qwen3.8-27B model card<\/a> \u2014 model details, benchmarks, and licensing<\/li>\n<li><a href=\"https:\/\/github.com\/microsoft\/foundry-local\/issues\/new\/choose\">Share feedback<\/a> \u2014 report issues, request features, and connect with developers<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Foundry Local brings agentic coding directly to developer hardware.<\/p>\n","protected":false},"author":222486,"featured_media":1563,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[193,38,39,35,117],"class_list":["post-3142","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-microsoft-foundry","tag-coding","tag-foundry-local","tag-github-copilot","tag-local-ai","tag-on-device"],"acf":[],"blog_post_summary":"<p>Foundry Local brings agentic coding directly to developer hardware.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/posts\/3142","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/users\/222486"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/comments?post=3142"}],"version-history":[{"count":3,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/posts\/3142\/revisions"}],"predecessor-version":[{"id":3218,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/posts\/3142\/revisions\/3218"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/media\/1563"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/media?parent=3142"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/categories?post=3142"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/tags?post=3142"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}