{"id":2707,"date":"2026-06-18T13:37:18","date_gmt":"2026-06-18T20:37:18","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/foundry\/?p=2707"},"modified":"2026-06-18T15:15:30","modified_gmt":"2026-06-18T22:15:30","slug":"outcome-driven-learning-systems-enterprise-rl-with-openenv-and-foundry","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/foundry\/outcome-driven-learning-systems-enterprise-rl-with-openenv-and-foundry\/","title":{"rendered":"Outcome-driven learning systems: Enterprise RL with OpenEnv and Foundry"},"content":{"rendered":"

We shipped a lot at Build 2026<\/a>: hosted agents, Toolboxes, Foundry IQ, Memory, Managed Compute, fine\u2011tuning, Frontier Tuning, and a new evaluation and optimization stack. Read as a feature list, it is a lot to hold in your head. So here is a simpler way to see it: these are the parts you need to build a learning system<\/strong>, with agents that get measurably better at your work over time, not a chatbot that answers once and forgets. This post is about assembling those parts into one loop you own, and the science that makes a small, owned model worth training.<\/p>\n

It builds on two pieces worth reading first. Jay Parikh\u2019s \u201cAI alone won\u2019t change your business, the system running it will\u201d<\/a> argues that the winners do not just adopt a model; they stand up a governed system that improves the longer it runs. Satya\u2019s framing in \u201ca frontier ecosystem, not just a frontier model\u201d<\/a> sharpens it: the durable asset is not the model you rent, it is the learning loop you own<\/strong>. Same idea from two angles. Build a system that improves against your outcomes, and make sure you own it.<\/p>\n

Foundry makes that loop something you can build in an open, interoperable, and modular<\/strong> way, so you can swap any piece (the model, the trainer, the tools) without rebuilding the whole thing. Two ingredients sit under every such system: a place for the agent to practice<\/em> (an environment<\/strong>) and a way to judge<\/em> it (an eval<\/strong>). To keep both open, Microsoft joined the OpenEnv<\/strong><\/a> community.<\/p>\n

\"The<\/a>
The hill-climbing loop.<\/strong> A hosted agent<\/strong> (your harness, the Microsoft Agent Framework, plus a swappable model) runs in a per-session ACA Sandbox<\/strong>; tracing<\/strong> captures every run and your rubric<\/strong> judges it. Then it learns two ways: non-parametric<\/strong> (Agent Optimizer + SkillOpt) and parametric<\/strong> (Foundry post-training with Tinker + ECHO on OpenEnv). Keep what wins, then ship the better agent to Teams, Microsoft 365, and any other channel<\/strong>. Grounded by Foundry IQ + Memory + Toolbox<\/strong>. The model is swappable; the learning stays yours<\/strong>.<\/figcaption><\/figure>\n

 <\/p>\n

However you come to this, whether PM, IT admin, developer, or AI engineer, the first half is the plain-language what<\/strong> and why<\/strong>. The second half is the deep science. Skim or dive as suits you.<\/em><\/p>\n

\n

Environments, evals, and rubrics, in plain language<\/h2>\n

An environment<\/strong> (an RLE, or reinforcement-learning environment) is a practice space<\/em> for your agent (harness + model). It is your real workflow and your standard operating procedure, codified so an agent can act inside it: the steps, the tools it is allowed to use, the data it sees. Think of it as a flight simulator for one of your business processes, close enough to the real thing that getting good in the simulator means getting good at the job.<\/p>\n

An eval<\/strong> is how you judge<\/em> a result, and the heart of an eval is a rubric<\/strong>: a clear, scored definition of \u201cdone right\u201d for your<\/em> outcome, not a public leaderboard. \u201cDid it reconcile the invoice to the contract? Did it cite a real clause? Did it stay inside policy?\u201d Foundry ships agent evaluation<\/strong><\/a> for writing exactly these judgments, and an optimizer (below) for acting on the scores.<\/p>\n

Here is the move that ties it together: an environment already contains its eval.<\/strong> Codify your workflow plus<\/em> your outcome rubric, and you have not just written a test, you have built a hill-climbing space<\/strong>. The agent practices, the rubric scores, and the system climbs toward your outcome. That is why RLEs are also evals: it is one artifact that both exercises the agent and grades it.<\/p>\n

\n

Codify your workflow and your outcome into an environment, and the model becomes a part you can swap. The expertise lives in the loop you own, so the learning stays yours.<\/p>\n<\/blockquote>\n

\n

A system learns in two ways<\/h2>\n

Before any science, the single most useful idea in this post: a system can get better in two different ways, and you should reach for them in order.<\/p>\n

1. Non-parametric learning (the weights stay frozen)<\/h3>\n

The first kind leaves the model\u2019s weights untouched and improves the harness<\/strong> around them: the system prompt, the skills<\/strong> (named, reusable procedures), the tool descriptions, and the context the agent retrieves through Foundry IQ and Memory<\/strong>, plus which model it runs on. No GPUs, no training run, results in minutes. Foundry ships this as the agent optimizer<\/strong><\/a>: it runs a closed loop (evaluate your agent against your criteria, generate better configurations, score them, deploy the winner) and will rewrite your instructions, synthesize skills, sharpen tool descriptions, or pick a better model for your quality and cost trade-off. In our example a bare support agent climbs from 0.60 to 0.92<\/strong> on its rubric with no retraining and no code changes, just a smarter prompt, skills, and tools.<\/p>\n

Microsoft Research is pushing harness optimization even further. SkillOpt<\/strong><\/a> treats the skill document itself as the trainable thing: it edits a single Markdown skill from scored rollouts and accepts an edit only<\/em> when a held-out validation score strictly improves, the same discipline that makes weight training reproducible, but with zero weight changes and zero extra calls at inference time. The deployable artifact is a compact best_skill.md<\/code> that runs against an unchanged model, and it lifts no-skill accuracy by more than 20 points<\/strong> on a frontier model across six benchmarks (paper<\/a>). Start here. Non-parametric learning is cheap, fast, and often enough on its own.<\/p>\n

2. Parametric learning (the weights change)<\/h3>\n

The second kind is for when you want the behavior in the model itself<\/em>: faster and cheaper to serve, and fully owned (sovereign). You change the weights with post-training<\/strong>. This is where a small open model can quietly overtake a frontier API on your task, and where the deepest new science lives, because you can teach the model not just what to do<\/em> but how its world responds<\/em>. That technique is ECHO<\/strong>, and most of the rest of this post is about it. The point to carry forward: do the cheap non-parametric learning first, and turn to parametric learning when the outcome, and the economics, justify owning the model.<\/p>\n

\n

One agent, one sandbox, both kinds of learning<\/h2>\n

Both kinds of learning run against the same thing: a hosted agent<\/strong><\/a> (your model plus prompt plus tools and skills) executing in an isolated, project-owned sandbox. Foundry runs hosted agents on Azure Container Apps (ACA) sandboxes<\/strong><\/a>: each gets its own filesystem, session, and state, with default-deny networking so a tool call cannot quietly exfiltrate a secret. The agent optimizer drives that hosted agent through its evaluation loop inside exactly this sandbox.<\/p>\n

And Microsoft\u2019s contribution to OpenEnv (more in a moment) makes the very same ACA sandbox an OpenEnv environment<\/em>, so the agent you optimize non-parametrically is the agent you post-train parametrically, in the same secure box. One agent, one sandbox, two ways to climb. The diagram at the top of this post shows the whole loop on a page.<\/p>\n

Mapped to what shipped: Toolboxes<\/strong><\/a> give the agent one governed set of tools; ACA sandboxes<\/strong> give it an isolated place to run; Foundry IQ<\/strong><\/a> is the knowledge plane that grounds it; agent evaluation<\/strong> is the rubric; the agent optimizer<\/strong><\/a> and post-training<\/strong> are the two improve steps; Managed Compute<\/strong><\/a> serves the result. One open standard underneath keeps the parts from locking you in: OpenEnv.<\/p>\n

We joined OpenEnv, with contributions for enterprise agentic learning loops<\/h3>\n

OpenEnv<\/a> is a protocol<\/em> for environments: a small, shared contract (reset<\/code>, step<\/code>, state<\/code>, carried over MCP, packaged with Docker, with the promise that the training environment matches production). It is not a reward framework and not a trainer; it is the thin interoperable layer that lets any model, harness, trainer, and environment compose. That is the interoperable in open-and-interoperable, and it is why Microsoft joined the community alongside Meta\u2019s PyTorch team, NVIDIA, Hugging Face, Unsloth, Prime Intellect, and others.<\/p>\n

Two of those contributions are already merged into OpenEnv<\/a>: a hosted Azure Container Apps sandbox provider<\/strong><\/a>, so an RLE can run rollouts in the isolated, project-owned Azure sandbox above, with default-deny egress that blocks token theft, the enterprise-grade isolation an RLE needs; and ECHO env-token world-modeling as RFC 010<\/strong><\/a>, which teaches trainers to learn from the environment\u2019s own tokens. Private RLEs and private evals, kept open and interoperable on purpose with more to come.<\/p>\n

\"Openenv<\/a><\/p>\n

\n

Post-training, without the heavy lifting<\/h2>\n

Before the frontier technique, the basics, because owning the weights is far more approachable than it used to be. Fine-tuning a small model on your task and serving it once meant standing up GPU clusters and a training stack. Foundry\u2019s managed post-training removes that: it exposes a Tinker-style training loop<\/strong>, the low-level primitives sample<\/code>, forward_backward<\/code>, and optim_step<\/code>, running server-side on Foundry\u2019s GPUs while you keep the data and the loop. You write the loop; the service owns the hardware. No client GPUs, no cluster to babysit. Two Build sessions walk it end to end: BRK231, Deploy. Observe. Learn<\/strong><\/a> and BRK232, Post-Training and Deploying Open-Source Reasoning Models in Foundry<\/strong><\/a>.<\/p>\n

That same managed loop is where Microsoft is pushing the frontier: not just consuming the stack but advancing it and contributing the pieces back to OpenEnv (the ACA sandbox provider, and the world-modeling work in the next section), so the whole ecosystem benefits. The clearest example is next. It turns the wasted half of every rollout into a free training signal, and it lands as a one-line change on exactly this loop.<\/p>\n

\n

Pushing OpenEnv environments to the frontier<\/h2>\n

Joining OpenEnv is not a logo exercise. An open standard stays relevant only if it keeps absorbing the research frontier, so part of our work is diffusing that frontier into it. The clearest example is a contribution we landed as a pull request, ECHO world-modeling (RFC 010)<\/strong><\/a>, which brings a Microsoft Research result (\u201cTerminal Agents Learn World Models for Free\u201d<\/a>) into OpenEnv, where any team can pick it up. That is how a lab technique becomes a shared capability and the learning loop gets democratized.<\/p>\n

Here is what it does. An agent transcript is half actions<\/strong> (what the model writes) and half observations<\/strong> (what the environment writes back). Standard agent-RL trains the actions and masks the observations away. ECHO keeps them: a small cross-entropy term that makes the policy predict the environment\u2019s own tokens, a world model, from logits it already computed in the same forward pass. No extra rollouts, no teacher, no labels.<\/p>\n

  L  =  L_GRPO(action tokens)   +   \u03bb \u00b7 CrossEntropy(observation tokens)<\/pre>\n
\"The<\/a>
ECHO in one step.<\/strong> One rollout, split by per-token role: actions get the RL loss, observations get a \u03bb-weighted cross-entropy loss, summed into a single optimizer step. \u03bb = 0 is vanilla RL, so it is safe to adopt incrementally.<\/figcaption><\/figure>\n
The free signal is large and real: on a captured agent episode, 4,659 of 5,247 learnable tokens (89%) are environment observations<\/strong>, 7.9\u00d7 the action tokens, exactly the half standard agent-RL discards. Prime Intellect reaches the same conclusion in \u201cTrue Agents Model the World\u201d<\/a>, restating supervised learning on tool-response tokens as RL with a constant positive advantage, foldable in at no extra cost. Two groups, one direction: world-modeling belongs inside the RL loop, not bolted on afterward.<\/p>\n
On the honest ablation (\u03bb on versus off), the training reward barely moves; the gain is generalization<\/strong>. ECHO\u2019s published results: held-out pass@1 roughly doubles<\/strong> on TerminalBench-2.0, RL reaches its target about 2.3\u00d7 faster<\/strong>, and it recovers 50 to 104% of expert-SFT with no teacher<\/strong>. Even verifier-free (reward off), held-out tasks improve. Keep \u03bb small and sweep it; the dense signal overfits if pushed (one open model collapsed at 0.05 and was stable at 0.005).<\/p>\n
 <\/p>\n
\"Bar<\/a>
What the weight update buys.<\/strong> Same training reward; held-out pass@1 roughly doubles. ECHO also reports about 2.3\u00d7 faster RL and 50 to 104% of expert-SFT recovered with no teacher (arXiv 2605.24517<\/a>, microsoft\/echo-rl on SkyRL; corroborated by Prime Intellect).<\/figcaption><\/figure>\n
You can watch the mechanism on a laptop in about 40 seconds: a small model on a deterministic toy terminal env drives held-out env-token cross-entropy toward zero. It reaches zero only because that world is fully predictable; a real environment keeps its irreducible entropy (near 4.4 nats), so ECHO sharpens predictions rather than perfecting them. Reproduce it: OpenEnv examples\/echo_world_model<\/code><\/a>, python train_echo.py --steps 60 --seed 0<\/code>.<\/p>\n
\"Reproduction<\/a>
Reproduce it on CPU.<\/strong> A toy, fully deterministic terminal env, so cross-entropy can approach zero; a real env keeps its irreducible entropy instead. The held-out line bottoms near step 40 and then mildly overfits, which is why \u03bb stays small. Run it: OpenEnv examples\/echo_world_model<\/code><\/a>, python train_echo.py --steps 60 --seed 0<\/code>.<\/figcaption><\/figure>\n
And it holds on the managed path. Because supervised learning on the observation tokens is just RL with a constant positive advantage, there is no second loss function: you reuse the same forward_backward<\/code> and add a small positive advantage \u03bb on the environment tokens. One vector changes, and the same one-line config runs on the open SkyRL reference, Tinker, and Foundry post-training unchanged. We ran it live with a small Qwen model, and it also runs with MAI-Reasoning-1-Flash<\/strong>; the backend metrics came back namespaced skyrl.ai<\/code>, the open reference stack running underneath the managed service.<\/p>\n
\n
The loop that improves itself: RSI<\/h2>\n
One last reason to own the environment, not just the model: the gym is where compounding begins. Once your workflow, tools, and rubric live in an OpenEnv RLE, the same trace data that post-trains the model can also improve the environment<\/em> itself. The OpenEnv roadmap points squarely at this, a family of self-improving-gym designs: curricula that generate harder tasks as the agent gets better, harness optimizers, and new environments built automatically from captured production traces. That is recursive self-improvement (RSI) in action. The system writes its own next set of exercises, and each cycle sharpens the next. The learning does not only accrue in the weights; it accrues in the gym, which is the part you own.<\/p>\n
\n
Start building the loop that is yours<\/h2>\n
Pull it back up to the top. Codify your workflow and your outcome into an OpenEnv-compatible RLE, and you have a hill-climbing learning system that is genuinely yours: open, interoperable, and outcome-driven. Improve it the cheap way first (tune the prompt, skills, tools, and model choice with the agent optimizer<\/a> and ideas from SkillOpt<\/a>), then the deep way when the economics justify it (post-train the weights, with ECHO turning the discarded 89% of your trajectories into a free world model). The model in the middle is a part you can swap; the loop around it is the asset that compounds the longer it runs.<\/p>\n
The managed on-ramp is Frontier Tuning<\/strong><\/a>: frontier-grade performance with superior token efficiency, improved through real usage in Foundry and Copilot, and secured inside your own environment. Early adopters including McKinsey<\/strong>, Bristol Myers Squibb<\/strong>, and Land O\u2019Lakes<\/strong> are already building with it.<\/p>\n
The fastest way in is a partnership. Ask your Microsoft Forward-Deployed Engineer (FDE) or your Microsoft account team to engage<\/strong>, and build the OpenEnv-compatible RLEs and outcome-driven learning systems where the model is swappable and the learning stays yours.<\/p>\n
Build it on Foundry:<\/strong> Frontier Tuning<\/a> \u00b7 Agent Optimizer<\/a> \u00b7 Hosted Agents<\/a> \u00b7 Foundry IQ<\/a> \u00b7 Toolboxes<\/a> \u00b7 Managed Compute<\/a> \u00b7 What\u2019s new at Build 2026<\/a><\/p>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
We shipped a lot at Build 2026: hosted agents, Toolboxes, Foundry IQ, Memory, Managed Compute, fine\u2011tuning, Frontier Tuning, and a new evaluation and optimization stack. Read as a feature list, it is a lot to hold in your head. So here is a simpler way to see it: these are the parts you need to […]<\/p>\n","protected":false},"author":172657,"featured_media":2712,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[141,112,163,1,27],"tags":[],"class_list":["post-2707","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-agent-optimizer","category-foundry-agent-service","category-microsoft-build","category-microsoft-foundry","category-whats-new"],"acf":[],"blog_post_summary":"
We shipped a lot at Build 2026: hosted agents, Toolboxes, Foundry IQ, Memory, Managed Compute, fine\u2011tuning, Frontier Tuning, and a new evaluation and optimization stack. Read as a feature list, it is a lot to hold in your head. So here is a simpler way to see it: these are the parts you need to […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/posts\/2707","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/users\/172657"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/comments?post=2707"}],"version-history":[{"count":2,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/posts\/2707\/revisions"}],"predecessor-version":[{"id":2725,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/posts\/2707\/revisions\/2725"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/media\/2712"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/media?parent=2707"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/categories?post=2707"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/foundry\/wp-json\/wp\/v2\/tags?post=2707"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}