{"id":2415,"date":"2026-06-03T09:00:43","date_gmt":"2026-06-03T16:00:43","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/foundry\/?p=2415"},"modified":"2026-06-03T09:29:24","modified_gmt":"2026-06-03T16:29:24","slug":"agent-optimizer-build2026","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/foundry\/agent-optimizer-build2026\/","title":{"rendered":"Introducing Agent Optimizer in Foundry Agent Service"},"content":{"rendered":"

With\u00a0hosted agents<\/em><\/a>, we made it straightforward to build and deploy agents on Foundry. You write your logic, run azd deploy, and your agent is live. But \u201clive\u201d and \u201cproduction-ready\u201d aren\u2019t the same thing.<\/p>\n

The gap shows up quickly. Your customer support agent handles requests, but it forgets to ask for an order number before looking up status. It answers warranty questions without checking the purchase date. It gives electrical wiring advice when it should decline and recommends a professional. Each fix means rewriting your system prompt, testing by hand, and hoping you didn\u2019t break something else in the process.<\/p>\n

For one agent, that\u2019s manageable. For a team running ten agents across different domains, it\u2019s a bottleneck that doesn\u2019t scale. We heard this from developers consistently:\u00a0the hard part isn\u2019t building the agent, it\u2019s getting the agent to behave correctly across all the scenarios it needs to handle.<\/em><\/p>\n

Today we\u2019re excited to introduce the\u00a0agent optimizer in Foundry Agent Service<\/strong>, in private preview and out in public preview in 30 days.<\/p>\n

To sign up for private preview here\u00a0aka.ms\/Agent-Optimizer-Private-Preview<\/a>.<\/p>\n

What is the agent optimizer in Foundry Agent Service?<\/h2>\n

Agent optimizer evaluates your hosted agent against defined criteria, generates better configurations, and ranks the results so you can deploy the best one. It automates the improvement loop that most teams do by hand today.\u00a0Here\u2019s how it works. The optimizer runs a closed-loop cycle:<\/p>\n

    \n
  1. Evaluate the baseline.<\/strong>\u00a0Your agent processes a set of tasks, each with explicit pass\/fail criteria. The result is a composite score from 0.0 to 1.0.<\/li>\n
  2. Generate candidates.<\/strong>\u00a0Guided by what failed, the optimizer produces new configurations. You choose the optimization target: instruction rewrites your system prompt, skill generates reusable procedures, or model finds the best deployment for your quality\/cost trade-off.<\/li>\n
  3. Evaluate candidates.<\/strong>\u00a0Each candidate runs against the same task set.<\/li>\n
  4. Rank and recommend.<\/strong>\u00a0Results are sorted by score. You see per-task breakdowns and token costs for each candidate before you commit.<\/li>\n
  5. Deploy the winner.<\/strong>\u00a0One command promotes the winning configuration to your live agent<\/li>\n<\/ol>\n

    \"optimization<\/a><\/p>\n

    The entire process runs in the cloud. Start it with\u00a0azd ai agent<\/code>\u00a0optimize and a typical run completes in a few minutes. You don\u2019t need to provision any additional infrastructure. If you have a hosted agent deployed, you\u2019re ready to optimize.<\/p>\n

    Developer experience<\/h2>\n

    Agent optimizer was made for developers making it easier to take their agents into production with confidence. Here\u2019s what this looks like in practice. Say you have a customer support agent for a consumer electronics company. Your current system prompt is bare: “You are a helpful customer support agent.<\/em>”<\/p>\n

    $ azd ai agent optimize\r\n \r\nOptimizing agent \"travel-approver\"...\r\n  Baseline saved to .agent_configs\\baseline\\metadata.yaml\r\n  Job ID: opt_999f814ed77a\u2026.\r\n  Status: pending\r\n  Portal: https:\/\/ai.azure.com\/nextgen\/....\r\n  \u2807 completed \u00b7 iteration 2 \u00b7 score: 1.00 \u00b7 9m0s\r\nResults:\r\n  Candidate              Score    Pass  Eval\r\n  \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u2500\u2500\u2500\u2500\u2500\u2500\u2500  \u2500\u2500\u2500\u2500\u2500\u2500\r\n  baseline                0.60     71%  View\r\n  candidate_1 \u2605           0.92    100%  View\r\n\r\n  Candidate IDs:\r\n      baseline             cand_200345f6c7\u2026\r\n    \u2605 candidate_1          cand_300d8e4e3\u2026\r\n\r\n  Apply the best candidate locally, then deploy:\r\n    azd ai agent optimize apply --candidate cand_300d8e4e3\u2026\r\n    azd deploy<\/code><\/pre>\n
    From 0.60 to 0.92. No model retraining. No code changes. Using synthetic data or historical traces of how your agent performed and evaluator signals that identified where it fell short, the optimizer rewrote the system prompt\/skills\/tools to strengthen return policies, escalation procedures, troubleshooting frameworks, and safety boundaries. The changes were driven by observed behavior and scored against the criteria you defined.<\/p>\n
    Get your agent optimizer-ready<\/h3>\n
    Create a\u00a0.agent_configs\/baseline\/\u00a0directory in your agent project. At minimum, you need an\u00a0instructions.md\u00a0file that captures your agent’s system prompt:<\/p>\n
    .agent_configs\/\r\n \u2514\u2500\u2500 baseline\/\r\n     \u251c\u2500\u2500 metadata.yaml       # points to your config files\r\n     \u251c\u2500\u2500 instructions.md     # your agent's system prompt\r\n     \u251c\u2500\u2500 skills\/             # skill definitions (optional)\r\n     \u2502   \u2514\u2500\u2500 <skill_name>\/\r\n     \u2502       \u2514\u2500\u2500 SKILL.md\r\n     \u2514\u2500\u2500 tools.json          # tool definitions in OpenAI function-calling format (optional)<\/code><\/pre>\n
    The integration is a load_config()<\/code> call at startup. It reads the optimized configuration when the service injects one during evaluation, and falls back to your defaults in normal operation.<\/p>\n
    from azure.ai.agentserver.optimization import load_config\r\n   \r\nconfig = load_config()  # baseline or optimized agent config\r\n   \r\n# includes skill catalog if skills were generated\r\ninstructions = config.compose_instructions()\r\n   \r\n# Model optimization \u2014 optimizer tries different models from a candidate list\r\nmodel = config.model or \"gpt-4o\"\r\n   \r\n# Tool description optimization \u2014 rewrites tool docstrings for better function-calling\r\ntools = [my_search_tool, my_calculator_tool]\r\nconfig.apply_tool_descriptions(tools)\r\n   \r\nagent = create_agent(\r\n  system_prompt=instructions,\r\n  model=model,\r\n  tools=tools,\r\n)<\/code><\/pre>\n
    That\u2019s it. Your agent works with or without optimization. No feature flags, no conditional logic. When the optimizer evaluates your agent, it injects candidate configurations through an environment variable. In production, that variable is absent and your defaults apply.<\/p>\n
    What gets optimized?<\/h2>\n
    Agent optimizer gives you three targets. You can use them individually or combine them in a single run.<\/p>\n
    Instruction<\/strong>\u00a0is the default target. The optimizer analyzes where your agent\u2019s responses fall short, then generates alternative system prompts that address those gaps. For our customer support agent, the optimizer added return window details, warranty coverage specifics, and clear boundaries around electrical and medical advice.<\/p>\n
    Skill<\/strong>\u00a0generates reusable, named procedures that get appended to your agent\u2019s instructions. Think of them as playbooks: an escalation procedure, a troubleshooting sequence, a formatting template. Each skill has a name, description, and implementation body that the agent follows when the situation matches. Use this when your agent needs repeatable multi-step behaviors that a single prompt rewrite can\u2019t capture.<\/p>\n
    Model<\/strong> lets the optimizer evaluate your agent across multiple model deployments in the same run. If you’re wondering whether gpt-5-mini handles your workload as well as gpt-5, or whether stepping up to gpt-5.4 gives you a meaningful quality bump on the dimensions that matter to your agent, the optimizer scores each option against your evaluators and shows you which one produces better responses. You pick the model based on what actually performs, not on gut feel.<\/p>\n
    Tool Descriptions<\/strong> lets the optimizer improve how your agent understands and uses its local function tools. It rewrites tool descriptions and parameter definitions so the agent picks the right tool more reliably. For a customer support agent, the optimizer might clarify when to call an order lookup tool versus a knowledge base search, tighten parameter requirements, define fallback behavior when a tool fails, or specify situations where the agent should answer directly instead of making a call. Today this covers the tools defined in your agent’s own tool set (not external tools like MCP servers); the optimizer refines what’s already wired up rather than reaching beyond the agent boundary.<\/p>\n
    azd<\/span> ai agent optimize   # auto detects the targest according to the provided agent config<\/code><\/pre>\n
    You can also combine targets in a configuration file to run them all at once.<\/p>\n
    From evaluation to action<\/h2>\n
    Foundry already gives you observability and evaluation for your agents. Tracing captures every interaction.\u00a0Agent evaluation<\/em><\/a>\u00a0scores your agent\u2019s behavior against quality criteria. Agent optimizer is where you take action on what those systems tell you.\u00a0When paired with Foundry\u2019s observability and evaluation stack, the improvement cycle becomes:<\/p>\n
      \n
    1. Your agent runs in production. Traces capture every interaction.<\/li>\n
    2. Evaluation scores behavior against defined criteria.<\/li>\n
    3. Agent optimizer generates better configurations based on what the evaluations surface.<\/li>\n
    4. You deploy the winning candidate as a new versioned hosted agent.<\/li>\n
    5. Evaluation runs again to confirm improvement.<\/li>\n<\/ol>\n
      Starting from zero.<\/strong>\u00a0Most teams don\u2019t have evaluation datasets ready on day one. We\u2019ve seen this over and over. The eval init command solves the cold-start problem by generating both a dataset and evaluation criteria from your agent\u2019s existing instructions:<\/p>\n
      $ azd ai agent eval init\r\n \r\n Eval suite created\r\n   Dataset:    customer-support (2.0), 15 tasks\r\n   Evaluator:  customer-support (1)\r\n \r\n   Evaluator dimensions (6):\r\n     Weight  Dimension\r\n     \u2500\u2500\u2500\u2500\u2500\u2500  \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\r\n         10  policy_compliance\r\n          6  resolution_accuracy\r\n          5  troubleshooting_structure\r\n          4  communication_clarity\r\n          3  safety_boundaries\r\n          5  general_quality<\/code><\/pre>\n
       <\/p>\n
      No manual test-writing required. The system looks at what your agent is supposed to do and generates tasks that test whether it actually does it. For our customer support agent, it produced tasks around order inquiries, returns, warranty claims, troubleshooting, escalation, and out-of-scope requests. Each task has specific pass\/fail criteria tied to the agent\u2019s responsibilities.<\/p>\n
      This connects to the broader evaluation capabilities in Foundry:<\/p>\n