Evaluating AI agents for NL-to-SQL generation across Azure Databricks AI/BI Genie, GitHub Copilot CLI, and Microsoft Agent Framework. We achieved ~75% accuracy with schema documentation and runtime validation, while discovering that business logic errors represent a fundamental limitation requiring domain expertise.
Learn how to streamline AI development by using Microsoft Devtunnels to connect local services with Azure Machine Learning evaluation pipelines, eliminating deployment delays while maintaining comprehensive cloud-based validation.
This blog details how the Azure AI Evaluation SDK can be used to assess the performance of a small language model for function calling, such as Phi-4-mini-instruct, and view the results in Microsoft Foundry.
This blog post introduces a comprehensive evaluation framework for enterprise chatbots powered by large language models (LLMs), specifically addressing the challenges of assessing Line of Business (LOB) agents in business-critical environments. The authors tackle the fundamental problem that traditional chatbot evaluation metrics fail to capture th...
Steps to Produce High Quality Ground Truth Pairs for AI Systems
This post presents a hierarchical waterfall framework for evaluating query classification, retrieval, and generation in multi-agent LLM systems.
data science and software engineering teams can choose the best tools for their respective roles in delivering machine learning models to production.
How to manage multiple independent python projects, with different dependencies, inside Visual Studio Code.
This blog post explores the development and evaluation of an AI assistant that converts natural language into robotic code.
Azure Machine Learning is known for training and deploying models, but can also be used for running experiments. This blog post will show us how we have implemented our Evaluation platform on Azure Machine Learning.
