Dev Blogs
ISE hackathon produced a reusable Japanese LLM evaluation pipeline with multi-turn, facet-based scoring, cross-provider model support, and publication-ready benchmarking outputs.
A multimodal LLM enrichment pipeline that extracts structured metadata from visual assets, constrains output to predefined values to minimize hallucinations, and uses a ground truth evaluation template to measure quality—all as a plug-and-play module.
How query decomposition separates metadata filters from visual intent to significantly improve media asset retrieval quality.
When distributing LLM calls across PySpark workers via mapInPandas, MLflow autolog silently fails. Here is how to fix it.
Evaluating AI agents for NL-to-SQL generation across Azure Databricks AI/BI Genie, GitHub Copilot CLI, and Microsoft Agent Framework. We achieved ~75% accuracy with schema documentation and runtime validation, while discovering that business logic errors represent a fundamental limitation requiring domain expertise.
Learn how to streamline AI development by using Microsoft Devtunnels to connect local services with Azure Machine Learning evaluation pipelines, eliminating deployment delays while maintaining comprehensive cloud-based validation.
This blog details how the Azure AI Evaluation SDK can be used to assess the performance of a small language model for function calling, such as Phi-4-mini-instruct, and view the results in Microsoft Foundry.
This blog post introduces a comprehensive evaluation framework for enterprise chatbots powered by large language models (LLMs), specifically addressing the challenges of assessing Line of Business (LOB) agents in business-critical environments. The authors tackle the fundamental problem that traditional chatbot evaluation metrics fail to capture th...
Steps to Produce High Quality Ground Truth Pairs for AI Systems
This post presents a hierarchical waterfall framework for evaluating query classification, retrieval, and generation in multi-agent LLM systems.