ISE Developer Blog
Solving global tech challenges, sharing insights, and empowering developers
Latest posts
How we Decide Between Keyword and Hybrid Search: 5 Enterprise Evaluation Criteria
A data-driven framework we use in enterprise deployments to decide between vector-only keyword and hybrid search, based on five measurable evaluation criteria.
Verification-driven tooling prompts for fast-moving codebases
A practical pattern for using GenAI research agents to produce reliable internal tooling by verifying assumptions with detection classes and a final checklist.
Coordinating AI-Assisted Development with AGENTS.md and Skills
How to move AI-assisted development from ad-hoc experimentation to a coordinated team-wide capability using AGENTS.md and reusable skills with GitHub Copilot CLI
WebAssembly Data Processing at the Edge with Azure IoT Operations
Safe, composable dataflow operators for Azure IoT Operations, built as WASM modules using the Component Model, WIT interfaces, and WASI Preview 2.
SQL query generation from natural language
Evaluating AI agents for NL-to-SQL generation across Azure Databricks AI/BI Genie, GitHub Copilot CLI, and Microsoft Agent Framework. We achieved ~75% accuracy with schema documentation and runtime validation, while discovering that business logic errors represent a fundamental limitation requiring domain expertise.
Propagating SharePoint Document Permissions to AI Search and RAG Pipelines
A security-first architecture that propagates SharePoint document permissions into downstream AI systems, using Microsoft Entra ID object IDs (GUIDs) for safe, query-time filtering in Azure AI Search, RAG pipelines, and Copilot extensions.
Lessons Learned from Building a Well-Matching Intelligence Layer
Key insights from the EDA and Ground Truth journey for solving well identity resolution in the Energy industry—challenges that extend to any entity matching problem across distributed systems.
Discoverable – Observable MCP Server
How to build an observable MCP Server with discoverable tools and agents.
Building Search-Enabled Agents with Azure AI Foundry and Semantic Kernel and A2A
A step-by-step guide to enable search in the semantic kernel agents using bing grounding in Azure Foundry AI Agent using A2A.
From Azure IoT Operations Data Processor Pipelines to Dataflows
In this post we explore the evolution from Azure IoT Operations Data Processor Pipelines to Dataflows, why we adopted a hybrid strategy with custom Rust pods, and the architectural lessons we learned building event detection systems at the edge.
Using Agents to Setup Experiments
When setting up an experiment is complex, agents can automate the process to make it faster and more reliable.
Building a Secure MCP Server with OAuth 2.1 and Azure AD: Lessons from the Field
How we built a production-ready MCP server with OAuth 2.1 authentication and On-Behalf-Of flow for Microsoft Graph, navigating a rapidly evolving specification.
Using Codes to Increase Adherence to Prompts
Agentic systems have some discretion in the parameters they sent to tooling, but there are cases, such as experimentation, when you need 100% adherence to a set of parameters.
Minimal GitOps for Edge Applications with Azure IoT Operations and Azure DevOps
How we built a minimal, scalable GitOps workflow for edge applications using Azure IoT Operations, Azure DevOps, and a multi-repo strategy. Includes pipeline templates and scripts for real-world deployments.
Bridging Local Development and Cloud Evaluation: Using Microsoft Devtunnels with Azure Machine Learning
Learn how to streamline AI development by using Microsoft Devtunnels to connect local services with Azure Machine Learning evaluation pipelines, eliminating deployment delays while maintaining comprehensive cloud-based validation.
Evaluate Small Language Model Function Calling using the Azure AI Evaluation SDK
This blog details how the Azure AI Evaluation SDK can be used to assess the performance of a small language model for function calling, such as Phi-4-mini-instruct, and view the results in Microsoft Foundry.
Introducing the Copilot Studio + Azure AI Search Solution
Introduction to a scaleable and secure turnkey architecture for deploying Copilots connected to Azure AI Search.
Agent Onboarding Process for Agentic Systems: Maintain accuracy at scale
A robust agent onboarding process is the backbone of scalable, reliable multi-agent AI systems—enforcing uniqueness, rigorous evaluation, and responsible AI practices at every step.
Accelerating AI Development with GitHub Copilot: Real-World Use-Cases
This blog post shares real-world experiences from Microsoft's ISE team using GitHub Copilot to accelerate AI development workflows, demonstrating how AI tools can significantly speed up tasks like creating documentation, class diagrams, data visualizations, and demo scripts when used by experienced developers.
Ensuring Seamless User Experience in Modernization Projects with AI‑Generated Stagehand and Playwright UI Tests
Using AI‑generated tools like Stagehand with Microsoft's Playwright helps ensure a smooth user experience during modernization efforts such as Hyper Velocity Engineering and monolith-to-microservices migrations.
Patterns for Building a Scalable Multi-Agent System
Practical patterns for designing scalable, high-performing multi-agent systems—grounded in real implementation experience.
Tuning and Optimization of Speech-to-Text (STT), Text-to-Speech (TTS), and Custom Keyword Recognition in Azure Speech Services
This blog outlines best practices for optimizing Speech-to-Text (STT), Text-to-Speech (TTS), and Custom Keyword Recognition in Azure Speech Services, helping developers build more accurate and responsive voice-enabled applications.
Integration Testing with Testcontainers
How to leverage Testcontainers for integration testing
Managing secrets on Azure KeyVault with a Tagging strategy to perform automations
Cloud and Infrastructure teams can manage secrets on Azure KeyVault with a Tagging strategy to perform automations.
Working with Git Submodules: Managing Dependencies Across Repositories
Working with Git Submodules and creating actions.
Leveraging Fabric Eventhouse to Store Great Expectations Validation Results
A step-by-step guide to storing data validation results into the Eventhouse & visualize data using queries and dashboard.
Taming Complexity: Intuitive Evaluation Framework for Agentic Chatbots in Business-Critical Environments
This blog post introduces a comprehensive evaluation framework for enterprise chatbots powered by large language models (LLMs), specifically addressing the challenges of assessing Line of Business (LOB) agents in business-critical environments. The authors tackle the fundamental problem that traditional chatbot evaluation metrics fail to capture the nuanced, non-deterministic performance of modern LLM-based systems, proposing a solution that combines realistic chat simulation using an LLM-powered User Agent, automated ground truth generation at scale, and comprehensive metrics including function call precision, r...
Unlocking Vector Search with OneLake Indexer and OpenAI Integration in Microsoft Fabric
Exploring how Microsoft Fabric OneLake indexer integrates with OpenAI
Ground Truth Curation Process for AI Systems
Steps to Produce High Quality Ground Truth Pairs for AI Systems
Customizable and Extensible Evaluation of a GenAI Application
Suggestions on how to evaluate a generative AI application in an adaptable and extensible manner.
Learnings from External Data Handling
This blog post discusses the challenges and solutions encountered by the ISE team at Microsoft while making a distributed system production-ready. It focuses on issues including slow processing speeds and out-of-memory exceptions, and provides insights into the methods used to address these problems.
Enhancing Analytical Queries with Semantic Kernel and In-Memory SQL Processing
Integrating Semantic Kernel with an ephemeral in-memory SQL plugin, enabling secure and efficient analytical queries over structured data.
AI Model Promotion with dstoolkit-mlops-v2
Evaluates various repository structures and designs for maximizing the efficiency of Data Scientists and Software Engineers developing, promoting and deploying AI models on the same project.
Use Agent to Update Dataverse Table Content
Create an agent using Copilot Studio to update the content of a Dataverse table.
Fleet Configuration Management
This post explains the challenge of fleet configuration management, the role of an automated fleet configuration management system, and describes key considerations for building such a system.
PACT Contract Testing – Because Not Everything Needs Full Integration Tests
PACT is a contract testing tool that allows you to define a contract between two applications, enabling
Dockerizing UV
How to Dockerize Python Package Management Tool UV
Hierarchical Waterfall Evaluation of Query Classification, Retrieval & Generation in Multi-Agent LLM Systems
This post presents a hierarchical waterfall framework for evaluating query classification, retrieval, and generation in multi-agent LLM systems.
Efficient Ground Truth Generation for Search Evaluation
This article discusses the challenges and methodologies for creating a reliable groundtruth dataset for search evaluation. It highlights the use of TREC Pooling and GPT-4o assisted ranking to overcome the labor-intensive process of manual labeling, ensuring high-quality search evaluations even with limited data.
Durable Functions for Indexing in RAG: A Practical Python Approach
Azure Durable Functions streamline RAG indexing by combining push flexibility with pull reliability for scalable,
Cloud infrastructure provisioning: best practices for IaC
Infrastructure-as-Code (IaC) is a common DevOps practice that enterprises use to provision and deploy IT infrastructure. Enterprises applying IaC and continuous integration/continuous delivery (CI/CD) pipelines can maintain high availability and manage risk for their cloud environments at scale. However, multi-environment challenges, manual processes, and fragmented guidance still lead to configuration drift, errors, and inconsistencies. These issues are likely to result in downtime, security vulnerabilities, and inefficient resource utilization.
Integration testing with Dapr and Testcontainers
This blog post discusses setting up a Pytest integration test framework for a system using Dapr and Testcontainers. This framework was initially setup for a customer to suit their needs and it has been extracted into a sample project to provide a starting point.
Running RAG with ONNX Runtime GenAI for On-Prem Windows
Exploring how to efficiently run a RAG pipeline with structured language models (SLMs) and guardrails on Windows, achieving inference under 5 seconds with ONNX Runtime GenAI.
Data Validations with Great Expectations in MS Fabric
In this blog post, we will explore how to integrate GX within the Microsoft Fabric environment and use it to validate data programmatically.
Using OpenTelemetry for flexible observability
OpenTelemetry offers flexible observability in distributed systems, this post looks at how to integrate multiple data sources when connected and disconnected from the internet.
The Hidden Risks of Docker Build Time Arguments and How to Secure Your Secrets
This post discusses the security risks of using Docker build-time arguments to pass secrets, and suggests alternative methods for handling secrets securely.
Flexible Tool Selection for ML Model Production
data science and software engineering teams can choose the best tools for their respective roles in delivering machine learning models to production.
Multi Root Workspaces in Visual Studio Code
How to manage multiple independent python projects, with different dependencies, inside Visual Studio Code.
Multi-Provider Strategy for App Configuration in Python
This post discusses a strategy for creating a custom extensible configuration module in python projects.
Instrumenting Apache Spark Structured Streaming jobs using OpenTelemetry
Apache spark monitoring using OpenTelemetry
Unlock Generative AI for Enterprise Scalability
Reusable templates, pipelines, and tools to streamline the setup of GenAI projects, saving time and effort, and ensuring consistency and reliability across projects.
Azure Functions vs. Indexers: AI Data Ingestion
This article compares Azure Functions with pre-built indexers for data ingestion in Azure AI Search, with a focus on using Azure Functions for a flexible, scalable approach. It explores key steps like data migration, index creation, and deployment automation.
Using Managed Identity on Logic App consumption
Deploy Azure Logic App Consumption with Managed Identity using Terraform by leveraging azapi_resource and ARM templates. This setup enables secure authentication without manual credential management.
Revolutionising Customer Feedback: Harnessing Large Language Models for Retail Insights and Intelligence
In this article, we delve into techniques for extracting valuable insights from customer feedback using Large Language Models (LLMs). By identifying themes, sentiment, and competitor comparisons from feedback, businesses can gain a competitive edge.
LLMOps in restricted networks and addressing continuous evaluation long run constraints
In this blog post, we'll explore the challenges we faced in establishing LLMOps and continuous evaluation within a pipeline using Azure Machine Learning, particularly when dealing with long runs and a restricted Bring-Your-Own (BYO) network.
Transforming Language into Code: Building and Evaluating a Robotic Code Generation Copilot
This blog post explores the development and evaluation of an AI assistant that converts natural language into robotic code.
Implementing Evaluation platform on Azure Machine Learning
Azure Machine Learning is known for training and deploying models, but can also be used for running experiments. This blog post will show us how we have implemented our Evaluation platform on Azure Machine Learning.
Semantic Kernel Learnings
A synopsis of our key learnings and findings from using Semantic Kernel.
Building AI Search for Production in Corporate Environments: Key Lessons
In the dynamic world of AI and data science developing production-level solutions for corporate environments comes with its own set of challenges and lessons. As a data science team working within Microsoft, we recently completed an engagement for a large company where we leveraged cutting-edge technologies, including OpenAI tools, GPT-4o for generating syntactic datasets, embedding models like text_embedding_3, and Azure AI Search for implementing both text and hybrid search solutions. Here are 10 key lessons we learned along the way.
Accessing Private Resources in Azure
A practical guide for developers on connecting to Azure resources with private endpoints through a VPN.
Three Ways to Simplify CI/CD Pipelines on GitHub Actions
This post focuses on three ways to simplify CI/CD pipelines deploying to Azure with GitHub Actions.
Building a Multi-tenant GenAI gateway using APIM
This blog shares insights on developing a GenAI gateway with multi-tenancy and quota management capabilities implemented using Azure APIM where customers can access the GenAI resources across different service tiers like Freemium, Basic, and Premium with each tier having it's own quota and rate limits. The solution used the concept of "Products" to group APIs related to specific entitlements and using Product Policies, simplifying the overall design and ensuring scalability.
Multimodal RAG with Vision: From Experimentation to Implementation
This blog post delves into the experimentation journey of fine-tuning a multimodal RAG pipeline to best answer user queries that require both textual and image context. We ran our experiments by systematically testing various approaches, adjusting one configuration setting at a time and using clearly defined evaluation metrics to validate the performance of each component of the RAG pipeline in isolation, as well as the end-to-end inference flow.
Streamlining Error Handling in Power Automate for Better Workflow Performance
This post explores effective error handling strategies in Power Automate to enhance workflow reliability and maintainability through practical techniques and integrations.
PromptFlow Serve – Benchmark Result Analysis
In this post we discuss how to test the throughput of PromptFlow pf-serve module and key learnings doing so. We explore the impact on throughput and performance the different WSGI and ASGI hosting methods have and the importance of engineering your Python nodes with the async await pattern for I/O.
Earning Agentic (and LangChain) Complexity
This post examines the challenges of adopting complex technologies like LangChain and agentic solutions in production environments, emphasizing the importance of understanding the necessity of such complexity. It provides insights on how to evaluate these technologies carefully, manage dependencies, and adhere to best practices for secure and stable AI applications.
Semantic Router using Azure AI Search
Potential semantic routing solution using Azure AI Search for scalable query handling.
Measuring Differentials of Product Images in AI-generated Ads
Various methodologies of measuring differentials between a product image and an AI-generated ad for the purpose of product representation integrity in serving AI-generated ads.
Converting multiple sources to FHIR using Liquid
A brief summary of how Microsoft ISE teams creation a solution to integrate patients data from multiple fragmented sources into a standard FHIR data model
Profiling Docker Compose services through Visual Studio
A brief summary of how Crew Clark were able to profile performance using Visual Studio and within a Docker Compose environment.
LLM Prompt Injection Considerations With Tool Use
This post discusses a pattern to prevent a class of prompt injection attacks in LLM-based solutions. It emphasizes the importance of building strong foundational patterns to mitigate risks and avoid potential pitfalls. By implementing this pattern, teams can enhance the security of their tool-based solutions.
Document Summarization Solution Patterns using Azure Open AI & Langchain
This post lists the various solution patterns that can be applied for document summarization. Document summarization comes with its challenges related to token limitation and chunk sizes. This blog post discusses about the solutions to tackle those challenges.
Azure IoT Operations Data Processor Pipelines: Unlocking Efficiency in the Industrial Metaverse
In this post we explore the implementation of Azure IoT Operations Data Processor pipelines in a manufacturing context, focusing on data processing at the edge.
Learnings around developing with Microsoft Graph Toolkit
Learnings of Microsoft Graph Toolkit from our recent project, including data leaking and privacy concerns that can help developers get a jumpstart with Microsoft Graph Toolkit.
Empowering Collaboration with a Tech Savvy Customer
Empowering Collaboration with a Tech Savvy Customer
Next-Level Boilerplate: An Inside Look Into Our .Net Clean Architecture Repo
In this article, we will take an inner look at the clean architecture boilerplate and how it helps kickstart our projects rapidly.
7 Essential Tips: Accelerate Prompt Flow Development
This post focusses on tips and tricks to accelerate from flow development through use of dev container & shell scripts.
Start Simple, Scale Fast: Azure Storage for Distributed Computing
Azure Storage Accounts are one of the simplest resources on Azure, but simultaneously one of the most useful and reused building blocks across the Microsoft Cloud ecosystem. Blobs, leases, and queues are fundamental components that enable powerful patterns. Let's dig into why Storage Accounts should be in your Azure toolkit.
Simplified Connection State Management
This post advocates for a simplifying a complex network of states into simple states that correlate with user expectations.
Synchronizing multiple remote Git Repositories
summary of the post
Making an Application More Robust With Data Mapping
How to use data mapping to make your application more robust against unexpected third-party data schema changes.