Introduction

Picture this scenario: you’ve implemented a critical fix in your AI service and want to validate it against your comprehensive evaluation dataset. However, the prospect of deploying to Azure for a quick test creates an unwelcome delay in your development cycle. This challenge is common among AI developers.

When developing AI applications, one of the most significant pain points is efficiently testing locally running services against cloud-based evaluation frameworks. While local development offers speed and debugging convenience, robust evaluations require the comprehensive pipelines that Azure Machine Learning provides. This creates tension between rapid iteration and thorough validation.

We’re excited to share an approach using Microsoft Devtunnels with Azure Machine Learning that has transformed our development workflow. This solution enables you to test local services against cloud evaluations seamlessly, maintaining both development velocity and evaluation rigor.

The Challenge: Local Development vs. Cloud Evaluation

Consider a common development scenario: you’re working on an AI-powered service that generates business intelligence dashboards from natural language queries (transforming requests like “show me last quarter’s sales by region” into interactive visualizations). During development, teams often find themselves navigating between competing requirements:

1. Running services locally for rapid iteration and simplified debugging
2. Executing comprehensive evaluations using proven Azure Machine Learning pipelines
3. Maintaining centralized tracking where team members can monitor progress and results
4. Avoiding frequent deployments to reduce overhead and development friction

Traditional approaches present significant limitations:

• Deploying every change for cloud testing introduces substantial feedback delays
• Maintaining separate local evaluation scripts often leads to inconsistencies and drift from production evaluation methods

These constraints can significantly impact development velocity and team productivity.

The Solution: Microsoft Devtunnels

Microsoft Devtunnels provides an elegant solution to this challenge. It acts as a secure bridge between your local development environment and the internet, creating HTTPS tunnels that make locally running services accessible from anywhere, including Azure Machine Learning pipelines.

The key advantage is that Azure ML can communicate with your local development server as if it were any other deployed service, eliminating the traditional “works locally but fails in cloud testing” disconnect.

Architecture Overview

The following diagram illustrates how this architecture functions in practice:

┌─────────────────┐    ┌──────────────────┐    ┌────────────────┐
│   Local Dev     │    │   Devtunnels     │    │  Azure ML      │
│   Environment   │    │   (Bridge)       │    │  (Cloud Eval)  │
│                 │    │                  │    │                │
│ ┌─────────────┐ │    │ ┌──────────────┐ │    │ ┌────────────┐ │
│ │AI Service   │◄├────┤ │HTTPS Tunnel  │◄├────┤ │Evaluation  │ │
│ │Port 4301    │ │    │ │g276r7b9...   │ │    │ │Component   │ │
│ └─────────────┘ │    │ │.devtunnels.ms│ │    │ └────────────┘ │
│                 │    │ └──────────────┘ │    │                │
└─────────────────┘    └──────────────────┘    └────────────────┘

The solution maintains clear separation between local development and cloud evaluation while providing seamless connectivity through the Devtunnels bridge.

Implementation Guide

Let’s walk through the complete setup process.

Step 1: Install and Configure Microsoft Devtunnels

Begin by installing the Devtunnels CLI. The installation process varies by operating system:

macOS Installation

For macOS users, use the automated installer script:

# Install Devtunnels CLI
curl -sL https://aka.ms/DevTunnelCliInstall | bash
source ~/.zshrc

# Verify successful installation
devtunnel -h

Windows Installation

For Windows users, you have several installation options:

Option 1: Using PowerShell (Recommended)

# Install using the automated PowerShell script
Invoke-WebRequest -Uri "https://aka.ms/DevTunnelCliInstall" -UseBasicParsing | Invoke-Expression

# Verify installation
devtunnel -h

Option 2: Using Windows Package Manager (winget)

# Install via winget
winget install Microsoft.devtunnel

# Verify installation
devtunnel -h

Option 3: Manual Download

1. Download the latest release from the Microsoft Devtunnels releases page
2. Extract the executable to a directory in your PATH
3. Verify installation with devtunnel -h

Linux Installation

For Linux distributions, use the automated installer script:

Ubuntu/Debian:

# Install Devtunnels CLI
curl -sL https://aka.ms/DevTunnelCliInstall | bash

# Reload your shell configuration
source ~/.bashrc

# Verify successful installation
devtunnel -h

CentOS/RHEL/Fedora:

# Install Devtunnels CLI
curl -sL https://aka.ms/DevTunnelCliInstall | bash

# Reload your shell configuration
source ~/.bashrc

# Verify successful installation
devtunnel -h

Manual Installation (All Linux Distributions):

# Download and install manually
wget (https://learn.microsoft.com/en-us/azure/developer/dev-tunnels/get-started?tabs=windows#install

chmod +x devtunnel-linux-x64
sudo mv devtunnel-linux-x64 /usr/local/bin/devtunnel

# Verify installation
devtunnel -h

Authentication (All Platforms)

Once installed, authenticate with your Microsoft account:

devtunnel user login

This command opens your browser and guides you through the standard Microsoft authentication flow regardless of your operating system.

Step 2: Create and Host a Tunnel

There are two primary approaches for creating tunnels, depending on whether you’re conducting initial testing or establishing a more permanent development setup.

Option A: Temporary Tunnel (Ideal for Initial Testing)

For quick validation of the overall approach, use this command:

# Create a temporary tunnel to your local service
devtunnel host -p 4301

This command will generate output similar to:

Hosting port 4301 at <devtunnels URL>
Ready to accept connections for tunnel: swift-horse-fj051061

The generated URL immediately provides access to your local port 4301.

Option B: Persistent Tunnel (Recommended for Regular Development)

For ongoing development work, establish a more robust configuration:

# Create a persistent tunnel with authentication options
devtunnel create --allow-anonymous

# Set an expiration to maintain security hygiene
devtunnel create --allow-anonymous --expiration 7d

# Configure the target port and begin hosting
devtunnel port create -p 4301
devtunnel host

The persistent approach provides greater control and enables reusing the same tunnel URL across multiple development sessions, simplifying Azure ML pipeline configuration management.

Authentication Methods

Devtunnels provides multiple security options to control access to your local service:

1. Anonymous Access (--allow-anonymous): Allows unrestricted access to anyone with the URL. Suitable for development environments but requires careful consideration of security implications.
2. Token-based Authentication: Generates JWT tokens with configurable expiration times (24 hours by default). This approach provides an optimal balance of security and convenience for Azure ML integration.
3. Microsoft Entra ID: Offers enterprise-grade authentication suitable for corporate environments with strict access control requirements.

For Azure ML integration scenarios, token-based authentication typically provides the most practical solution.

Step 3: Authentication and Token Management

For Azure Machine Learning pipelines to authenticate with your tunnel, you have several options including anonymous access, token-based authentication, or Microsoft Entra ID authentication (detailed in the Complete Authentication Workflows section below). For most Azure ML scenarios, token-based authentication provides the best balance of security and usability.

Token Generation (Recommended):

# Generate a token with the default 24-hour expiration
devtunnel token swift-horse-fj051061 --scopes connect

# Configure longer expiration if needed
devtunnel token swift-horse-fj051061 --scopes connect --expiration 48h

Important considerations:

• Default expiration is 24 hours, which accommodates most evaluation scenarios
• Duration can range from 1 hour to several weeks
• Token validity cannot exceed the tunnel’s own expiration period
• Critical limitation: Tokens cannot be automatically refreshed once Azure ML pipelines begin execution. Ensure your token lifetime exceeds the expected completion time.

Complete Authentication Workflows for Azure ML

Choose the workflow that best matches your security requirements:

Workflow 1: Anonymous Access (Quick Testing)

# Step 1: Create and host tunnel
devtunnel create --allow-anonymous --expiration 7d
# Note the tunnel ID from output (example: swift-horse-fj051061)
devtunnel port create swift-horse-fj051061 -p 4301
devtunnel host swift-horse-fj051061

# Step 2: Get tunnel URL (in a separate terminal)
devtunnel show swift-horse-fj051061

# Step 3: Azure ML pipeline configuration
# url: "<devtunnels.ms URL>"
# devtunnel_token: ""  # Leave empty for anonymous access

Workflow 2: Token-based Authentication (Recommended)

# Step 1: Create secured tunnel
devtunnel create --expiration 7d
# Note the tunnel ID from output (example: clever-cat-gh892341)
devtunnel port create clever-cat-gh892341 -p 4301

# Step 2: Generate authentication token
devtunnel token clever-cat-gh892341 --scopes connect --expiration 48h

# Step 3: Begin hosting
devtunnel host clever-cat-gh892341

# Step 4: Get tunnel URL
devtunnel show clever-cat-gh892341

# Step 5: Azure ML pipeline configuration
# url: "<devtunnels.ms URL>"
# devtunnel_token: "eyJhbGciOiJFUzI1NiIsImtpZCI6..."

Workflow 3: Enterprise Authentication

# Step 1: Create tunnel with tenant access
devtunnel create --expiration 30d
devtunnel access create brave-lion-kx456789 --tenant
devtunnel port create brave-lion-kx456789 -p 4301

# Step 2: Generate token and start hosting
devtunnel token brave-lion-kx456789 --scopes connect --expiration 24h
devtunnel host brave-lion-kx456789

# Step 3: Get tunnel URL
devtunnel show brave-lion-kx456789

Step 4: Configure Your Azure ML Pipeline

Update your Azure ML pipeline configuration to utilize the Devtunnel endpoint. Here’s an example configuration:

# pipeline.yaml
jobs:
  service_evaluation:
    type: command
    component: azureml:service_evaluation@latest
    environment_variables:
      SERVICE_USERNAME: ${{secrets.SERVICE_USERNAME}}
      SERVICE_PASSWORD: ${{secrets.SERVICE_PASSWORD}}
    inputs:
      input_data: ${{parent.inputs.input_data}}
      # Replace production URL with tunnel URL for local development
      url: "<devtunnels.ms/api/visualization URL>"
      # Include tunnel authentication token
      devtunnel_token: "eyJhbGciOiJFUzI1NiIsImtpZCI6..."
      retry_count: 3
      retry_delay: 5

This configuration enables seamless switching between local development and production environments by simply modifying the URL and token parameters.

Step 5: Update Your API Client Code

Your Azure ML component requires modification to include tunnel authorization headers in requests. Here’s the implementation approach:

def call_api(url, bearer_token, data, tunnel_token="", logger=None):
    headers = {
        "Authorization": f"Bearer {bearer_token}",
        "Content-Type": "application/json",
        "Use-Advanced-Mode": "true",
    }

    # Add tunnel authorization header when token is provided
    if tunnel_token:
        headers["X-Tunnel-Authorization"] = f"tunnel {tunnel_token}"

    try:
        response = requests.post(url, json=data, headers=headers)
        response.raise_for_status()
        return response.json()
    except requests.exceptions.RequestException as e:
        logger.error(f"API call failed: {e}")
        raise

The X-Tunnel-Authorization header is essential for Devtunnels to authenticate and authorize requests to your local service. Without this header, authentication will fail and requests will be rejected.

Step 6: Component Parameter Configuration

Design your Azure ML component to intelligently detect whether it’s communicating with a local tunnel or production service. This approach eliminates the need for multiple component versions:

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--url", type=str, required=True,
                        help="API endpoint URL")
    parser.add_argument("--devtunnel_token", type=str, default="",
                        help="Devtunnel token for local development")

    args = parser.parse_args()

    # Log the operational mode for debugging clarity
    if args.devtunnel_token:
        logger.info(f"Operating in development mode: {args.url}")
    else:
        logger.info(f"Operating in production mode: {args.url}")

This logging approach provides clear operational context when reviewing Azure ML Studio logs, particularly valuable during troubleshooting sessions.

Real-World Example: AI-Powered Business Intelligence Service

To illustrate this approach, let’s examine an AI-powered business intelligence service that transforms natural language queries into interactive charts and dashboards. The service exposes REST API endpoints for visualization generation, semantic search, and data analysis.

The Evaluation Pipeline

The Azure ML pipeline consists of two components:

1. Service Evaluation Component: Executes test queries against service endpoints
2. Metrics Processor Component: Analyzes responses and calculates evaluation metrics

Local Development Workflow

The typical workflow includes:

1. Initialize the local service (port 4301) with required endpoints
2. Establish a Devtunnel connection
3. Generate authentication tokens as needed
4. Update pipeline configuration with tunnel URL and credentials
5. Execute the Azure ML pipeline
6. Review results in Azure ML Studio

This workflow reduces development cycles from 30+ minutes to 5-10 minutes for complete evaluation feedback.

Pipeline Configuration

Here’s the actual configuration used in production:

service_evaluation:
  inputs:
    # Production URL (commented out during development)
    # url: "<PROD_URL>"

    # Devtunnel URL for local development
    url: "<DEV_URL>"

    # Devtunnel authentication token
    devtunnel_token: "eyJhbGciOiJFUzI1NiIsImtpZCI6..."

    # Other configuration
    user_tenant: "Development Team"
    truth_column: "expected"
    query_column: "User Query"
    retry_count: 3
    retry_delay: 5

Best Practices and Tips

The following recommendations are based on practical experience implementing this approach across multiple development scenarios:

Service Limits and Quotas

Before implementing Devtunnels in your development workflow, it’s important to understand the service limits that apply. These limits reset monthly:

Resource	Limit
Bandwidth	5 GB per user
Tunnels	10 per user
Active connections	1000 per port
Ports	10 per tunnel
HTTP request rate	1500/min per port
Data transfer rate	Up to 20 MB/s per tunnel
Max web-forwarding HTTP request body size	16 MB

Planning considerations:

• Bandwidth usage: Monitor your evaluation pipeline’s data transfer to stay within the 5 GB monthly limit
• Request rate limits: Azure ML evaluation pipelines should respect the 1500 requests/minute limit per port
• Connection limits: Large-scale parallel evaluations may hit the 1000 concurrent connections per port limit
• Tunnel management: With a 10-tunnel limit per user, consider reusing tunnels across development sessions

For questions about these limits or requests for increases, you can open an issue in the Microsoft dev-tunnels GitHub repository.

Security Considerations

Authentication Method Selection:

• Anonymous access: Convenient for rapid prototyping but exposes services publicly. Reserve for temporary development use only.
• Token-based authentication: Provides balanced security and usability. Recommended for regular development workflows.
• Microsoft Entra ID: Enterprise-grade solution suitable for corporate environments with strict compliance requirements.

Token Management Best Practices: Devtunnel tokens cannot be automatically refreshed within running Azure ML pipelines. Plan token lifetime to exceed expected pipeline completion time.

Credential Security: Never commit devtunnel tokens to version control. Use Azure ML secrets or environment variables.

Development Workflow Optimization

Environment Configuration Management: Use configuration files or environment variables to enable seamless switching between local and production endpoints.

Comprehensive Logging: Implement clear logging that identifies whether requests target local or production services to reduce debugging time.

Network Resilience: Implement retry logic and appropriate timeouts. Network latency can vary when routing requests through tunnels.

Performance Optimization

Rate Limiting: Local services may not handle full Azure ML evaluation pipeline volume. Implement rate limiting to prevent resource exhaustion.

Geographic Considerations: Network latency varies based on proximity between development environments and Azure regions.

Troubleshooting Common Issues

The following troubleshooting guide addresses frequently encountered issues and their resolutions:

Connection Problems

• Tunnel connectivity failures: Verify tunnel status using devtunnel show to confirm the tunnel is active and properly configured.
• Authentication errors: Confirm that tokens haven’t expired and match the tunnel’s configured authentication method. Both tokens and tunnels have expiration times that must be managed appropriately.
• Port configuration issues: Verify that your local service is running on the expected port.

Azure ML Pipeline Issues

• Component execution failures: Review Azure ML logs for specific error messages related to URL accessibility or authentication problems.
• Token format errors: Ensure the X-Tunnel-Authorization header uses the exact format tunnel <your-token> (not Token or Bearer).
• Token expiration during execution: If evaluation pipelines exceed token lifetime, they will fail mid-execution. Set token expiration to at least twice the expected pipeline duration.

Performance Issues

• Response latency: Consider geographic distance between development environments and Azure regions when investigating performance concerns.
• Rate limiting effects: Monitor local service logs to identify whether rate limiting or resource constraints are causing request failures or delays.

Conclusion

This approach using Microsoft Devtunnels with Azure Machine Learning has significantly transformed our AI development workflows. The ability to test local modifications against production-quality evaluation pipelines without deployment overhead provides substantial development velocity improvements.

Key benefits we’ve observed include:

1. Accelerated development cycles: Complete evaluation feedback is available within minutes rather than the traditional deployment timeframes
2. Evaluation consistency: The same pipeline validates both local development and production code, eliminating environment-specific discrepancies
3. Reduced infrastructure costs: Decreased cloud compute requirements during development iterations
4. Enhanced debugging capabilities: Full local debugging access while maintaining comprehensive cloud-based evaluation

This implementation demonstrates how to establish secure connectivity between local development and cloud evaluation environments, configure Azure ML pipelines for both local and production targets, implement proper authentication, and optimize workflows for efficient development practices.

This pattern proves particularly valuable for AI applications where rapid iteration and comprehensive evaluation are critical success factors.

Additional Resources

• Microsoft Devtunnels Documentation
• Azure Machine Learning Pipeline Documentation
• Azure ML Components Documentation
• REST API Best Practices for AI Services

---

This blog post is based on a real-world implementation of AI service evaluation using Azure Machine Learning and Microsoft Devtunnels. The code examples and configurations shown are adapted from production use cases for evaluating business intelligence and analytics services.