{"id":3182,"date":"2017-05-09T23:06:02","date_gmt":"2017-05-09T23:06:02","guid":{"rendered":"https:\/\/www.microsoft.com\/reallifecode\/?p=3182"},"modified":"2020-03-15T05:21:16","modified_gmt":"2020-03-15T12:21:16","slug":"provision-configure-infrastructure-azure-using-saltstack","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/provision-configure-infrastructure-azure-using-saltstack\/","title":{"rendered":"Provision and Configure your Infrastructure on Azure using SaltStack"},"content":{"rendered":"

Recently we partnered with LogDNA<\/a> to help migrate their existing infrastructure to Azure using SaltStack<\/a> and the Azure Python SDK. LogDNA is based in the San Francisco area, and their product offerings include log data ingestion and analytics. The company has been leveraging SaltStack to provision and manage their infrastructure across AWS and DigitalOcean. In this post, we will use ElasticSearch as an example to illustrate the end-to-end process of how we provisioned and configured the deployment of an ElasticSearch cluster on Azure using SaltStack.<\/p>\n

SaltStack<\/a> is an open source project, supported by the company of the same name that delivers an enterprise version<\/a> of the project. SaltStack delivers infrastructure as code and configuration management with an\u00a0abstraction of the cloud provider selected. At the time of writing this post, support for Azure ARM by SaltStack is still in preview, and the documentation for Azure ARM support is still in development. We hope this post serves as both an example and guide.<\/p>\n

Overview<\/strong><\/h2>\n

The model of SaltStack is based on masters and minions, with each minion agent reaching back to the master; this approach makes the solution very scalable. The master holds the configuration of the minions in a set of configuration files. Those files provide an idempotent configuration that will be applied when the minion role is deployed or at any time the configuration is re-applied. In this scenario, we will be using SaltStack to install and configure an ElasticSearch cluster on Azure.<\/p>\n

Installation<\/h3>\n

Clone this repo:<\/p>\n

$ git clone https:\/\/github.com\/ritazh\/azure-saltstack-elasticsearch<\/pre>\n
Get Azure CLI 2.0 from here<\/a> if you don’t already have it. Then log into the CLI:<\/p>\n
$ az login\r\n$ az account show<\/pre>\n
Create a service principal following these steps<\/a>.The output should look something like the following:<\/p>\n
{\r\n\"appId\": \"59db508a-3429-4094-a828-e8b4680fc790\",\r\n\"displayName\": \"myserviceprincipalapp\",\r\n\"name\": \"https:\/\/myserviceprincipalapp.azurewebsites.net\",\r\n\"password\": {the password you supplied displayed here},\r\n\"tenant\": \"72f988bf-86f1-41af-91ab-2d7cd011db47\"\r\n}\r\n<\/pre>\n
Now you are ready to kick off the scripts. Use appId<\/em> from the previous step for servicePrincipalAppId<\/em>, and password<\/em> from the previous step for serviceprincipalsecret.<\/em>\u00a0Sit back and enjoy a cup of coffee. (This might take a while!) Once the script is done, you will have a working ElasticSearch cluster ready\u00a0for use.<\/p>\n
$ deploy-salt-cluster.sh -o create \r\n-u <adminUsernameForVM> \r\n-n <namespaceForResourceGroup> \r\n-c <servicePrincipalAppId> \r\n-s <serviceprincipalsecret> \r\n-t <tenantid>\r\n<\/pre>\n
When prompted, create\u00a0a password for the SaltStack cluster master node admin user. This credential\u00a0will be used by the scripts to complete installation and configuration on the master node. You can also modify this example to use an ssh key.<\/p>\n
After the deploy-salt-cluster.sh<\/em> script is done, look for the IP address of the minionesmaster<\/em> node from the Azure portal<\/a>. This IP address is the public IP of the ElasticSearch master node. Now ElasticSearch is accessible via http:\/\/{minionesmaster-public-ip}:9200<\/em>.<\/p>\n
You can use a tool like the\u00a0Sense Chrome extension<\/a> to add new content to your search index and to conduct search queries.<\/p>\n
\"Image
ElasticSearch Demo<\/figcaption><\/figure><\/p>\n
What is the script doing?<\/h2>\n
Now let’s take a look under the hood. When we run deploy-salt-cluster.sh<\/a>, we are actually doing two things:<\/p>\n
    \n
  1. Deploying resources needed to provision a SaltStack cluster on Azure<\/li>\n
  2. Creating and configuring an ElasticSearch cluster using SaltStack<\/li>\n<\/ol>\n
    Deploy Resources needed for a SaltStack Cluster<\/h3>\n
    In deploy-salt-cluster.sh<\/a>, we are using the Azure Service Principal created in the previous section to create an Azure Resource Group and all the resources needed to provision a SaltStack cluster using an Azure ARM template<\/a> and the Azure CLI<\/a>.<\/p>\n
    $ az group deployment create \r\n-g $resourceGroupName -n $NamePrefix \r\n--template-uri $TEMPLURI --parameters \"$PARAMS\"<\/pre>\n
    From the Azure ARM template azuredeploy.json<\/a>, we’ll provision the following resources on Azure for the SaltStack cluster:<\/p>\n