Deploying an Azure Red Hat Linux VM Running Apache Tomcat for use with Visual Studio Team Services and Team Foundation Server

Paul T. Barham

August 18th, 20160 0

Visual Studio Team Services and Team Foundation Server (TFS) now has at least three mechanisms (i.e. deployment and utility tasks) for deploying to a Linux host or virtual machine (VM). This walkthrough will show specifically how to setup and configure an Red Hat (v. 7.2) VM on Azure to run Tomcat and other necessary services to support three different and distinct Team Services deployment tasks to enable continuous integration and deployment. We will configure the VM to enable the Apache Tomcat Deployment task, the Copy Files over SSH task, and the FTP Upload task (using ftps) to enable deployment of web applications from Team Services.

The Apache Tomcat Deployment task is available on the Visual Studio Marketplace. The Copy Files over SSH and FTP Upload tasks have just been released with their respective blog announcements below:

Get going quickly

If you want a quick and simple method for deploying an Azure Red Hat Linux VM with Tomcat, SSH and FTP for use with Visual Studio Team Services and TFS, then you can use the Azure QuickStart ARM template found here:

https://github.com/Azure/azure-quickstart-templates/tree/master/vsts-tomcat-redhat-vm

Using the above template and script, you can configure and deploy a new Azure Red Hat Tomcat server in a matter of minutes by providing a few required parameters. Be sure to allow the full Azure deployment to complete before logging into the VM (as logging in to soon can cause permission issues).

On the other hand, if you want to setup and tailor the Azure Red Hat VM yourself, then follow the detailed instructions below. Following these instructions, you can setup Tomcat for deploying from Team Services / TFS with either or all of the Apache Tomcat extension deployment task, the built-in Copy Files over SSH deployment task or the built-in FTP Upload task.

If you would prefer to setup an Azure Ubuntu VM (instead of or in addition to Red Hat) running Tomcat, then follow the instructions in this sister blog post:

https://blogs.msdn.microsoft.com/visualstudioalm/2016/08/18/deploying-an-azure-ubuntu-linux-vm-running-apache-tomcat-for-use-with-visual-studio-team-services-and-team-foundation-server

Deploy a Linux Red Hat VM on Azure

The first step for our walkthrough is to create a Linux Red Hat VM running in Azure. If you don’t yet have an Azure account, you can sign up for one free.

Login to Azure and go to the Azure Portal, https://portal.azure.com. In the left-hand navigation panel, click on Virtual machines. Under the Virtual Machines panel, click “+ Add”. This brings up the Virtual Machines marketplace window. In the search window (by the magnifying glass), enter “Red Hat” and press Enter. Find and click on the line containing “Redhat Enterprise Linux 7.2”. Ensure the “Resource Manager” deployment model is selected and press the “Create” button.

Under the “Basics” panel, choose and enter a “Name” for your VM and a “User name”. Select “Password” as the Authentication type, and enter and confirm a password for your VM. Create a new Resource group (e.g. “demo”) and select a Location nearest to your physical location. Other values can remain as their defaults. When you are ready, press the “OK” button.

Under the “Choose a size” panel, click on the option to “View all.” For the purposes of this walkthrough, we recommend choosing a size of “F1S Standard” but any will work. It is easy to upgrade to a more powerful VM later if you like. Click on the size you want to use and then press the “Select” button at the bottom of the panel.

Under the “Settings” panel, the defaults should all work for this demo, so just press the “OK” button at the bottom of the panel.

Assuming all the parameters for the VM are OK, you should see a blue banner stating “Validation passed”. If you are satisfied with the setup displayed, press the “OK” button at the bottom of the panel to actually start the deployment of the VM.

To view the progress with deploying the VM, you can click on the bell looking icon in the upper right of the dark blue menu bar and then click on the “Deployment started…” message. A panel will be displayed showing the status of the deployment, likely with a blue banner stating the VM is “Deploying.” You can “Refresh” to see current status or even “Cancel” the request to create the VM from this panel. After a few minutes, press “Refresh” to see of the deployment is finished (i.e. Status is Succeeded). Once the VM is deployed, click on the “Virtual machines” in the far-left navigation panel and then click on your newly created VM in the list under the Virtual machines panel.

Note (and record) the Public UP address of your new VM. We will use it to login and configure the VM. If you like, you can give your VM a public DNS name. To do this, click on the word “<none>” which appears beside the public IP address. Then click on “Configuration”, and finally enter the DNS name label in the text box provided. You should write down and note the name you entered as well as the data center (i.e. myVM.southcentralus.cloudapp.azure.com). When you have entered the name of your choosing, press “Save” by the disk icon near the top of the panel. You can use either the public IP address or DNS name to refer to your VM using popular tools. If you have noted and saved the IP and DNS names, you can close the Azure portal in the web browser.

Using a tool of your choosing, log in to your newly created Red Hat 7.2Server VM using the User name and Password you selected above when deploying the VM. Free tools are available such as MobaXterm. You can also use a Windows Command prompt and use the command: ssh <hostname> -l <username>

You will need to know how to use a basic text editor (such as ‘vi’, ‘vim’ or ‘nano’) in order to edit files to configure your server.

0. Install Apache Tomcat On Your Azure Linux Redhat VM

On Red Hat Linux on Azure, by default you will need to enter your password when executing the ‘sudo’ command. To keep from having to enter your password for every sudo command in this walkthrough, let’s start by configuring passwordless su privileges for your account.

0.0: Edit the file /etc/sudoers.d/waagent (e.g. sudo vi /etc/sudoers.d/waagent) and edit the one line there to read:

azureuser ALL = (ALL) NOPASSWD: ALL

azureuser is where you substitute the User name you chose when you setup and logged into the machine. Save the file (if using ‘vi’, you may have to use w! to write the file if you get a warning that the file is read only).

To get Tomcat running, we will install the standard Apache2 and Tomcat7 packages, and download and build the mod-jk package (thus also requiring gcc, c++ and http-devel).

0.1: From a terminal window running on your newly created Red Hat 7.2 Server, enter the following commands (you will need to allow each command to complete before entering the next command):

sudo yum install -y httpd

sudo yum install -y tomcat

sudo yum install -y tomcat-webapps tomcat-admin-webapps

sudo yum install -y gcc

sudo yum install -y gcc-c++

sudo yum install -y httpd-devel

wget http://www-us.apache.org/dist/tomcat/tomcat-connectors/jk/tomcat-connectors-1.2.41-src.tar.gz

tar xvfz tomcat-connectors-1.2.41-src.tar.gz

cd tomcat-connectors-1.2.41-src/native/

./configure –with-apxs=/usr/bin/apxs

make

sudo make install

Now that the required packages are installed on your VM, you need to configure the services.

0.2: Create a new file mod_jk.conf in the /etc/httpd/conf directory with the following content (e.g. sudo vi /etc/httpd/conf/mod_jk.conf):

# Load mod_jk module # Specify the filename of the mod_jk lib LoadModule jk_module modules/mod_jk.so

# Where to find workers.properties JkWorkersFile conf/workers.properties

# Where to put jk logs JkLogFile /var/log/httpd/mod_jk.log

# Set the jk log level [debug/error/info] JkLogLevel info

# Select the log format JkLogStampFormat “[%a %b %d %H:%M:%S %Y]”

# JkOptions indicates to send SSK KEY SIZE JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

# JkRequestLogFormat JkRequestLogFormat “%w %V %T”

# Mount your applications JkMount /* worker1

JkShmFile /var/run/mod_jk/jk-runtime-status

0.3: Create a new file /etc/httpd/conf/workers.properties (e.g. sudo vi /etc/httpd/conf/workers.properties ) and

Add/insert the following 10 lines and save the file:

# Define 1 real worker using ajp13

worker.list=worker1

worker.worker1.type=ajp13

worker.worker1.host=localhost

worker.worker1.port=8009

worker.worker1.socket_keepalive=true

worker.worker1.lbfactor=1

worker.worker1.connection_pool_size=50

worker.worker1.connect_timeout=5000

worker.worker1.prepost_timeout=5000

0.4: Edit the file /etc/httpd/conf/httpd.conf (e.g. sudo vi /etc/httpd/conf/httpd.conf ) and

Change the line containing ServerName to point to the name of your Azure VM and uncomment the line:

ServerName raisaredhat.southcentralus.cloudapp.azure.com:80

And add the following two lines at the end of the file:

# Include mod_jk’s specific configuration file Include conf/mod_jk.conf

0.5: Edit the file /usr/share/tomcat/conf/server.xml (e.g. sudo vi /usr/share/tomcat/conf/server.xml):

Change the line

0.6: Update the permissions on the Tomcat webapps and install directory:

sudo chown -R tomcat.tomcat /var/lib/tomcat/webapps

sudo chown tomcat.tomcat /usr/share/tomcat

sudo chown tomcat.tomcat /var/lib/tomcat

0.7: Set the default umask for Tomcat:

Edit the file /usr/libexec/tomcat/server (e.g. sudo vi /usr/libexec/tomcat/server):

Add the following umask line on a separate line just before the “run start” line and save the file:

umask 002

Then reload the daemon definitions:

sudo systemctl daemon-reload

0.8: Configure SELinux to allow mod_jk to work:

sudo yum install -y policycoreutils-python

sudo mkdir /var/run/mod_jk

sudo semanage fcontext -a -t httpd_var_run_t “/var/run/mod_jk(/.*)?”

0.9: Remove unnecessary http modules (that create warnings in the logs):

sudo vi /etc/httpd/conf.modules.d/00-proxy.conf

Add a ‘#’ as the first character to comment out the line containing the “heartbeat_module”, then save the file.

# LoadModule lbmethod_heartbeat_module modules/mod_lbmethod_heartbeat.so

0.10: Configure the system to run httpd and tomcat every time the server is booted:

sudo systemctl enable httpd

sudo systemctl enable tomcat

0.11: Restart the Tomcat7 and Apache2 servers:

sudo service httpd start

sudo service tomcat start

0.12: Open Red Hat software firewall for port 80:

sudo firewall-cmd –zone=public –add-port=80/tcp –permanent

sudo firewall-cmd –reload

Each service should have reported OK for the above restart commands when they started successfully.

0.13: Your Redhat Server is now running Tomcat, but Azure needs to be configured to allow incoming traffic to reach your server. To do this:

Open Azure portal: https://portal.azure.com
Select “Virtual machines” in the left-hand navigation panel (not Virtual machines (classic))
Under the “Virtual machines” panel, select the machine you want to update
Click on “Network interfaces” under Settings
Click on the network interface to be updated
Click on “Network security group” under Settings
Click on the network security group to be updated
Click on “Inbound security rules” under Settings
“+Add” a new rule with Name “default-allow-tomcat” on Destination port range 80, and click the “OK” button. The other default values should be sufficient.

The VM will show as “Updating” and then reports back the rule has been added. Your Azure Redhat 16 Server is now fully configured for Tomcat.

The image below shows the final state for the Inbound Security Rules at the completion of this entire walkthrough. Only the first two rules (default-allow-ssh and default-allow-tomcat) will be configured initially after step 0.13 is completed. The “default-allow-ftps” and “default-allow-ftp” rules are configured later in steps 3.6 – 3.8 and are only needed if you want to deploy using FTP (ftps).

0.13: To verify the operation, open a web browser window pointing to your new server…

Example: http://raisatomcat.southcentralus.cloudapp.azure.com/

The webpage should display a Apache Tomcat/7.0.54 welcome page. If not, go back and make sure all of the above steps were completed in the order specified.

In the next sections, we will describe three different deployment methods for deploying web applications to this Tomcat server in Azure from Team Services (or TFS). Each section is independent of the other section so you can choose to configure only one option, any two options, or all three as your needs require. That is, Option 2 is not dependent on Option 1, Option 3 is not dependent on Options 2 and 3, etc.

Option 1: Configure Tomcat Azure VM for Web UI Manager Access and the Team Services Apache Tomcat Deployment Task

Add permissions to be able to access the Tomcat manager (Web UI and/or Team Services task).

1.1: Edit the file /etc/tomcat/tomcat-users.xml (e.g. sudo vi /etc/tomcat/tomcat-users.xml) and add the following lines just above the final </tomcat-users> tag near the bottom of the file and save the file:

<role rolename=”tomcat”/>

<role rolename=”manager-script”/>

<role rolename=”manager-gui”/>

<role rolename=”manager”/>

<role rolename=”admin-gui”/>

<user username=”tomcat” password=”tomcat” roles=”tomcat”/>

<user username=”manager” password=”bitnami” roles=”tomcat,manager-script,manager-gui,admin-gui”/>

Use your own values for the Tomcat username and password. You will need to note and save these values as you will need to them to access the Tomcat manager web UI and also to setup Team Services to deploy with the Tomcat Deployment Task.

1.2: Restart the Tomcat7 and Apache2 servers:

sudo service tomcat restart

sudo service httpd restart

Each service should have reported OK for the above commands when they started successfully.

1.3: To verify you have Tomcat manager permissions configured, open a web browser to the your host’s URL but add “manager” to the end…

Example: http://raisatomcat.southcentralus.cloudapp.azure.com/manager

When prompted, enter the username and password you used when editing the /etc/tomcat/tomcat-users.xml file above.

The webpage should successfully display the Tomcat Web Application Manager page similar to what is partially shown below:

You can now install the Apace Tomcat Deployment task from on Team Services project to deploy to your Tomcat server:

Then create a build or release definition containing the Deploy Application to a Tomcat Server deployment task — you should not specify port 8080 as the task documentation mentions (you do not need to specify a port at all) and use the username and password from step 1.1 above (note: most applications have an Application Context of “/” which is different than what is shown below):

Option 2: Configure Tomcat Azure VM for Team Services Built-in Copy Files Over SSH Deployment Task

2.1: Edit the file /etc/ssh/sshd_config (e.g. sudo vi /etc/ssh/sshd_config )and ensure password authentication is disabled and set the default umaks to 002:

Change PasswordAuthentication from yes to no:

PasswordAuthentication no

and then add the following two lines at the end of the file and save the file:

Match User tcdeploy ForceCommand internal-sftp -u 002

2.2: Create an account (tcdeploy) on the server with the same group ID as the Tomcat service to enable easier deployments into the Tomcat webapps directory:

sudo adduser -g tomcat tcdeploy

sudo passwd tcdeploy

(select a password, and make a note of the account “tcdeploy” and the password)

2.3: Configure the default umask for SSH to enable RW for user and group

sudo vi /etc/pam.d/sshd

Add the following line at the end of the file then save the file:

session optional pam_umask.so umask=002

2.4: Then start the SSH daemon:

sudo systemctl daemon-reload

sudo systemctl start sshd.service

sudo systemctl enable sshd.service

2.5: Open Red Hat software firewall for port 22:

sudo firewall-cmd –zone=public –add-port=22/tcp –permanent

sudo firewall-cmd –reload

2.6: Create an RSA public and private key for SSH:

su tcdeploy (enter password you selected for the tcdeploy account above)

ssh-keygen (use defaults for all prompts except DO pick a passphrase and choose one that you will remember)

cd .ssh

cp id_rsa.pub authorized_keys

cat id_rsa

You will need to cut-n-paste this RSA private key to save later. You will need this private key and the passphrase when setting up your SSH endpoint in Team Services.

The key will look something like this:

—–BEGIN RSA PRIVATE KEY—– Proc-Type: 4,ENCRYPTED DEK-Info: AES-128-CBC,D4E587DECFADB218703A5431F7CFD6F7

PwkwPlFpm7JAT8Oc7pLpTzXEL9bl8LiDl8nnhuC9VZSOU58+cyR5m/e+6Odfx9eU RUyhInyFm2D9y9ud79GgCW9aUR2aH61L1TgoZjM1aNWlIYvDr+Eq63yNfZHvz1C/ plDkEPp0qOrmYdzL38hxG8c8y8TPjpqSxeITVu9Ql+S5Jz/BodNQPvqOnR//3SgI E3ur2XxSkgvLg02k4XS8q4J3jJksZ5SW+qvkdpeGIl1GwBeHBlqBs9xjcRoZhdOZ 6/QSor0RkbKaUUCQ9cLouSkIKCqVzDqecjOsIC4QdcXI6jP1MBQSdmsRylIa1KXC htxyBpB4sQc+wnEBiMy7dzPC9mBKb5Yc816jPTS54kfUZgvOsJNpe/5GMDjyV5r8 MjNkyguEFC35fgNeTBXZFI/JumQcWRt8A8myFv73ETUKRZ9//tICclTO3uE6ZJD+ K6KohoYWRvj3dMrPPu/PP2091ygwO2BRO4vEC3qIoIis4tR0X3jM/HOtzARDXdVG 6k13CSQDq7zBj4zmHu4XVoaIA1mFpYuqYOR5hRMDhV5UTIEiKKDCh+YREaD6EEFE lB6cy4tuLnbcOdj7xphXkCh+tWUD+MVAPmBk/EcDH1efKl8Iv4RK2UYRJyQi5nx/ bhU621WlO8jQTs9yHWjkc4/eJIzCPvmfZDFYpF+vSDQz9U/X/XhFfJQs0JCIBQgU /ZLIdIrQiSylVsviQmQKfxLtO28YxvVYWZZwqVWwnviDF3XCWCin6FNSGOXRzchS cvTp+Srp5i3ON0I+ZlVI4JW7EccrjTQa+ZK/1p1s06JhBbt8dlL8rgbbGBeJ/L3w o878ZRq+LRjlCCwwS4xhSQJkOtd9sJ/kzfOB+UIEkp6nGUmTjp9uC+Ca5T6EYn6Z a7AZyZrkW9gM0CFiBG/aVTnE70bcTKxLwi4CppUGddyVBjY+7LcD9fPEtqujxr6z f+ARCuJpwlRa7PVdfKrOHOHppZfbnRObWFu7qjhIan02hm3lUiZ4ndVThcjsXVD+ c/XEWVbqd1w1tGzJ2FJC8yLxCb4cY7H0Uoi8/iq0vLs912Wi8Fe6J0f4/3Mh1c/y YgvIUNZi8keJhxONgBjKCRDrtn6LB//VmBYfLlTY6G9rMoPrjDM4Q5DA60X7nUbI FyieUIV46KZhk1RrPKtvF8NhJhZcHHTlbd1yUCKRR58ygMwBuKtwvhTjvf/dkm/6 taBED4hhjPCJ/1fF0zcYmcbAMzB6wY1+EHuUgPyOouvZvagTipJBe6o/6R1gSCIE glNXmTozK4Gz1D42JnNog2O3p/5NdCZ6vdiwQLNFHOM38z9OM6l0SXuxgEXBImoC tmRMbuM8ag+PtN8LnZGDDgv4ez7B1UQ5b0V7aZ6Dk5biU6YrGLAD6d/7BN5MWgSL +ZoF7SUYJF/E8FUy0i6ZLtMHnLEnlroCOH0D4p/yJuCbgTKyrnn+taW6lNNKxaFT xERRRIaV+bdo07NkHpFTBYNy6biTR5Mht1uZmBZbDqDhCcP1uCJCTUUqSh5tMSNi AHajG40V+sJjDCvlOmbvd0rTK8XRMn4WAEnbHWoZx93GuZNMhQdLfsDRPKZshpTV —–END RSA PRIVATE KEY—–

Your VM is now setup and configured to enable deployment using the Team Services Copy Files Over SSH built-in deployment task.

To learn more about the Copy Files Over SSH task, read the announcement blog and the how-to video.

To use Copy Files over SSH with this server in Team Services, first, setup an SSH endpoint (using the SSH endpoint option) – make sure to use User Name “tcdeploy” (from step 2.2 above), the passphrase you chose in step 2.6 above, and cut-n-paste the entire RSA private key you generated in step 2.6 above into the Private key field:

Then create a build or release definition containing the Copy Files over SSH deployment task – make sure the Target folder is “/var/lib/tomcat/webapps” to deploy to the Tomcat server:

Option 3: Configure Tomcat Azure VM for Team Services Built-in FTP (ftps) Upload Utility Task

3.0: ONLY IF you are proceeding here as a continuation from Option 2, make sure you have “exited” the shell for the tcdeploy account login:

exit

whoami

The output of the whoami command should indicate you are the Username you initially selected when you setup the VM (not ‘tcdeploy’).

SKIP this next step IF you already created the tcdeploy account as the first step in Option 2 (i.e. Step 2.2) above…

3.1: Create an account (tcdeploy) on the server with the same group ID as the Tomcat service to enable easier deployments into the Tomcat webapps directory:

sudo adduser -g tomcat tcdeploy

sudo passwd tcdeploy

(select a password, and make a note of the account “tcdeploy” and the password)

You have now configured “tcdeploy” to be your ftps username and the password you entered in this step as your ftps password.

3.2: Install the vsftp package (i.e. FTP) and create some needed directories:

sudo yum install -y vsftpd

cd /etc/vsftpd

sudo mv vsftpd.conf vsftpd.orig.conf

sudo mkdir /var/run/vsftpd

sudo mkdir /var/run/vsftpd/empty

3.3: Generate an SSL self-signed certificate:

sudo mkdir /etc/ssl/private

sudo openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout /etc/ssl/private/vsftpd.pem -out /etc/ssl/private/vsftpd.pem

(answer all relevant questions when prompted as to your organization, name, etc.)

You will use the new SSL certificate filename you just created in two locations (shown in red) below when setting up the configuration file.

3.4: Create the new vsftpd configuration file /etc/vsftpd/vsftpd.conf to ensure the following parameters are set in the file (e.g. sudo vi /etc/vsftpd/vsftpd.conf):

listen=NO

listen_ipv6=YES

anonymous_enable=NO

local_enable=YES

write_enable=YES

local_umask=002 # this is different than the default 022

dirmessage_enable=YES

use_localtime=YES

xferlog_enable=YES

connect_from_port_20=YES

xferlog_file=/var/log/vsftpd.log

ls_recurse_enable=YES

secure_chroot_dir=/var/run/vsftpd/empty

pam_service_name=vsftpd

# ftps/ssl specific cofig stuff below this line

rsa_cert_file=/etc/ssl/private/vsftpd.pem

rsa_private_key_file=/etc/ssl/private/vsftpd.pem

ssl_enable=YES

allow_anon_ssl=NO

force_local_data_ssl=YES

force_local_logins_ssl=YES

ssl_tlsv1=YES

ssl_sslv2=NO

ssl_sslv3=NO

require_ssl_reuse=NO

ssl_ciphers=HIGH

debug_ssl=YES

pasv_enable=YES

pasv_address=168.61.54.109

pasv_min_port=13450

pasv_max_port=13454

Make sure to use the path/name of the self-signed certificate you created in step 3.3 above on the rsa_cert_file and ras_private_key_file lines in this config file.

Make sure to use the public IP address of your VM on the pasv_address line in the config file.

3.5: Restart the ftp service:

sudo service vsftpd restart

sudo systemctl enable vsftpd

3.6 Configure SELinux to use Linux ACL’s for file protection

sudo setsebool -P allow_ftpd_full_access 1

3.6: Open ports 13450-13454 on the Azure VM using a similar process to that used above as the final step (i.e. Step 0.13) in setting up Tomcat on your server.

Name the inbound security rule “default-allow-ftps” and set the allowed Destination port range to be 13450-13454 and specify “Any” protocol.

3.7: Open port 21 as well using the inbound security rule name “default-allow-ftp” and set the Protocol to “TCP” and the Destination port range to 21.

3.8: Open the ftp ports on the Red Hat software firewall:

sudo firewall-cmd –zone=public –add-port=21/tcp –permanent

sudo firewall-cmd –zone=public –add-port=13450/tcp –permanent

sudo firewall-cmd –zone=public –add-port=13451/tcp –permanent

sudo firewall-cmd –zone=public –add-port=13452/tcp –permanent

sudo firewall-cmd –zone=public –add-port=13453/tcp –permanent

sudo firewall-cmd –zone=public –add-port=13454/tcp –permanent

sudo firewall-cmd –reload

3.9: To verify your system is configured correctly, use a tools such as FileZilla to connect to your server using ftps. You would specify the Host name of your VM, the username of “tcdeploy” and the password you used when creating the tcdeploy account above in the first step in this section.

Your VM is now setup and configured to enable deployment using the Team Services FTP Upload built-in utility task.

To learn more about the FTP Upload task, read the announcement blog and the how-to video.

To use ftps with this server in Team Services, first, setup an sftp endpoint (using the Generic endpoint option) – make sure to prefix the Server URL to your server with “ftps” to use secure ftp:

Then create a build or release definition containing the FTP Upload utility task – make sure the Remote directory is “/var/lib/tomcat/webapps” to deploy to the Tomcat server and also that the “Trust server certificate” Advanced option is selected so that your self-signed certificate (from step 3.3 above) will be trusted: